The celebration will occur in Martha's Vineyard in Massachusetts on Saturday. Obama turned 60 on Wednesday.
Obama and former first lady Michelle Obama originally planned a star-studded party with attendees that were to include actor George Clooney, filmmaker Steven Spielberg and talk show host Oprah Winfrey.
Massachusetts Gov. Charlie Baker, however, said such a gathering is "not a good idea" now that coronavirus cases are going back up, due mainly to the more contagious Delta variant.
"Due to the new spread of the Delta variant over the past week, the president and Mrs. Obama have decided to significantly scale back the event to include only family and close friends," Obama spokeswoman Hannah Hankins told The New York Times.
"He's appreciative of others sending their birthday wishes from afar and looks forward to seeing people soon."
Dukes County, which includes Martha's Vineyard, has been identified by the Centers for Disease Control and Prevention as a "moderate" risk community. Last month, Martha's Vineyard issued an indoor mask advisory.
"It's stopped short of a mandate," Tisbury health official Maura Valley told the Vineyard Gazette.
"We're saying, with the Delta variant, and the uptick in cases, we're strongly advising that you wear a mask."