The "Brokeback Mountain" and "I'm Not There" star died Jan. 22 in New York at age 28.
The cause of his death has not yet been determined. The New York Medical Examiner's Office said it expects to make an announcement this week after it has the results of toxicology tests.
Ledger's father Kim, his mother, Sally, and his older sister, Kate, were among 15 family members who flew out of Los Angeles Sunday night, Australia's Herald Sun reported.
They left the United States a day after a memorial was held at Sony Pictures.
Ledger's ex-girlfriend Naomi Watts, models Gemma Ward and Lily Cole, musician Ben Harper, actors Sean Penn, Josh Hartnett, Orlando Bloom, Portia de Rossi, Tom Cruise, Katie Holmes, Sienna Miller, and director Todd Haynes all reportedly attended.
The Herald Sun said it is believed the Ledgers will hold a private funeral and then bury Heath in a family plot where two of his grandparents are interred.
Jordana Brewster on Paul Walker: 'He was an enormous presence in my life'
Benedict Cumberbatch's dramatic reading of R. Kelly lyrics is just what you need