The image shows the 7-year-old with a weapon over his shoulder holding the head of a Syrian soldier by the hair with the caption, "That's my boy!"
Sharrouf escaped to Syria last year using his brother's passport. Once he arrived, he joined forces with the Islamic State (IS) -- the radical militant group that has taken control of one-third of Syria and large parts of Iraq.
IS has been beheading Syrians for blasphemy and crucifying others for abandoning their religious principles.
"There are more photographs in newspapers in Australia today of the kind of hideous atrocities that this group is capable of," Abbott told ABC radio. "This does pose extraordinary problems -- extraordinary problems, not just for the people of the Middle East, but for the wider world and we see more and more evidence of just how barbaric this particular entity is."
Sharrouf's Twitter account was shut down after he posted multiple pictures of severed heads and his young children holding weapons.