Child protective services took custody of the children in 2009 when the children were found unattended and living in squalid conditions, The Local.se reported Wednesday.
A disabled 8-year-old girl was found tied to a chair, and a 5-year-old boy spent hours cycling around his neighborhood in Halmstad, officials said. Their then-1-year-old sister was found sitting in her own waste.
Because the youngest child has a Swedish father, she is being allowed to stay in Sweden.
Appeals by child services officials were rejected by the Migration Board, which said it wasn't its job to determine if the mother was fit to care for the older children.
However, board officials said Tuesday they would check whether there were adequate support services for the children in Serbia.