He then allegedly attempted to dispose of the remains by burning them in his home's fireplace Sunday, police said.
Smoke from the fire brought local firefighters to the house in Johnston, R.I., where the suspect, Donald Greenslit, attempted to prevent them from entering the home.
"He kept saying everything was all right, that he was just burning wood," Deputy Chief David M. DeCesare told the Providence (R.I.) Journal Monday.
Upon entering the house, firefighters found two children in a bedroom and took them to Hasbro Children's Hospital.
DeCesare said Greenslit, already on probation from a domestic assault case against the children's mother, the girlfriend he allegedly fatally stabbed, will be charged with domestic homicide.