Jackson told TheInsider.com her commitments to an upcoming summer tour prevent her from taking part in the singing competition.
She said she is "very flattered that 'X Factor' let me know that I was being considered for next season, but it just wouldn't be possible."
Representatives for Jackson, though, said Thursday "there have been absolutely no meetings between Janet and anyone with regard to 'X Factor,'" the New York Daily News reported Friday.
"I worked with Katy once. I think she would be fun because she is hard -- in a good way -- and she's feisty," he said. "She would be great."
Beautician charged with giving client fatal silicone butt injection
Handler slams Piers Morgan: 'You're a terrible interviewer'