Although the couple took their romance public by walking the red carpet Monday night at a Speedo promotional event in London, UsMagazine.com said Phelps declined to answer questions about their relationship.
Phelps, 27, has earned 22 medals, 18 of them gold, during the past three Olympics.
Rossee, 25, has appeared in runway shows for fashion designers Michael Kors and Betsey Johnson.
Ray Liotta sues skin care company over use of likeness
Boston schools pull out free condoms over wrapping complaints