Although the couple took their romance public by walking the red carpet Monday night at a Speedo promotional event in London, UsMagazine.com said Phelps declined to answer questions about their relationship.
Phelps, 27, has earned 22 medals, 18 of them gold, during the past three Olympics.
Rossee, 25, has appeared in runway shows for fashion designers Michael Kors and Betsey Johnson.
Martin Bashir resigns from MSNBC over Sarah Palin comments
Florida bear attack: Black bear mauls woman's face