New Orleans police said passersby saw two workmen put up a screen and start to use power tools to try to remove the section of a wall containing the 2008 mural, known locally as the "Rain Girl."
The men reportedly told onlookers they were removing the mural for display at a Banksy exhibit at the Tate Modern art gallery in London, but as more people questioned their story, they eventually gave up and drove off in a white rental truck.
A lawyer for the building confirmed that no authorization had been given to remove the mural.
Police said they were searching for two white men, 25 to 35 years old, one of whom was 6 feet tall and the other about 5 foot 9. A local photographer, Charlie Varley, was at the scene and snapped photos of the men attempting to remove the section of the wall.
"Umbrella Girl" or "Rain Girl," which mysteriously appeared in the Marigny neighborhood of New Orleans in 2008, is said to be a commemoration of Hurricane Katrina, which killed more than 1,800 people in 2005.
A guard remained stationed at the building through the weekend and into this week.