The measure, signed Friday, was opposed by gun-rights activists.
The bill's author, Democratic Assemblyman Anthony Portantino, said the goal is to prevent potential shootouts on the street. He said the ban would avoid situations in which police officers had to approach someone toting a possibly loaded shotgun or rifle in a public place.
The Los Angeles Times said the bill was inspired in part by recent actions by gun-rights advocates who entered coffee shops and other places carrying rifles as a demonstration of their right to bear arms.
Brown also signed a bill allowing movie and television producers to use working sawed-off shotguns as props while filming in the state. Such weapons are banned in California.