NBC News and CNN projected Mourdock, Indiana's state treasurer, had derailed Lugar's attempt to win a seventh term. Results posted on the Indiana secretary of state's Web site gave Mourdock 134,152 votes to 94,905 for Lugar.
U.S. Sen. John Cornyn of Texas, the National Republican Senatorial Committee chairman, said Lugar's defeat in the primary "will probably make it more of a contest" in the general election "but I'm confident we'll hold the seat."
Mourdock will face off against Democratic Rep. Joe Donnelly in November.
The White House issued a statement by President Barack Obama expressing "deep appreciation for Dick Lugar's distinguished service in the United States Senate."
"While Dick and I didn't always agree on everything, I found during my time in the Senate that he was often willing to reach across the aisle and get things done," Obama said. "My administration's efforts to secure the world's most dangerous weapons has been based on the work that Senator Lugar began, as well as the bipartisan cooperation we forged during my first overseas trip as Senator to Russia, Ukraine and Azerbaijan."
Obama said Lugar came "from a tradition of strong, bipartisan leadership on national security that helped us prevail in the Cold War and sustain American leadership ever since. He has served his constituents and his country well, and I wish him all the best in his future endeavors."
Mourdock was backed by national conservative groups, including the Club for Growth, the National Rifle Association and FreedomWorks, an organization that has helped build the Tea Party.
Mourdock has painted Lugar, 80, as too moderate, too friendly with President Barack Obama, too removed from his home state and too old.
Mourdock's characterization of Lugar as too removed from Indiana was brought into sharp relief in March when Lugar temporarily had his voter registration revoked because he hadn't lived in Indiana for 35 years. He moved to a Washington suburb after getting elected in 1976.
Lugar had difficulty explaining to a local TV reporter what address he uses for his driver's license, The Washington Post said.
Eventually, the state attorney general ruled Lugar could register at the 600-acre farm he and his family run, so he became eligible to vote in his own primary election.
Lugar is the first incumbent senator defeated in a primary battle this election year.