The Raiders suffered the loss of quarterback Jason Campbell to a shoulder injury in the second quarter, but came up with two big plays to seize the lead and then held on to win their second in a row.
Oakland is 4-2 for the season and Cleveland dropped to 2-3 with its second consecutive loss.
Campbell was hurt while stretching for a first down as linebacker Scott Fujita was falling on top of him. Kyle Boller came on and completed 8-of-14 passes for 100 yards.
After Smith's touchdown, Ford fielded the kickoff a yard deep in his end zone and ran it all the way back to put the Raiders in front for good.
Oakland extended its lead early in the third period on a 48-yard field goal from Sebastian Janikowski and the Raiders' kicker lined up to try another one with the ball on the Cleveland 35-yard line.
Punter Shane Lechler, who holds for field goal attempts, took the snap and then threw the ball to Kevin Boss for what turned out to be a touchdown that gave Oakland a 24-7 lead with 1:23 left in the third.
It was the first NFL pass for Lechler, who was a high school quarterback.
Cleveland rallied for 10 fourth-quarter points and threw a scare into the Raiders by scoring on McCoy's 12-yard pass to Mohamed Massaquoi and then recovering an onside kick with a minute to go.
The Browns, however, were unable to pick up a first down and turned the ball over.