Cleveland had the best regular-season record in the NBA and with two-time reigning MVP LeBron James seeking his first league title, the Cavaliers were deemed the favorite going into the playoffs.
Boston, however, eliminated the Cavaliers in six games, winning the last three in a row including a 32-point triumph in Cleveland Tuesday.
The Celtics will now meet the Orlando Magic to determine the conference's representative in the NBA finals. The Los Angeles Lakers will face Phoenix in the Western Conference.
James, nursing an elbow injury and coming off the worst post-season game of his career, scored 27 points while making just 8-of-21 shots from the field and committing nine of Cleveland's 22 crippling turnovers.
The most important of those turnovers ignited the Boston run midway through the fourth.
James had hit consecutive 3-point shots to cut the Cleveland deficit from 10 to four points and the Cavaliers had a chance to draw closer with 8 minutes to play.
The Cleveland superstar, however, fumbled the ball away, Rondo scooped it up and he drove for a layup that gave Boston an 80-74 lead. Paul Pierce followed with a 3-pointer, as did Rasheed Wallace, and Kevin Garnett converted a fast-break dunk off another turnover with 5:50 to go that made it 88-74 and all but clinched the victory.
Garnett led the Celtics with 22 points, Rondo had 21 and both Pierce and Wallace had 13. Cleveland shot just 37 percent from the field.