Barbosa scored 19 points off the bench while Washington faded over the final quarter. The Wizards have lost three straight and have only four victories in their 23 games this season.
Washington reduced an 11-point halftime deficit to just one early in the third quarter, but the Raptors rebuilt a 12-point cushion heading into the fourth period and the Wizards could not put together another run at Toronto.
Amir Johnson added 18 points and 13 rebounds for the Raptors, who have lost twice as many games as they have won this season.
Nick Young led Washington with 21.
The Wizards shot just 37 percent from the field and were outrebounded by a staggering 61-37. Toronto claimed 23 offensive rebounds.