The move was announced by Bills coach Gregg Williams, who went 3-13 in his first season.
"I just thought we needed to see if there was someone else out there who could improve the offense," Williams said.
The Bills scored 265 points this season, better than only Cincinnati in the AFC. The firing coincided with the placing of Johnson on the unprotected list for next month's expansion draft.
Sheppard was hired by Williams last February as Bills' offensive coordinator and installed the West Coast offense after spending the previous two seasons as the quarterbacks coach of the Seattle Seahawks under Mike Holmgren.
But the new offense, based on short drops and quick reads by the quarterback, did not help Johnson, who could not shake his propensity for holding on to the ball too long and taking numerous sacks. He suffered a broken collarbone Nov. 11 at New England and missed the rest of the season.
In 216 pass attempts, Johnson was sacked 31 times and threw for 1,465 yards and five touchdowns with seven interceptions.
Backup Alex Van Pelt had more success in the system, throwing for 2,037 yards and 12 touchdowns with 10 interceptions. He was sacked just 13 times in 300 attempts.