Padgett's father played for the University of Nevada, his uncle played for the University of New Mexico, his grandfather played for Oregon State, and his sister played for the University of San Diego.
Padgett attended Reno High School, where he averaged 27 points and 14 rebounds per game his senior year. He was a McDonald's High School and Parade All-American. He also was a member of the 2004 USA Basketball Junior World Championship Qualifying Team, earning a gold medal at the event.
As a high school senior, he was the top-rated center and considered the fourth-rated prospect overall by Inside Hoops, the seventh overall by Rivals Hoops, and fifteenth overall by ESPN.com. In 2003, he was the Nevada player of the year.