"My name's Joe Biden, I'm a Democrat, and I loved John McCain," Biden said, pointing to the partisan divide between the two former colleagues in the U.S. Senate.
"I always thought of John as a brother," he added, "We had a hell of a lot of family fights."
A procession by the Arizona National Guard carried McCain's flag-draped casket from the Arizona State Capitol -- where he was lying in state Wednesday -- to North Phoenix Baptist Church for the memorial service.
"The image I carry with me of Beau is strong, vibrant -- the best man his brother Hunter and I ever knew. I'm sure Vicky [Kennedy] has her image of Teddy -- maybe looking so alive on that beautiful sailboat of his. Find your image of John, remember it -- his smile, his laugh, a moment on the ranch, at a dinner, on a vacation, when you would turn and see him just staring at you, or when you saw the sheer joy that crossed his face the moment he knew he was about to take the stage or go to the floor of the Senate -- and start a fight. God, he loved it."
"John's story is the American story."
Earlier in the day, Biden encouraged local supporters to come out and pay respects along the motorcade route.
"If you are in town and would like to honor his legacy of service, please come out to the route to say one last farewell," Biden wrote on Twitter.
McCain's family and friends, along with state and local officials, business and civic leaders were invited to attend the service. In addition, about 1,000 seats were made available to the public.
After the service, there will be a small private ceremony at Sky Harbor Airport, witnessed by National Guard members, as McCain departs Arizona for the last time.
McCain's body will be flown to Washington, where he will lie in state Friday in the Capitol Rotunda. Family, colleagues and the public can honor his life and service in a ceremony there at 11 a.m. ET.
House Speaker Paul Ryan, Senate Republican leader Mitch McConnell, Vice President Mike Pence and Senate Democratic leader Chuck Schumer will present wreaths during the service. Members of the public will be allowed to pay their respects Friday from 2 to 8 p.m. EDT.
McCain, a Republican who served in Congress for more than 30 years, will be the 13th senator to lie in state at the Rotunda, an honor reserved for the nation's "most eminent citizens," according to the Architect of the Capitol.
On Saturday, McCain's body will be carried from the U.S. Capitol by Armed Forces Body Bearers, secured and moved by motorcade to the Washington National Cathedral, where a private funeral will take place. The motorcade will pause at the Vietnam Veterans Memorial, where his wife, Cindy McCain, will lay a ceremonial wreath honoring those who died during the Vietnam War.
McCain was a prisoner of war for seven years during the war and visited the memorial often to speak with other veterans.
McCain's office said the public is welcome to line the procession route.
McCain will be buried Sunday at the U.S. Naval Academy in Annapolis, Md.