Francis Xavier was born in the family castle of Xavier in the Kingdom of Navarre, on 7 April 1506, according to a family register. He was born to an aristocratic family of Navarre, the youngest son of Juan de Jasso, privy counselor to King John III of Navarre (Jean d'Albret), and Maria de Azpilicueta y Xavier, sole heiress of two noble Navarrese families. Following the Basque surname custom of the time, he was named after his mother; his name is accurately written Francisco de Xavier (Latin Xaverius) rather than Francisco Xavier, as Xavier is originally a place name.

Joint Castilian and Aragonese troops commanded by Fadrique Álvarez de Toledo, 2nd Duke of Alba conquered the Kingdom of Navarre in 1512. After a failed French-Navarrese attempt to reconquer the kingdom in (1516), in which Saint Francis's brothers had taken part, the outer wall, the gates and two towers of the family castle were demolished, the moat was filled, the height of the keep was reduced in half, and land was confiscated. Only the family residence inside the castle was left.

Francis's father died when he was only 9 in 1515. Francis Xavier went to Venice, Italy and there he was ordained to the priesthood on June 24, 1537. After ordination, he served for a brief period in Rome.