The British news network said it has learned the additional troops are a backup plan should the private security contractor G4S be unable to gather enough trained staff by the time the games begin late this month.
The armed forces are already scheduled to provide about 13,500 personnel. Putting 3,500 more troops on standby would mean some will likely have their summer leave canceled.
The BBC said it understands G4S has not been able to guarantee it can supply the 10,000 guards it has been contracted to deliver. A G4S spokeswoman said the contractor had accepted "that the government has decided to overlay additional resources," the BBC said.
"This has been an unprecedented and very complex security recruitment, training and deployment exercise, which has been carried out to a tight timescale," she said.
"We have encountered some issues in relation to workforce supply and scheduling over the last couple of weeks, but are resolving these every day and remain committed to providing a security workforce for the start of the London 2012 Games."