Boko Haram aims to set up an Islamic state in Nigeria, which is divided along Muslim and Christian lines.
Human Rights Watch said a review of satellite imagery shows at least 2,200 buildings were destroyed after Nigerian forces launched a mid-April operation against Boko Haram in the northern town of Baga.
Residents of the city told the rights organization the campaign against Boko Haram followed a militant attack that left one soldier dead. Baga residents told the Human Rights Watch the military destroyed the town.
Director of African affairs at Human Rights Watch Daniel Bekele said the Nigerian military has a right to protect itself and the Nigerian population from militants but evidence suggests the response was uneven.
"The glaring discrepancies between the facts on the ground and statements by senior military officials raise concerns that they tried to cover up military abuses," he said in a statement from South Africa.
The Nigerian military said about 30 houses were destroyed. Human Rights Watch said most of the buildings its seen in the satellite images were "likely" residences.
The Nigerian military has since limited access to the area, the rights group said.
Accounts from Human Rights Watch correspond with an April report from CNN.