Police said the attack occurred at the student housing facility off-campus from the Federal Polytechnic campus, the BBC reported.
A school lecturer told the BBC 40 students were killed, but a police source declined to comment on that number.
The killings occurred several days after a major operation against Boko Haram militants in the community.
The lecturer, speaking anonymously, told the BBC students were asked to recite their names after being lined up. He said the gunmen killed some students and spared others.
"Everybody is scared and staying indoors now," he said.
People who were trying to leave were placing tree branches on their vehicles, a traditional sign of neutrality in Nigeria, the BBC said.
Boko Haram militants are fighting to establish Shariah law in Nigeria.