The operation was launched early in the morning
"It is a final push to clear the place of armed militants," said military spokesman Major General Waheed Arshad.
The army said at 20 militants were killed in the operation, as loud explosions and gunfire were heard.
Twenty children escaped from the mosque, where women are also being held. Three soldiers are reported killed and some 20 others injured.
The military operation began at about 0400 (2300 GMT Monday).
The troops entered the compound and exchanged fire with the militants holed up inside.
The army said 20 militants were killed and another 15 injured.
Local medical officials said 20 government troops were injured, and three had died of their wounds after being taken to local hospitals.
The army says it has taken over the building's roof.
However, it is meeting particularly strong resistance from militants in the basement, reports the BBC's Syed Shoaib Hasan at the scene.
Those inside the mosque are using hand grenades, light machine guns, petrol bombs and other weapons, and the army says it expects the operation will last another four hours.
Ambulances are waiting nearby to help any wounded.
It is not clear exactly how many people were left inside the mosque when the assault began.