DON'T remove the male. The male will only eat the babies if they aren't his (and he doesn't know they aren't his unless you add them after they were born... mice aren't smart, if they have babies while he is in the tank he will think they are his, even if they aren't). By removing the male your colony will be counterproductive. The femaes are ready to breed once they have given birth, if you remove the male you cannot reintroduce him to the female until she has raised her babies and they have been weaned, at which point she could already be close to dropping another litter if the male had been left in. Also mice aresensitive, if you keep jumpling them around they will stop breeding and get stressed out. You would need to house all the mice individually if you chose to do it that way. You cannot introduce ANY new members to the colony if their are babies, their will be fights as well as babies may be eaten.
Some of the main causes of cannabilism in colonies are 1) females first litter 2) sick babies 3) too many disturbances 4) not enough food and/or water 5) new members added to the colony. You may also get the occassional repeat offender thats just a bad mom. Easy enough to fix