It really sucks attacking a swarm if you're numerically disadvantaged but I've seen some amazing play by small commando team that pick apart unorganized swarms.
As for defender side, the Family hold and earn polymer labs against numerical disadvantages.
But I do agree, numbers advantages is huge and I understand that it sucks however, I personally wouldn't want to limit the dynamics of PvP by enforcing arbitrary restrictions. Perhaps better reward for the side that fought with less people ( provided they fought well ).
Unless your speaking from the perspective of the 50 and you had to wait in line behind 20 guys to even get a chance to see the enemy, I have no sympathy for you.