According to the creator of Fate and to the Fate/Zero Light Novel, if Tokiomi hadn't called Gilgamesh back he would have annihilated Berserker along with every other person in the vicinity. Berserker was barely defending against 16 portals. When Tokiomi called him back, he'd opened 32.
The one time Gil got his head out of his ass to fight seriously, he wasted it
From the Light Novel:
"In anger, Archer howled at Berserker, his eyes burning crimson. For the third time, the space around him warped to let a herd of blades materialize...
The next count of glowing Noble Phantasms reached thirty-two. This time, even Rider kept silent. Berserker had endured a continuous attack of sixteen Noble Phantasms, but there was no way to resist twice that number. That was the same for all the other Servants. Nobody could estimate the limits of the latent power of the golden Archer anymore."