grinding noise

I recently replaced my starter, when I was done I went to drive off and I heared this aweful grinding noise when I accelerated from a stop but not in any other gear, So I took it out and put my old one back in.....same problem???????????
It only happens when I take off fast from a stop, if I'm gentle with the clutch there is no problem.

Thanks
Peter