It's probably not your torque. Are your retainer plates in one piece, or did you cut them in half so you wouldn't have to get a new bearing/seal pressed on to begin with?
If they are halves, I probably know what your problem is. If they are wholes, I'm not so sure unless your seal was damaged...