Yes, you have to push the whole cross one way with one cap going in to push the other cap out, then push it the cap-less end of the cross back in and to the other side to push the other cap out. There isn't enough room in the yoke to do it any other way.
Even then, you may have to try yanking at the first cap with vise grips or something to get it out all the way, because the cross won't push it far enough just to drop out. If they're rusty, they can be mighty stubborn.
If you have a press, you can make the job simpler on a really galled and rusted joint by adding a step: Take the press, and push one cap out most of the way, but don't try to remove it if it's tight. Soak it with oil, and soak the hole in the yoke from the inside, and also soak the bore and the cap on the other side with oil, and then press the whole mess back together, and proceed to remove the cap you pushed in on the first try, which has now been freed up and oiled. The last pressing gets out the cap you almost got out on the first, and often you can just use a hammer and punch on the end of the cross without bothering to reset the press.