Nota Bene Leaving the slight "step-down" going out into the manifold will actually help to prevent exhaust gas reversion into the chamber. Exhaust gas reversion has a negative effect on scavenging, and increases pumping losses (and therefore decreases power output slightly.) You're better off leaving that alone, unless you're going to fit "anti-reversion baffles" to the exhaust primaries. (I've done enough research on this topic to change my mind - I used to think the way you do.)
Realistically, you're better off leaving that step in place than you are trying to "ramp it down." Cleaning up the sand-cast surface will help, but I wouldn't bother trying to remove the "step down." If it was a "step up" (the primary was smaller than the port exit,) I'd really want to do something about it - since that would create significant reversion and pumping losses, and just kill scavenging. But, having the step there helps to create an extra boost of low pressure behind the exhaust pulse, which helps to draw more exhaust gas out of the cylinder and draw more fuel/air charge into the cylinder - which reduces pumping losses!
Short form? Leave it alone. Long form? If you've read this far, you know what's going on and why...