should I rebuild..or can I clean

After looking down into the lifter area of my 4.0 I noticed lots of burnt oil and mess between them where there are no drains back to the bottom. I made myself a long "spoon" to try to get some of it out but of course it won't all come out. I'm wondering it what's left could float around and stop up a lifter or pushrod and prevent the engine from oiling properly.
Is there any way to clean it? Maybe put oil back in the engine and roll it over with the starter to flush out the lifters?

Any advice at this point is greatly appreciated.

You are correct to be cautious, I hosed a 2.8L that way when the the pickup plugged at 70mph on the interstate after I cleaned the sludge out of the top end.
There are a couple of ways to do it, first is with an engine flush and oil change, this works the fastest but can leave a lot of flush soaked sludge in there that will break down between then and the next oil change. Some even use a quart of ATF or Kerosene as a flush, not my cup of tea.
Second method now that you have the cover off is to take a shop vac in one hand an a plastic scraping tool in the other and use them together, as you break the crud loose the vac wil suck the pieces up.
Third method is slower and more gentle, either use a full oil change of Mobil-1 or Castrol Syntec, both are very high detergent and will break down the sludge as you use them, couple that with keeping an eye on the dipstick so that when you can no longer read the hash marks [black oil] on the dipstick you can change the filter and add a quart. Second method and the third method work well together.
Fourth method is to take it to a shop that has a flushing machine where they pump solvent in thru an oil filter adapter with the engine running. It does a good job but again leaves a residue of cleaner that will have soaked into the heavy sludge, if you do that you really need to keep an eye on the oil color and change the filter. If you don't the filter will plug up and a bypass will kick in, this bypass will cycle unfiltered engine oil until you change the filter again.
Just for laughs I dropped my Mobil-1 for it's normal oil change and refilled with some mobil 10w30 normal oil and drove it 3 miles to my friends shop, he put it on the flushing machine with a new filter and ran it, came out clean, the filter was a very very light oil colored tan. I then refilled and put my usual Mobil-1 and Mobil-1 filter on [or K&N oil filter, whichever I can get] My 98XJ has over 192,000mi on it.
I run Mobil-1, after the 5K break in when new, always have and I can see how clean it is around the filler area. My wifes 97 Olds blew a water pump and overheated, warped the motor so we had to get a new rebuilt Jasper, when our mechanic pulled the valve covers on that V6 the valve train still looked new and it had over 160,000mi on it and like my XJ had only been run on Mobil-1 after break in.

Thanks Rich,
I finally decided to call the shop that I had rebuild an engine for me a couple of years back. He suggested putting some Marvel Mystery Oil in, and that from what I described it wasn't near as bad as what he's seen. He too mentioned what you did, that the pickup screen could get blocked. I considered doing the vacuum thing and may still do it to try and get some of the gunk out of the lifters.
I guess when I put the head back on I'll fill up with some "affordable" oils and run it for a few miles and change it. But like you said I'll try to keep an eye on it.

I did not realise you had the head off, I thought you just had the valve cover off. Heck, put the head in a big pan and dump a gallon of Gunk engine cleaner in there with it and go to town. That should clean it up nicely. You could also, with the head off, pour cleaner down the return gallerys in the block and it will drain right out of the oil pan or let it collect down there and then drain it, that should remove any sludge in the pan itself then just let it drip itself dry over a day or two. You could even remove the lifters, mark them first or put them in a metal lifter holder, then soak them in cleaner. just use some assembly lube when you put them back. They need to go into the same holes they came out if and they need to be oriented in the same direction. One guy I know heats up kerosene in a microwave and pours it in hot, says it works better, dunno, my wife gets a bit testy if she sees me looking thru kitchen drawers if I'm working on our jeeps ever since the 'baster affair', I was just using it to top off the batteries with clean distilled water.... Nukin kero, not me, the shed is too full to open up a cot

Yeah, the lifter area is what concerned me. I poured some Marvel's into it to let it soak a while.
Don't think I'd trust warming up kerosene in a microwave, that just don't sound to good.
Not too sure I could get those lifters out either that's a small space to get into.

Something about heating a fuel in a microwave just doesn't sound like a great idea to me... A couple of metal shavings in it would really ruin your day.