This question came up recently during a Twitter discussion with @freehit_mj. I assumed it was universally accepted that being a cricket umpire is among the toughest — if not the most difficult — referee job in global sports today. The reason I say that is simple: 1) Cricket is an unbelievably complex game, and its rules and regulations are notorious to outsiders for their specificity and exceptions. For a good recent example, head over to Deep Backward Point for a lesson on why batsmen can’t hit the ball twice (except to protect their stumps). And 2) I can’t think of another popular game that comes close to this level of nuance. In tennis, you have a crew of linesmen to help you adjudicate calls. In soccer, the hardest thing is the off-side rule, and even then, you have help. (Soccer and other sports are certainly more physically demanding, but that’s a different matter.)
Freehit_MJ suggests that contact sports may be more difficult. I’m not sure why. I confess I have only a vague idea of what American football referees do, but it can’t be that hard if the most disputed call is whether a foul occurred or not. I also don’t think these questions raise to the level of importance of appeals in cricket. Losing a wicket is tremendously important to a batting side, whereas a foul in soccer/basketball/etc. is only a big deal in certain situations. I think cricket accords umpires so much protection from dissent precisely because we understand how difficult the job is. For more that, go here.
Ducking Beamers: A Cricket Blog 
One thing I’d say is that cricket’s laws make it a yes/no answer to an appeal. “Is that out?” “Yes”. The decision comes down to something like ‘would it have hit the stumps’, or ‘did he hit it’. (I’m not discounting how hard these decisions are).
The thing that makes a sport like football(soccer) hard to referee is the level of grey area. It may be a foul of some sort, but *how bad* was it? “Was it careless, reckless, or intentional? Is it worthy of a free kick, yellow card, red card, penalty” etc etc.
That’s a good point, Gareth. And it’s true; in questions of degrees, cricket isn’t all that accommodating. On the other hand, I still say the consequences of a wrong decision in soccer — i.e., whether a free kick or a yellow card should be handed out — aren’t all that great. Handing out a red card wrongly would be disastrous, of course, but I think that almost never happens (feel free to correct me if I’m wrong).
Depends who you support! 🙂
I’d say as a general rule you’re right. The consequences of getting it wrong while on the half way line are minimal (red cards excepted), but as football games are often decided by a small scoreline (1-0 etc), if you get it wrong at the goal mouth then the consequences are game changing.
For example, Ireland vs France World Cup qualification. : http://www.telegraph.co.uk/sport/football/teams/republic-of-ireland/6599687/Thierry-Henry-admits-to-handball-that-defeated-Ireland-in-World-Cup-play-off.html
There’s also plenty of occasions in cricket where the umpires decision has decided the game, but I think we’re probably used to it more.We accept that as the umpire has such a hard job, they’re going to get it wrong every now and then. The best umpires are the ones who get it wrong the least amount of times.
Having said that, as a NZ cricket follower, I’m fairly used to needing *everything* go right for us to win…. so every wrong decision feels like a kick in the nuts.