Actually I think that perhaps the mission to get Han back was done due to several factors (other than the obvious desire by Luke, Leia, Lando and Chewie to save their beloved friend Han Solo)
1) They didn't want to get the Empire's attention with a large strike force.
I say this because Jabba is said to have a LARGE PERSONAL ARMY. Granted they probably aren't as well trained and organized as the Imperial army, being composed of irregular troops, mercenaries and bounty hunters alongside his personal entourage, but still. He's got a fortified palace and bunker for him to hide in with plenty of supplies, in addition to his own land and air ships. In the prequels we learn that he's got lots of money AND that the entire planet is controlled by the Hutts (maybe even the whole system for all we know). I suspect (but don't know for sure) if the Empire has ANY influence at all on Tatooine in the time of the Empire (they seem to tolerate crimelords like Jabba at the very least, rather than executing him and placing an Imperial governor in his place or something).
The Rebels would have had to commence orbital bombardments of key installations or blockade the planet until Jabba surrender's Han (a la Bin Laden in Afgahnistan as was planned) or else send in a massive invasion force to get him out. I doubt a simple squad of Rebel commandos could do it alone.
So the key was inflitration, with a major trump card: their Jedi Knight, whom Jabba had nothing but scorn for, at a time when Jedi were thought to be extinct and powerless.
2) Luke was probably powerful enough to handle it on his own, though we the audience didn't know it at this point yet.
3) Luke probably (this is my theory) "foresaw in the Force" that this was the "best" way to do it (ie: so all his friends survive and Jabba is punished for his crimes). Very Paul Muad'dib-esque. ; )
4) The Alliance was probably unwilling to give away their presence to the Empire and Bring a large force to bear (probably couldn't spare it) to rescue this one "unimportant ex-smuggler" despite his hero status being instrumental in the destruction of the Death Star which saved the Rebellion from extinction.
Then again, saving Han Solo would have large propaganda value for the Alliance, not to mention freeing Tatooine from the grip of a slimey crimelord like Jabba.
Then again we don't know Jabba's actual status. Is he a major important crimelord on the planet? Is he in charge of the whole of Tatooine or the entire system? Is he simply a part of the ruling class/family/tribe? Or is his somebody's crazy uncle that just got his own palace and occasionaly makes public appearances to make money? In anycase, few people will miss a guy like that outside of his own crimelord circle (and hey, a bigger piece of the pie for them.. the ones who aren't scared they will be next of course).
Fixing the Plot Holes in ROTJ
Moderator: Vympel