If you find yourself switching Spaces frequently and use a laptop with multi-touch capability, you can use MultiClutch to map gestures to Spaces commands.

First (and the order is important) disable any keyboard shortcuts you use for Spaces. Then go into MultiClutch and add (in the Global application) Swipe gestures to the key commands you want for Spaces up/down/left/right. Finally, set those key commands in the Exposé and Spaces pane. This lets you "sweep away" your old space with a single gesture (no finger juggling), kind of Minority Report-style.

One caveat: although this overrides most applications' built-in three-finger gestures, it doesn't override the Finder's back/forward support, so you get stuck sometimes when a Finder window is in the foreground. Just choose another app, and you're on your way.