When you're viewing all of your Spaces (F8), you can press the C key to collect all the windows from all spaces into the first space. Press C again to send them back where they started.

If you collect windows and then exit out of Spaces (by pressing F8 again), you will not be able to uncollect them.

[robg adds: I couldn't find any reference to this feature in Spaces' help, and it's quite useful if you've been tossing your windows into various spaces throughout the day, to the point where you sort of lose track of where you've sent them ... not that that's ever happened to me.]