Note: This doesnt work for iPhone 4S or iPad 2, stay away from 5.1 for now if you care about jailbreaking.
Step 1: Download RedSn0w 0.9.10b6 for Windows here, Mac here and run it.
Step 2: Click jailbreak on RedSnow, and it will go through the process of finding the necessary firmware. It will then ask you to install Cydia, make sure that is selected and click next.
Step 3:Place your iPhone 4, or iPod touch, etc. into DFU mode.
Step 4: You should see a lot of scrolling text on your device, and eventually you will be back to your Home screen. You will see a Cydia icon but it will be white. That is because you have to perform a tethered boot.
Step 5: Place your iPhone 4, or iPod touch, etc. back into DFU mode.
Step 6: Click back on redSn0w and select Just boot. This will perform a tethered boot, and once your device returns to its Home screen, you will see a usable Cydia app icon. Open Cydia, and enjoy your jailbreak.
Note: Each time you reboot your device, you will have to perform steps 5 and 6 again to perform a tethered boot.