Sensitivity (ISO) is very different from every other variable in photography. The best way to think about it is that it is merely a technical means to achieve an end: to increase the brightness of images that would otherwise be very, very dark.
Shutter speed and aperture are real variables with both a technical and a creative aspect. By adjusting them you’re using physical mechanisms to make your image brighter or darker. At the same time, you’re making a creative, aesthetic choice. When you change your shutter speed, you enter the realm of motion blur. (Intended or not.) When you change aperture, you enter the realm of depth-of-field (blur in space).
When you change your sensitivity (ISO value), you enter the realm of noise. There is no reward, only the technical price you pay for digitally amplifying your sensitivity: noise.
ISO is not a real variable. Increasing your ISO will degrade the physical quality of your photo. It reduces the amount of fine detail, it corrodes your image.
When it comes to setting your ISO there’s only one rule of thumb: keep it as low as possible.
That being said recent digital cameras are getting incredibly good at reducing noise at higher ISO settings. A few years ago the highest ISO setting on digital cameras was 1600 and those images were rendered technically almost useless by noise. Today most cameras can go up to 6400 or 12800 (many of them even higher) and you can take very fine images with them at 1600 or 3200.
Whether you have a brand new camera or you inherited one from a friend, my advice is: test your camera. Take a series of images going through your ISO range, starting from 100 all the way to your highest setting. Compare your pictures on a computer monitor and you’ll find out how your camera reacts to boosting ISO, up to what point noise is acceptable to you and when it gets too much for your taste.
Different cameras will, naturally, produce different amounts of noise at different ISO settings. The images below show you just how great this loss of quality might be. The pictures were taken with a Canon EOS 550D, an entry-level DSLR.
1. Side-by-side comparisons. On the left, the photo was taken at 100 ISO. On the right, you have images from 200 all the way to 6400. (Download the images to see them at their full resolution.)
2. These are crops from the above photo to make the presence of noise even more obvious.
3. Another set of crops.