Time for maximum infos:
BMP = Full quality, full color, huge filesize
GIF = Full quality, 256 colors or less, decent filesize, allows transparency, allows animation
JPG = Good to horrible quality, full color, small filesize
PNG = Full quality, full color, decent filesize, allows transparency/alpha channel (IE displays transparency/alpha channels incorrectly)
The good quality for JPG generally only applies to photographs. It's a lossy compression, meaning anything which needs to retain actual pixels (such as with sprites) won't work well.
GIF is usually used by folks for sprites, as long as you don't get the colors mangled. Some programs are better than others at handling the colors; MS Paint for example has a tendency to mess them up if you're not careful. If you don't want to worry about that, use PNG. I think it will one day RULE THE INTERNET.