to make things simple.
PSD is the best, but cannot be hosted. It is the original file. It contains all the layers.
PNG is the best that can be hosted, it is a copy of the original all into one layer.
JPG is the middle class of images that can be hosted using html. It is a lower quality (compressed) version of the original. No layers.
GIF is the worst quality, but most adaptable one. GIF's can be hosted with HTML. There are only 256 colours allowed, and compression is very high. Only one layer. HOWEVER, GIF's support transparency and animation.