Right now my memory usage is almost 120 MB for this test, which is 5 tracks of music. Will a smaller file format stay small in memory, or is it decompressed to that large in memory anyway?
The file must be decompressed to be played, to the best of my knowledge. So even oggs will use comparable memory, they'll just have a smaller disk footprint.
Is it standard to stream for this, or load all the tracks into memory in the beginning?
Streaming is the way go, in my opinion. OpenAL supports buffer queues, which allow you to do streaming fairly easily. The tricky part is decoding the ogg vorbis data into the buffers.
This unfortunately dead project of mine contains code that demonstrates how to do this. Check out SoundWrapper.h/cpp in the source/base directory. It's fairly well commented.