Help me work out colour choices!

Hey everyone. I'm doing bits and pieces of art style development for my game Bleak and I've run into a road block. The game's going to have these five population groups, and they all need a colour. I've tried all sorts of workflows to decide them, Adobe Kuler, Palleton, Randomly spinning the colourwheel and choosing whatever sticks, but nothing feels right.

My problems are thus:

- There's two opposing pairs, and one outlier, and I don't know if I want to show that in the colours as well as the 'narrative'
- socially, both Prisoners and the military have colours associated with them already( Luminous orange and drab green respectively)
- hue/saturation balance, keying everything to the prisoner values makes the military luminous green, and vice versa.

Whilst the course I'm doing at University is arts based, I've gravitated towards modelling and the more technical aspects, so my artistic knowledge has dried up somewhat. I don't suppose any of you wonderful people would be kind enough to offer advice in this matter?