Weekly Update #4Neural Networks and Genetic Algorithms!Since I'm starting work on flight AI, I figured it's a perfect excuse to try developing a NN/GA system! I got most of the technical details from these excellent articles on ai-junkie:
Neural Nets,
Genetic AlgorithmsTurns out they aren't much work to setup the basic code, but they are very complex to understand at a deep level. Here's what I learned over the past week:
The Neural NetMy requirement was simply to discover a way to reduce an agent's distance from a target, hopefully in an "interesting" way. The ships have 3 degrees of freedom:
pitch,
roll, and
forward thrust. These become the output of the neural net. The inputs are the angular correction in yaw, and pitch required to face the target. You could also use the dot product, and save 1 input (thereby making the system less complex, and training faster) although, when I tried, it
appeared to learn faster using discrete yaw/pitch angles. I don't know enough about how the hidden layers in NNs work for me to say why, maybe it was just coincidence
.
2 inputs (plus a bias), 3 outputs, and 1 hidden layer, with 6 neurons (using sigmoid activation).
From what I gather, the GA setup I is pretty standard: mutation rate of 10%, chromosome crossover rate of 0.7, and a population of 50 agents.
The fitness equation is just the grade each agent receives, based on how well an agent satisfies it's goal. My initial fitness equation was the inverse of the distance between an agent and it's target (closer = higher fitness). It was enough that the AIs eventually learned to fly towards their target.
Of course, emergent behavior is all part of genetic algorithms, though often undesirable. The agents learned to close the distance to a target, but in my case they also barrel roll continuously. Even though it's technically solving the problem, it looks ridiculous, so I had to modify the fitness equation to be more specific. Z-axis rotations are now penalized heavily, but not too much as to prevent them from rotating at all, or interfere with the primary goal of reaching their target.
Because of the relative simplicity of my setup, training was quick, and I can evolve decent looking candidate after around 25 generations.
It's a lot of fun to watch the agents evolve over time, and produces decent results. But I could have spent a fraction of this time manually writing the flight code myself with very similar results. A lot of time is spent tweaking values, and waiting for the system to do it's thing, before you see how your changes. It becomes tedious, and made me realize using a NN/GA for such a basic problem of "fly from A to B" is silly -- unless you're training an
actual spacecaft to fly in actual
space. I simply hoped a visually interesting technique would come out of it, but I realize now that there isn't enough freedom or complexity for stylish "creativity" to emerge in how I set it up.
I think a better use case might be in high-level decision making, like when to switch to chase/evade when given inputs like health, target distance, and other situational variables.
Thanks for reading!
Community
