To extend my knowledge of machine learning's use in video games, I am exploring how I could teach a 'dumb' AI
to play a game by itself. This led to begin implementing an artificial neural network that could take several
inputs, apply some random calculation, and turn them into a number of outputs. I then decided to create a
genetic algorithm to train this AI by recording a fitness for each agent before removing the worst and breeding
the best (with a random chance of mutations) to theoretically create a better generation next time around. This is
called unsupervised learning, since after initially giving each agent the opportunities to increase or decrease
their fitness level, I can just leave them run and figure things out for themselves.
I decided to start off simple and create an AI that will eventually learn to navigate a course. Version 1 of this (seen
below) has five distance sensors facing outwards from the frontside of its body and it uses these to gather the distances
to any walls either in front of it or to its sides before sending them into the neural network inputs. Once calculations have
been applied to these five inputs, they are then sent to two output nodes, which in turn apply their own random calculations
and signal whether the AI should rotate left or right whilst it continuously drives forward.
The fitness in this version is simply how long the AI can stay alive without touching a wall (there is a problem with this that
will be explored later). Agents that apply the incorrect calculations will steer into a wall and therefore have limited fitness,
which in turn will cause them to be culled from the array; whilst agents that randomly find a better method will live on through to the
next generation in order to spread its genetic code.
Currently, version 1 takes around 30-40 generations before it achieves a 50% success rate of agents reaching the finish line.
Instead of necessarily improving on this, version 2 will instead have a new fitness system and also use its current speed as an input
for the neural network, to allow it to decide whether to accelerate or brake through two new output nodes. Hopefully, this will allow
agents to fine tune their cornering to better find the most efficient path around the track.
Source code for version 1, as well as a glimpse at my progress into version 2 can be found on my GitHub.