next »

[Fuzzy Branch]

Hi everyone, I've been prototyping an idea for awhile now but have come to a bit of a crossroads, so I thought i'd take a step back and share what I have so far to get some badly needed outside perspective. I'll explain more at the end of the post but for now, here's the game overview and prototype:

-----------------------------------------------

Planet Crawlers is a simulation game where you are in charge of training autonomous self-learning robots to navigate remote planetary landscapes, set in a retro 1950's vision of the future.

Players choose from a variety of robot types and customize their hardware. Train them by choosing which behaviors to reward and which to punish, and twisting the knobs of the training simulator. Watch as they go from dull lifeless metal to... slightly less dull, but still lifeless metal.

Each round, a population of robot candidates are tested in a physics-based environment. Depending on which factors the player has chosen to reward, the top performers are selected as the foundation for a new, slightly modified candidate population. Then the cycle repeats, resulting in the robots’ performances gradually improving. The test environments are generated dynamically and evolve over time, including hazards and obstacles, all of which the player can influence.

Engine: Unity 5.6
Platforms: Windows PC

Design Inspiration:





Visual Targets:

Features:
-Multiple base character types: Quadraped, Biped, Snake, Wheels, etc.
-Modular design for mix and matching accessories:
-Variety of muscle/joint/thrusters/gyros as well as sensors & gadgets
-Customizable cosmetic widgets, buttons, lights, doodads
-Various environmental hazards, obstacles, and conditions.
-Visualizations of the brain’s development over time and current status of training.
-Visualization of candidate trajectories to see a population's performance at a glance
-Graphs and charts to show performance metrics of robot.
-Reward/Punish a wide array of behaviors and properties, such as robot velocity, distance traveled, and energy expended.
-Control the training simulator by adjusting new candidate variety, population size, test thoroughness, etc.

-----------------------------------------------

I've gotten to a point developing the prototype where it's clear that to hit the quality bar I'd like, the scope of the project is too large to accomplish solo in a reasonable amount of time (even after considerable trimming). So if i'm going to move forward on it I'd like to get a better sense if it's something that people might be interested in. Especially since it's a weird experimental game that doesn't have much direct control, and can be quite slow paced. Is this something you would play?
 Thanks everyone!

[Fuzzy Branch]

Some results of different training runs:

[ActualDog]

I'm probably a bit biased but I think this is super cool. The evolved walks have a ton of personality to them and I love the look of these guys. The bendy legs, the little faces and tails... so good!

Trying to figure out a fun, accessible, and reasonably scoped way to actually package up a simulation like this is something I've been struggling with for a long time myself. It's a really tough problem to solve. I'd be curious to know how long it takes and how easy it is to actually evolve the sorts of bot movements you're showing off in the gifs. I definitely think that there's some amount of patience potential players will have but it all depends. If it takes 15 minutes, that seems pretty reasonable and I can see people really getting into it, specifically if there are actual goals and rewards. If it takes a few hours of training, that's a bit less appealing. Regardless, I think time becomes less of an issue the more player input and tinkering is required along the way. I'd be much more interested in spending a long time evolving a walk cycle if there was stuff for me to be actively doing rather than if I just kinda had to sit back and let things play out passively.

Anyways, going just from the results you're showing and from what I've seen previously of your tech (I'm assuming you're using the same base tech you've been working on for a while) I can absolutely see there being interest in this. I'm interested, at least!

I'd love to hear more about where you'd like to take this from here.

[jmas]

Simulation looks amazing and graphics are incredible, definitely watching this

[Fuzzy Branch]

Trying to figure out a fun, accessible, and reasonably scoped way to actually package up a simulation like this is something I've been struggling with for a long time myself. It's a really tough problem to solve. I'd be curious to know how long it takes and how easy it is to actually evolve the sorts of bot movements you're showing off in the gifs. I definitely think that there's some amount of patience potential players will have but it all depends. If it takes 15 minutes, that seems pretty reasonable and I can see people really getting into it, specifically if there are actual goals and rewards. If it takes a few hours of training, that's a bit less appealing. Regardless, I think time becomes less of an issue the more player input and tinkering is required along the way. I'd be much more interested in spending a long time evolving a walk cycle if there was stuff for me to be actively doing rather than if I just kinda had to sit back and let things play out passively.

Thanks ActualDog! I've been enjoying following your project!

Yeah that's been one of the main glaring problems for this project and the area that's been giving me the most anxiety. The good news is that I haven't spent much time on the algorithm itself yet, so there should be plenty of room for improvement, however, there's no way to know how much room for improvement there is until I implement it.

So much of the enjoyment can rest on the performance of the learning process -- If it's too slow to run it's boring, if it doesn't produce a variety of solutions it's boring, if the final solutions aren't good then it's boring.  These training runs took a few hours or more to get to the point in the gifs, although they get maybe 70-80% of the way there in less than an hour, usually.  Seeing it change visible amounts in real-time would be the best. This sort of happens right at the beginning of training, but quickly slows. It's not where I would like it to be, certainly. I would be curious to see if players would be ok with letting it run overnight and seeing results in the morning.

That brings up the other path, which is to have meaningful things to do during training. I definitely want to experiment with this. I have a bunch of options but need to start prototyping and getting an idea of what works and what doesn't. I will definitely have customization options where the player can continuously edit the robot's appearance, sensors, gadgets, etc, and will have great freedom to morph the environment by adding different types of terrain, hazards, and obstacles.  I'd also like to see if it would be fun to give the player direct control over throwing obstacles at the robot or pushing/pulling it with a virtual hand/force.

Beyond that, I'd like to have a series of visualizations/graphs to convey the status of training and give the player feedback in order to adjust the settings of the training simulator -- for example, if the robots are moving quickly but fall down a lot, the player can adjust the priority of staying on its feet, lower the mutation rate, or add a balancing gyro gadget for example.

One of the difficulties is that there are a ton of options for how to expand the scope of game to make it more interesting, but I worry that if the core process isn't fun or engaging then it won't matter how much extra stuff I tack on. This is the area that I'll be primarily focusing on in the near future.

Simulation looks amazing and graphics are incredible, definitely watching this

Thank you very much!

[s0]

That looks cool as heck.

[Sundrop]

Wow four-legged Benders!

On the gameplay side; how about after a series of set obstacles that help you approximate your Crawler's growth, you get to roam around ripping limbs from other bots and maybe get an upgrade?

Again, very cool project!

[Fuzzy Branch]

That looks cool as heck.

Thanks!

On the gameplay side; how about after a series of set obstacles that help you approximate your Crawler's growth, you get to roam around ripping limbs from other bots and maybe get an upgrade?

Combat (and other multi-agent games) were part of my original idea, but after developing for awhile I got a better sense for how big the project's scope actually was (spoiler: a lot more than I thought before I started, even being aware of that exact danger).  So recently I've tried to really narrow the focus and see how far I can cut it down in order to be manageable, because it's important to me that I create a finished product for sale.  Thus I decided to just focus on walking and general movement in the beginning. However, my hope is that I can use that simple foundation and build things like multi-agent games and more exploration in the future.

I like your idea of finding part upgrades out in the wild, and general durability / damage to the robots. I've been thinking about these and am excited about the possibilities of having to compensate for an injured leg, or even taking damage that affects the functionality of the brain network itself rather than just a -20 health kind of system.

Speaking of brains, these are some experimental visualizations of the robots neural network brains, composed of the neuron balls connected by axons. My goal is to have a number of data visualizations like this to serve as feedback for the player. I'm toying with the idea of keeping the graphs, charts, and data visualizations in very organic naturalistic visual style, to contrast with the stark robot and environment designs, but we'll see :-)

The final look will probably be vastly different but I was having fun playing with compute shaders:

[2ndStudio]

I love the style, it's great Are the animations psychics based or how does it work?

How many buttons do you use to control the robots?

[Fuzzy Branch]

Thank you!

All the movement is physics based (rigidBodies connected by motorized joints). The "brain" (neural network) gets sensory input about its current limb orientations (plus some other stuff), and uses that information to determine how much to actuate each of its motors. The best-performing neural networks are kept and slightly modified, and the poor performers are discarded each generation, based on what the player has chosen to prioritize/reward.

They are fully autonomous right now, but I'm going to be revisiting this in the future, probably giving player control over the robot's goal location, similar to the way you would issue move commands to units in an RTS game.

The gameplay is still subject to change, possibly dramatically, while I try to refine and distill the concept.

[ActualDog]

Thanks ActualDog! I've been enjoying following your project!

Yeah that's been one of the main glaring problems for this project and the area that's been giving me the most anxiety. The good news is that I haven't spent much time on the algorithm itself yet, so there should be plenty of room for improvement, however, there's no way to know how much room for improvement there is until I implement it.

So much of the enjoyment can rest on the performance of the learning process -- If it's too slow to run it's boring, if it doesn't produce a variety of solutions it's boring, if the final solutions aren't good then it's boring.  These training runs took a few hours or more to get to the point in the gifs, although they get maybe 70-80% of the way there in less than an hour, usually.  Seeing it change visible amounts in real-time would be the best. This sort of happens right at the beginning of training, but quickly slows. It's not where I would like it to be, certainly. I would be curious to see if players would be ok with letting it run overnight and seeing results in the morning.

That brings up the other path, which is to have meaningful things to do during training. I definitely want to experiment with this. I have a bunch of options but need to start prototyping and getting an idea of what works and what doesn't. I will definitely have customization options where the player can continuously edit the robot's appearance, sensors, gadgets, etc, and will have great freedom to morph the environment by adding different types of terrain, hazards, and obstacles.  I'd also like to see if it would be fun to give the player direct control over throwing obstacles at the robot or pushing/pulling it with a virtual hand/force.

Beyond that, I'd like to have a series of visualizations/graphs to convey the status of training and give the player feedback in order to adjust the settings of the training simulator -- for example, if the robots are moving quickly but fall down a lot, the player can adjust the priority of staying on its feet, lower the mutation rate, or add a balancing gyro gadget for example.

One of the difficulties is that there are a ton of options for how to expand the scope of game to make it more interesting, but I worry that if the core process isn't fun or engaging then it won't matter how much extra stuff I tack on. This is the area that I'll be primarily focusing on in the near future.

Very interested to see where you end up with all this. Without knowing too much about how your system is set up, how feasible would it be to have multiple simulations running simultaneously? I could see the long simulation times being much less of an issue if you had the option of running a few in the background and checking in occasionally while you tweaked some settings and got ready to set up your next trainer.

Also I'm super into those brain visualizations! Really really great stuff.

[Fuzzy Branch]

Very interested to see where you end up with all this. Without knowing too much about how your system is set up, how feasible would it be to have multiple simulations running simultaneously? I could see the long simulation times being much less of an issue if you had the option of running a few in the background and checking in occasionally while you tweaked some settings and got ready to set up your next trainer.

Also I'm super into those brain visualizations! Really really great stuff.

This version actually does have multiple instances training in the background simultaneously, but it's still slow due to how high I have to crank physX settings to get stable movements (plus a general lack of optimization so far, but there's only so much that will help).

Sort of on that note, I've been thinking about it for awhile, and I've decided to put this project on hold for now and take some time to explore my options and go back to the drawing board a little bit.

In order to get the concept up to the level of quality/polish that I'd like, it's going to be a larger task than I had originally intended or expected, even after stripping it down to its leanest version. Furthermore, if I hunker down and build out that simplified but polished version of the project, I don't feel like the final product will be substantial or competitive. My potential solutions to its weaknesses all seem to be bandaids or extensions built on top, rather than true solutions, and I don't feel good about blindly pursuing it without taking a step back to consider all my options.

It was always my intention to start on a very small project to gain experience building a game from scratch to release, before attempting a more ambitious, multi-year endeavor. This project originally seemed like it could fit that description, but at this point it's clear that is no longer the case. Given that, I'm taking some time to work on a series of quick experiments in order to clear my head, practice my dev skills (especially GPU-based tasks), and for a change of pace. The goal is to experiment and fail quickly so I can try a variety of things in a shorter amount of time.

I've been interested in different pattern-generation techniques for awhile and there are a number of algorithms I haven't had a chance to explore, so my early experiments are in that sphere. I'm not sure if it's ok to use this thread to post these side projects but I figured it would be better than starting a new thread. Please let me know if this is frowned upon.

The first thing I messed around with was a GPU implementation of the diffusion-reaction algorithm. Here are some resources if anyone is interested in trying it out themselves: (the youtube video tutorial is not for GPU, I just loosely skimmed it and implemented a version in Unity using Compute Shaders, but I think it would be helpful for someone trying to code it)
http://www.karlsims.com/rd.html
https://en.wikipedia.org/wiki/Reaction%E2%80%93diffusion_system




Lessons learned:
-Sort of cool
-Extremely sensitive to parameters
-Limited to a smallish subset of patterns

I just made a quick application where you can 'paint' density with the mouse cursor:





The next mini-project was to experiment with a variant of Convolutional Neural Networks on the GPU, inspired by some recent machine learning applications for image generation, particularly GAN's (Generative Adversarial Networks). They're basically a collection of many image filters (think of photoshop's gaussian blur or sharpen, those are convolutional filters)
https://en.wikipedia.org/wiki/Convolutional_neural_network
https://blog.openai.com/generative-models/

Because I'm starting from scratch in Unity and don't have a giant image library to train on, I decided to try a different approach and basically evolve image filters that take random noise and transform it into a new image. Then I specify a target image and compare that reference image to the generated images by the similarity of their histograms.

Here's an example training run:



Lessons Learned:
-Pretty limited use, in its current state.
-Most basic filters produce a small subset of patterns much more easily than others, so many generated images look similar. It lacks the expressive power to generate any arbitrary image. This might be less of an issue if I created 'wider' networks (more parallel filters) which is possible but the way I implemented it was to use color channels as filters, so I only had 4 filters per layer.
-My comparison algorithm of comparing histograms is only a very coarse approximation of image similarity.
-The learning algorithm itself could use some improvements finding good candidate filters.
-Overall not a great success, but for a subset of patterns it's pretty decent, and quite fast (usually a minute or two per run)

Thank you everyone!

[flex$]

hey bud, this is really neat and i'm definitely interested. you're obviously a really talented programmer

[io3 creations]

Hehe, looks like Blender has been turned into a doggy: