Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length

Advanced search

1370248 Posts in 64449 Topics- by 56501 Members - Latest Member: liquidpigstudios

December 09, 2019, 12:59:35 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsCommunityDevLogsGearBlocks - Build working physics based machines and mechanisms [DEMO]
Pages: 1 ... 7 8 [9]
Author Topic: GearBlocks - Build working physics based machines and mechanisms [DEMO]  (Read 23904 times)
Level 1

View Profile WWW
« Reply #160 on: October 07, 2019, 10:58:08 AM »

Saved game serialization

Well, seems like another two months has gone by in a flash!  I was away visiting family for some of this time, and the work I've been doing on the game has resulted in a frustrating lack of progress to show for it, but here's an update on what I've been up to anyway.

Saved game serialization

Up until now, saved games and constructions have been serialized via a binary stream, with no formatting, just using BinaryReader and BinaryWriter directly.  This is fast and results in a compact file size, but has one huge disadvantage, a lack of version tolerance.  In other words, if I add or remove any variables to be saved, or reorder them, then old saved game files will no longer load correctly.  To work around this I wrote code to check a version number in the saved file, and then convert things over for each added or removed variable.  This is a hack really, and has resulted in rather messy and hard to maintain code.

This situation is bad enough with just the demo version of the game out there, with a cut down feature set.  Maintaining saved game backwards compatibility will only get harder once the full version is released.

Ideally, I need a properly structured save file format, with some kind of key value pairing that would allow for version tolerance, but wouldn't bloat the file size too much.


First I investigated using BinaryFormatter, because it allows for version tolerance via optional fields, but I couldn't get it to work when deserializing MonoBehaviour classes.  I need to be able to instantiate the MonoBehaviour and then populate serialized values into it, not have the deserialization process itself try and allocate a new MonoBehaviour (which is not allowed by Unity).  I thought maybe using a serialization surrogate might allow for this, but couldn't figure out a way to make it work.  The other downside of BinaryFormatter is all the assembly and type data it saves out adds quite a bit to the file size.

Json serialization

So after looking around for other possible solutions, I decided to try Json.  This provides the key value pairs I need in a fairly compact structured format.  I used Json.NET from Newtonsoft (provided via a Unity asset store package for ease of integration) which seemed really promising, it's very easy to use and highly configurable.  In most cases there's very little additional code to write, you can just use the JsonProperty attribute to specify which class properties to serialize, and configure how they're serialized.  Also, it allows for populating properties of a MonoBehaviour that has already been allocated, by using JsonSerializer.Populate() inside a JsonConverter.

Still, it took me several weeks to get Json serialization working for both saved constructions and full saved games, there were a few stumbling blocks along the way which took time to work around, as did just learning how to best use the Json.NET API.  The end result seemed great though, it solved the version tolerance problem, and the code was so much simpler and cleaner.

One issue was that the resulting file sizes of the text based Json format were huge.  Given that the game uses the same serialization code path to send construction data between networked players, this was a problem.  So, I switched over to using Bson (the binary equivalent to Json), and also compressed the data via a DeflateStream.  This worked well, the resulting file sizes actually ending up smaller than my original binary stream format.

Performance and memory problems

At this point I thought I was good to go, but then I started profiling the Json serialization with large saved game files (more than a thousand parts), and realized I was in trouble.  Firstly, deserializing the saved game was more than twice as slow using Json vs. the old binary stream method.  This wasn't a complete disaster as the load times weren't terribly long in the first place.  The other more serious issue was that the Json deserialization did an enormous number of tiny GC allocations (as in millions of allocs, totalling hundreds of MB!).

I found that reducing the JsonProperty name lengths helped slightly with this but not to any significant degree.  I spent quite a lot of time restructuring the code that loads the various modules in the game (player, constructions, time of day, etc.) to try and deserialize from the file stream more efficiently, but this made very little difference to performance or memory usage unfortunately (the resulting code was cleaner than before though so I guess the refactoring was worth doing anyway).

I'm annoyed with myself that I didn't do enough tests to pick up on these problems before putting all the effort in to convert the game over to use Json.  If I'd known ahead of time, I probably wouldn't have bothered.

So now I'm not sure what to do.  If I stick with the old binary stream solution, then all the Json serialization effort will have been wasted and I'm still stuck with difficult to maintain code for backwards compatibility.  But the Json serialization option as it stands isn't acceptable, I'd need to do something to resolve the memory and performance issues.  One possibility would be to manually serialize everything (i.e. use JsonReader / JsonWriter directly rather than JsonSerializer), supposedly this is the fastest way as it avoids overhead from reflection and so on.

I've decided for now to put all this to one side, think about it some more, and come back to it later.  In the meantime I really need to get back to trying to make some positive progress with the rest of the game!

Level 1

View Profile WWW
« Reply #161 on: November 03, 2019, 06:12:24 PM »

Character customisation

Over the past few weeks I've been finishing up the player character work I started a while back, along with some code refactoring and other things.

Building a player character

I've now implemented functionality so that a construction built out of body parts can be treated as a player character model.  Such a "character construction" differs from a normal construction as follows:-

  • It doesn't get serialized out to saved games, because the player's character construction gets spawned separately (based on what was selected in the character screen, see below).
  • It doesn't collide with other non-character constructions or objects in the world (the player's CharacterController takes care of collisions).
  • It can't be selected, frozen, or destroyed.
  • Body parts in it don't move via physics, instead they are locked to the appropriate animated bone (e.g. head, upper arm, lower leg, etc.)

I added a new character screen accessible from the main menu.  This allows the user to select a character construction, which will then be spawned as the player character when entering a game.

To create a custom character, first the user assembles a construction as normal out of body parts, and then they save it via the save construction screen.  As shown below, I added a tab to this screen to allow a construction to be saved as a character so that it'll then appear in the character screen and be available for selection.

To switch to the new character, the user then has to exit back to the main menu, go into the character screen and select the one they just saved, before re-entering a game.

In game, the character construction can be seen in the world tool UI just like any other construction.  However there are some operations that can't be performed on it as mentioned before: select, freeze, or destroy.  This can be seen below, note that the character construction has an icon to identify it as such.

This pretty much completes the player character system although, as always, there are a few loose ends to tie up:-

  • I need to add checks to ensure that the construction is valid to be a character (e.g. has the correct body parts, and they are connected together appropriately).
  • Currently, to change the player character construction the user has to exit back to the main menu, I'd like to implement a way to swap it during game play.
  • The body part meshes need a bit more modelling work.
  • I'd like to add more attachment points to the body parts to allow for more customisation options.
  • I also want to add more body part variants and accessories.

Free flight mode

I've also finished working on a "free flight" mode for the player.  This turns off gravity for the player and changes the controls slightly so that the user can fly up and hover in the air, handy for when building large constructions!

Singleton squashing

Finally, I also did another code refactoring pass to eliminate the remaining singleton MonoBehaviours in the game (well nearly).  This primarily involved converting them to ScriptableObject assets, and replacing any public method calls with events and handlers.  I covered this topic in the ScriptableObjects For Fun and Profit blog post a while back, so I won't go into detail here.  Suffice to say I'm done with this refactoring process now, there are only a couple of singleton MonoBehaviours left, which are for my networking abstraction layer (something I also blogged about a while ago), and I think I'm just gonna leave these as they are.

Level 1

View Profile WWW
« Reply #162 on: November 27, 2019, 07:03:45 PM »

Player physics

Over the past few weeks I fixed a bunch of bugs, and then turned my attention to player physics and movement control.

Problems with using CharacterController

Since the very earliest days of the game's development I've been using a CharacterController for the player along with a "character motor" script (adapted from a Unity sample) to control it using the CharacterController's Move() function.  The character motor would apply forward / back, strafing, and jumping movement, acceleration due to gravity, and if the player was standing on a moving object it would make them move with it.

This setup worked reasonably well, but the CharacterController doesn't provide proper interactions with Rigidbodies in the world.  The player is treated as a static immovable object when things collide with them.  The player can't push objects around, and they can't be pushed by objects either.  It just looks wrong when for example a vehicle hits the player at high speed and they don't move at all.

Also, the character motor implementation had a bug where, if the player collided with a Rigidbody, they would occasionally get thrown across to the other side of the map.  Most often this happened when exiting a seat, very annoying!  I suspect this issue was related to the code that moved the player when standing on something, but I'm not 100% sure.

Despite all these problems, I didn't particularly want to add more work to my plate by throwing out this character motor code and starting again from scratch.  So I tried playing around in the OnControllerColliderHit() message to try and get the player to push stuff around, and in OnCollisionStay() I tried to make the player be pushed by objects that collide with them.  I couldn't get this to work nicely though, I think it really is necessary for the player to have a Rigidbody to give proper physical interactions, anything else is always going to be a bit of a hack at best.

At this point I realised I needed to ditch the CharacterController and start from scratch after all, using a Rigidbody instead.

A physics based player

So now instead of a CharacterController, the player has a CapsuleCollider and a Rigidbody.  I implemented new code that replicates the original forward / back, strafing, and jumping behaviour, by applying a force to the player's Rigidbody.  It uses a Physics.SphereCast() to detect contact with the ground and to find the Rigidbody that the player is standing on, if any.  If they are standing on a Rigidbody, its velocity is transferred to the player.  Also, the force used to move the player is applied in the opposite direction to the Rigidbody that they're standing on, thus obeying Newton's third law.

To sum up the features and benefits of this new approach:-

  • Correct player collisions with other Rigidbodies.
  • Player can push other Rigidbodies around.
  • Player can be pushed by other Rigidbodies.
  • Player can stand on a moving Rigidbody.
  • Obeys Newton's third law when player accelerates while standing on a Rigidbody, this results in really cool "conservation of momentum" behaviour.
  • Player falls due to gravity automatically from having a Rigidbody, but even better, the player's weight now pushes down on the Rigidbody that they're standing on.
  • Eliminates the old "player thrown across to the other side of the map" bug.
  • Code is simpler and cleaner than the old character motor implementation.

Here are some examples of all this in action:-

As you can see this provides a level of interaction between the player and the constructions they build that simply wasn't possible before, so I'm happy I bit the bullet and re-implemented things this way!

However it's by no means perfect, and there are a few issues / missing features that will need addressing at some point, for example:-

  • I'd like to implement code to handle climbing steps, I think the player needs an extra force to push them up steps, as they tend to get stuck at the moment.
  • I also want to add something to make the player slide off ledges, as right now the player can stand more than half way off a ledge with their feet in the air, which looks weird.
  • The old character motor had code to make the player slide down steep slopes, not especially crucial but it would be good to get this working in the new implementation.

Level 1

View Profile WWW
« Reply #163 on: December 01, 2019, 02:30:38 PM »

A bit later than I was hoping, but here’s a new demo build!  This one includes a whole bunch of player character improvements and several bug fixes too.


Level 0

View Profile
« Reply #164 on: December 02, 2019, 03:17:32 PM »

Awesome stuff! Garry's mod always had frustrated me with the instability of mechanical construction and the general 'flimsiness' of a lot of the constraints.  I'll definitely be checking out the test build tonight. 
Level 1

View Profile WWW
« Reply #165 on: December 05, 2019, 09:11:02 PM »

Join the GearBlocks Discord Server!

Hey everyone, I’ve finally gotten around to creating a Discord server for GearBlocks!

You can upload your save files there, so I think it will be a great place to share your creations, at least until I implement a proper way to do this from within the game itself.

It’s also a good place to discuss the game, share tips & tricks, give feedback, report bugs, and so on.

I’m still learning my way around the whole Discord thing, so let me know if you have any suggestions for the GearBlocks Discord.  Welcome aboard!
« Last Edit: December 05, 2019, 09:27:27 PM by dangersam » Logged

Pages: 1 ... 7 8 [9]
Jump to:  

Theme orange-lt created by panic