next »

[J-Snake]

I learned to come around, still not always a fan of garbage collection though even when most consider it a big plus. In games consisting of bubbles of gameplay I can deal with it.

[Klaim]

Serialization in C++.

What are you serialising? We've found a very network compatible method for this but it's not exactly syntactically light, I haven't seen serialisation code beyond literally copying the memory that is though.

I'm using protobuf for both serialization of game state and event messages spread on both network and internal even dispatching system.

The problem is just that: I'm using protobuf to solve the serialization problem by defining messages that are immediately serializable; But it's not a good idea to use these message types as base to build other types (neither through inheritance or composition) so it's better to have your C++ data model which really model your game world structure or at leat it's skeleton, then generate state messages that can or cannot be complete depending on the kind of message you're using the state message for.
Anyway, all this means you still need to write code to make your C++ code match the protobuf generated code for serialization and deserialization of states. It's easier for even messages as you just have to use directly protobuf generated code.

Now, other than that, protobuf is not cool to use on windows, but it's a bit easier than Thrift. There is also CApnProto but it's not production ready yet and works only on Linux for now.


The lack of Reflexion mechanism in C++ is acknowledged and there are work from the standard committee to fix it, but form the current timeline it seems it will be considered reeady after c++17 (assuming it's released in time) which basically mean for the next century.

J-Snake> I know all the  reflection stuffs about C# and Python and a bit about Ruby and I knokw what's missing, but C# isn't perfect either (better on reflection though) than C++ and pose some problem when you want to express some things. I'd say if I wanted to change language I would go try D (it's also the only useful language I don't have real experience yet with evne if I've been following it's progression for years).

[Geti]

protobuf

Fair enough, the lack of windows support and it being a non-native format would make me wary of it though - I can certainly appreciate why you'd not want to swap it out now for some homebrew solution though.

We use our own BitStream class which reads and writes arbitrary memory and any native type into/from a stream of bits. This is important for us because it means that booleans are represented as bits, and smaller types take smaller space over the network. There's also a built in compress/decompress function so once everything's packed into a larger stream it can all be squished down even more for the network.

For any serialisable object we just write code to serialise and unserialise the object that looks like

Code:

void Example::Serialise(BitStream& bt)
{
    bt.write<u8>(someNumber);           //write some uint8_t as 8 bits
    bt.write<bool>(someFlag);           //write some bool as 1 bit
    if(someFlag)                        //conditional serialisation
        someMember.Serialise(bt);       //of some internal structure
}

void Example::Unserialise(BitStream& bt)
{
    someNumber = bt.read<u8>();         //we do the exact reverse
    someFlag = bt.read<bool>();         //of the above to unserialise
    if(someFlag)
        someMember.Unserialise(bt);
}

As I said it's not syntactically light but its a convenience for us to have something that's very light on the network and pretty easy to use. Might be food for thought.

It's possible to overload the read method to take a reference so you can basically replace write with read and serialise with unserialise to automate it but we found it was easier to make mistakes that way, leaving a stray write in instead of a read obviously doesn't work so well.

[Abraxas]

Interesting, you compress every packet been sent over a connexion?

[Klaim]

Fair enough, the lack of windows support and it being a non-native format would make me wary of it though - I can certainly appreciate why you'd not want to swap it out now for some homebrew solution though.

I'm open to other solutions though, if it can fix my problems in less efforts than it was to setup protobuf in a useful way for me. That's why I'm following the work on CapnProto which seems promising and I can at least have a discussion with the author see if he can solve my problems (and use CMake...)
However, it looks like you're not solving exactly my problem.

We use our own BitStream class which reads and writes arbitrary memory and any native type into/from a stream of bits. This is important for us because it means that booleans are represented as bits, and smaller types take smaller space over the network. There's also a built in compress/decompress function so once everything's packed into a larger stream it can all be squished down even more for the network.

I'm using RakNet that have something like this too. So basically protobuf gives me types which are already ready to be sent on the network but I'm also using RakNet to not have to bother with this level of abstractoin: I just put my messages in a BitStream and it works (theorically) with any platform.

For any serialisable object we just write code to serialise and unserialise the object that looks like

Code:

void Example::Serialise(BitStream& bt)
{
    bt.write<u8>(someNumber);           //write some uint8_t as 8 bits
    bt.write<bool>(someFlag);           //write some bool as 1 bit
    if(someFlag)                        //conditional serialisation
        someMember.Serialise(bt);       //of some internal structure
}

void Example::Unserialise(BitStream& bt)
{
    someNumber = bt.read<u8>();         //we do the exact reverse
    someFlag = bt.read<bool>();         //of the above to unserialise
    if(someFlag)
        someMember.Unserialise(bt);
}

As I said it's not syntactically light but its a convenience for us to have something that's very light on the network and pretty easy to use. Might be food for thought.

That's exactly the part we can't avoid in C++: I use Protobuf to make sure I have cross-platform representation of the data, then I use RakNet to communicate this data IF it goes on the network.
My problem is that the types representing the game state and the types representing their saved state cannot be the same, so I have to do exactly the same serialization thing.
One advantage I have though is that I have no specification of the underlying type of each parameter in the serialization functions implementation, it's all protobuf generated code that take care of this.

To be clear: In my game I have to manipulate a lot of different communication protocols plus serialization of the data (both messages and state). Frankly it's not that common in games, so it's normal that I have a slightly more complex use case of serialization.

At least I managed to fully automate sending messages (through whatever communication system), only state serialization requires additional code.

[Geti]

@Klaim: what other platforms are you supporting?

I understand the concerns about portability but in my admittedly somewhat limited experience it's been better to go with something more efficient than something portable to _everything_.
For example, we use fixed width types very often to reduce the amount of casting we have to do during serialisation or delta calculation for sending things on the network, but using fixed width types means we're in trouble if a platform doesn't support the types we're using. We've accepted that because doing it this way lets us reach the audience we need and deliver a quality product within those limitations. It seems like it's mostly portable to ARM at the moment as well but we haven't bothered fixing the issues with a few libraries because it's not likely to help sales at all

I'd be much more concerned about something not running under windows than it not running on something other than x86 and ARM, personally.

Either way yes, having to manually unwind each structure is lame, but if most of your types that need to be serialised are pure data it seems like it's pretty trivial to do. If most of your types are part of some hellish inheritance tree with pointers to unwind everwhere then good luck, haha

Sorry I can't be of more help.

Interesting, you compress every packet been sent over a connexion?

We compress at a higher level than UDP packets, but more or less, yes. You want to fit as much data over the network as possible, trading some CPU usage to fit more in is a good trade as far as we're concerned. Of course, we try to minimise the amount of data that needs to be sent but compressing it helps squeeze more out of the shithouse VPS that people try to host gameservers on these days.

[Klaim]

@Klaim: what other platforms are you supporting?

Only windows/macosx/linux (ubuntu).

I understand the concerns about portability but in my admittedly somewhat limited experience it's been better to go with something more efficient than something portable to _everything_.

Maybe I was not clear, that is not at all what I'm doing. My need is to be able to generate message protocols which are portable as in a client on windows can communicate with a server on macosx, and vice-versa.

For example, we use fixed width types very often to reduce the amount of casting we have to do during serialisation or delta calculation for sending things on the network, but using fixed width types means we're in trouble if a platform doesn't support the types we're using. We've accepted that because doing it this way lets us reach the audience we need and deliver a quality product within those limitations. It seems like it's mostly portable to ARM at the moment as well but we haven't bothered fixing the issues with a few libraries because it's not likely to help sales at all

I'd be much more concerned about something not running under windows than it not running on something other than x86 and ARM, personally.

I'm not targetting something exotic, only desktop, Windows being the priority and I don't check my code with other platforms yet. Still the code needs to be portable with minimal effort when I start porting the version.
I don't expect to port the game on anything without a mouse or a keyboard actually (because of the nature of the game) but if I was to target tablets or consoles I could without any problem (other than shitty controls for this kind of game).

Either way yes, having to manually unwind each structure is lame, but if most of your types that need to be serialised are pure data it seems like it's pretty trivial to do. If most of your types are part of some hellish inheritance tree with pointers to unwind everwhere then good luck, haha

Fortunately I have no inheritance at all in my game state. But I'll be modifying properties a lot while adjusting the gameplay so I'll have to serialize additions. I'll not have to do any additional code for script data so it's good, the game data skeleton will not move that much I think.

Even without inheritance, it don't prevent the problem that if you generate the data code using something like protobuf, the interface of the data will not prevent logical errors and you still haven to encapsulate it in some ways. The problem is not the data, the problem is that generated code, in this specific case, is not directly usable as data to represent the game. You still have to convert it to something with semantic apparent in the interface.

All my data have value semantic, but it's all interfaced to avoid inconsistencies on the game semantic level. It's not a generic engine either, it simulate something very specific.

Sorry I can't be of more help.

Don't be, the origin is a language design problem, nobody can solve it here.

[Geti]

My need is to be able to generate message protocols which are portable as in a client on windows can communicate with a server on macosx, and vice-versa.

Fwiw our bitstream approach works just fine for this, no need for a layer in between In our case most servers are linux and most clients are windows.
The game I'm speaking mostly about here is almost entirely multiplayer with a heterogeneous (platform wise) playerbase. You don't need something generating portable messages for you if you're just serialising data for use between two desktop OSes.

I thought you were saying protobuf was badly supported on windows - for something you don't need, that was ringing alarm bells

Looking into capn proto though, it seems like a cool thing. I can see why you'd be interested.

Either way I'll stop banging on, I think we're more or less on the same page about it being lame.



Ontopic: Today I spent about an hour looking into how to stop a mine from using its cached "doesCollideWith" value for a door when you put it part way through a door, then threw it. Making it refresh the value next frame didn't work, presumably due to some ugly order of operations issue involving it not being properly "dropped" at the time the check was made. Ended up solving the issue by making it collide with the doors while in someones hands. Easy.

[Klaim]

I don't think protobuf is an additional layer compared to what you do, it's similar but more ....complex or verbose. Anyway yeah we're on the same page, didn't mean to argue.

I thought you were saying protobuf was badly supported on windows - for something you don't need, that was ringing alarm bells

It's just hard to use on windows because of lack of support of the project files, not because it's windows, it works fine otherwise, I was not clear about that sorry.

Looking into capn proto though, it seems like a cool thing. I can see why you'd be interested.

Looks like a future protobuf++, by the same guy at origin of protobuf.

[sublinimal]

happy_debugging.png

After some twiddling, this error changed into one about a name clash, so I have no clue why the compiler couldn't properly catch it. I think I just witnessed a legit compiler bug.

[Kekskiller]

Consider yourself lucky, I never noticed any compiler bugs with GCC or Visual Studio.

[Geti]

Just had one of those days where you from the moment you pull, update and compile you're fixing bugs made last night by a bunch of hasty branch merges and library updates without warnings about what might have changed. Got about 1 thing on my todo list done and added three (not counting the several larger bugs fixed due to the aforementioned botches).

10am start, 1pm home from uni and start coding, 1am finish... gonna be coffee time tomorrow.

[Code_Assassin]

Just had one of those days where you from the moment you pull, update and compile you're fixing bugs made last night by a bunch of hasty branch merges and library updates without warnings about what might have changed. Got about 1 thing on my todo list done and added three (not counting the several larger bugs fixed due to the aforementioned botches).

10am start, 1pm home from uni and start coding, 1am finish... gonna be coffee time tomorrow.

u got dis man 

[Kekskiller]

I want direct neural interface with my code editor. Sucks to always write the same shit over and over again cause C/C++ doesn't nicely support code generation except using a myriad of macros.

[Klaim]

I want direct neural interface with my code editor. Sucks to always write the same shit over and over again cause C/C++ doesn't nicely support code generation except using a myriad of macros.

Not exactly what you're thinking about but this might interest you: http://www.youtube.com/watch?v=8SkdfdXWYaI

[Kekskiller]

I want direct neural interface with my code editor. Sucks to always write the same shit over and over again cause C/C++ doesn't nicely support code generation except using a myriad of macros.

Not exactly what you're thinking about but this might interest you: http://www.youtube.com/watch?v=8SkdfdXWYaI

Wow. That was... pretty amazing. I feel like stone-age now. My solution to the code generation problem was (so far) to design a syntax that replaces and extends C with operators for loops and code generation/templatation but voice recognition is quite amazing on it's own. Not sure if I could get this to work with C/C++ but it sounds like an interesting solution to some annoying problems, yes.

[Liza]

I hate that my score formula is for some reason outputting a completely nonsensical value even though following the exact same formula manually gives me the expected result.

Then again, this was at 1am last night when I was half asleep and thinking mostly about the cheese crisps I had burnt the day before. They were going to be delicious.

[Eigen]

I hate that my score formula is for some reason outputting a completely nonsensical value even though following the exact same formula manually gives me the expected result.

Then again, this was at 1am last night when I was half asleep and thinking mostly about the cheese crisps I had burnt the day before. They were going to be delicious.

That can happen if one of your variables used in the formula contains unexpected values or pure garbage.

[Liza]

I hate that my score formula is for some reason outputting a completely nonsensical value even though following the exact same formula manually gives me the expected result.

Then again, this was at 1am last night when I was half asleep and thinking mostly about the cheese crisps I had burnt the day before. They were going to be delicious.

That can happen if one of your variables used in the formula contains unexpected values or pure garbage.

I print all the variables to the console individually to check before running the calculation with them But yeah, it's likely something really stupid that will be extremely obvious when I look at it later today in a non-sleepy state.

[Thomas Hiatt]

I had that problem before because I was using a local variable that was getting destroyed after the function was done. Took me hours before I noticed it.