Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411630 Posts in 69393 Topics- by 58447 Members - Latest Member: sinsofsven

May 11, 2024, 04:26:39 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Quick Socket Question (C++)
Pages: [1]
Print
Author Topic: Quick Socket Question (C++)  (Read 1846 times)
dspencer
Level 3
***


View Profile WWW
« on: January 14, 2010, 12:23:13 AM »

I couldn't find a source that really talks about this online - I'm surely searching for the wrong thing, so if anyone has any suggested reading that would be great.

Anyway, I'm implementing some networking logic, and right now I have it sending a class across the network:
Code:
sendTo(address, myInstance, sizeof(myClass))
This isn't necessarily cross-platform safe, is it? Assuming it isn't how can I make it so?

any and all help is appreciated. THANK YOU.
Logged

resistor
Level 0
**


View Profile
« Reply #1 on: January 14, 2010, 12:58:31 AM »

That's not portable at all, for two important reasons:

1) Different operating systems, and even different compilers on the same operating system, can and do choose different in-memory representations of an object.  Some will reorder fields, others will insert more or less padding.  In short, it's a mess.

2) Different computers can have different endian-nesses.  This is basically the order of the bytes in a multi-byte value in memory.  On X86 PCs, it's little endian (least-significant-byte first), while PowerPC as in all the current gen consoles is big endian.

The solution is to #1 create a defined serialization format.  You could serialize to strings, or to a binary format, but it needs to be something independent of how the compiler chooses to layout your objects.  The solution to #2 is to pick one canonical endianness for your protocol, and perform input byteswapping on platforms that don't match it.
Logged
mcc
Level 10
*****


glitch


View Profile WWW
« Reply #2 on: January 14, 2010, 01:03:48 AM »

Hm.

sendTo is a windows function. It exists nowhere else.

The kind of sockets used everywhere except windows are called "BSD sockets".

Personally I use mingw when targeting Windows, so I expect to be able to just haul off and use BSD sockets calls. I couldn't really tell you how a "real" Windows programmer would do this, though checking google for "BSD sockets windows" there seems to be a decent amount of information about coding in the BSD sockets style while maintaining Winsock compatibility.
Logged

My projects:<br />Games: Jumpman Retro-futuristic platforming iJumpman iPhone version Drumcircle PC+smartphone music toy<br />More: RUN HELLO
bateleur
Level 10
*****



View Profile
« Reply #3 on: January 14, 2010, 01:05:16 AM »

The solution to #2 is to pick one canonical endianness for your protocol

...or alternatively make it a byte-wise protocol in the first place. I prefer that approach because endianness bugs can be really horrible sometimes.
Logged

kometbomb
Level 0
***


View Profile WWW
« Reply #4 on: January 14, 2010, 01:07:50 AM »

I suggest you not to do that. Even if you disabled alignment, took care of endianness etc. and sending an instance worked across two different executables, you'll probably end up accidentally sending a pointer to the other computer or something like that.

BTW, If you need cross-platform sockets, I suggest SDL_net for that kind of stuff. I'm not sure but I think it's usable even if you don't use SDL.

Quote
The solution to #2 is to pick one canonical endianness for your protocol, and perform input byteswapping on platforms that don't match it.

I think it's better to have an ascii string that has the numbers as something you can read. That way you'll get free debugging help in addition to the endianness issue (including languages that do numbers with strings).
Logged

mcc
Level 10
*****


glitch


View Profile WWW
« Reply #5 on: January 14, 2010, 01:08:10 AM »

1) Different operating systems, and even different compilers on the same operating system, can and do choose different in-memory representations of an object.  Some will reorder fields, others will insert more or less padding.  In short, it's a mess.
Oh, yeah, wow, I wasn't even thinking of the question that way.

The really fun situation here will be if myClass/myInstance has a vtable... or any pointer members really.
Logged

My projects:<br />Games: Jumpman Retro-futuristic platforming iJumpman iPhone version Drumcircle PC+smartphone music toy<br />More: RUN HELLO
mewse
Level 6
*



View Profile WWW
« Reply #6 on: January 14, 2010, 01:22:42 AM »

Yeah, totally not cross-platform.

There are a couple of reasons.

1.  Integer sizes.

Different computers have different sizes for particular types of integer.  For example, on the PC, a "short" is typically 16-bit, and a "long" is 32-bit.  On a PS2, a "short" is 32-bit, and a "long" is 64-bit.  Other platforms have other lengths.  This means that your classes will likely be different sizes on different platforms.

For this reason, network packets are typically not built using "short" or "long" variables;  instead, you use the special networking types:
int16_t (16-bit integer)
uint16_t (16-bit unsigned integer)
int32_t (32-bit integer)
uint32_t (32-bit unsigned integer)

...etc.

But you'll almost certainly find regular 'short' and 'long' and 'int' variables in your classes, and those mean that the classes on different platforms are not likely to match.

2.  Endianness.

Different computers represent integers in different ways.  For example, Intel-based computers are "little endian", whereas PowerPC-based computers are "big endian".  It's basically about the order of the bytes within the number.  If we pretend that decimal digits are bytes, then a big-endian computer would store the number one-hundred and twenty-seven as "127".  By contrast, a little-endian computer would store it as "721".  For this reason, you can't simply copy the memory directly from one computer to another;  they store their values differently!

3.  Padding

Compilers pad data structures, to arrange values in memory in the best way.  Here's a typical example of what you  might see on a standard PC:

struct
{
  char valueA; // 1 byte, stored in byte 0.
  int valueB; // 4 bytes, stored in bytes 4-7  (1-3 are padding, to make sure that valueB is four-byte aligned)
  char valueC; // 1 byte, stored in byte 8
};

sizeof(struct) == 12.  Padding added to bring the struct out to a four-byte boundary (to make it pack nicely in case somebody puts it into an array)

But different computers pad their storage in different ways.  The locations of data within a struct on one type of computer won't necessarily match those on another platform.  Plus, depending on the padding, a struct or class can actually be different sizes on different platforms!

4.  vtables

Since you mention that it's a class, it's worth mentioning that any class with virtual functions will have some extra data attached to it, which includes pointers to the code containing those functions.  Needless to say, overwriting those pointers on another computer is going to be Extremely Bad.

5.  MTU

Since you mention that you're using 'sendTo' which transmits data using UDP, it's worth mentioning that UDP has a "Maximum Transmission Unit";  the maximum amount of data that you can send in one UDP packet.  The fun thing about this is that it's going to be different depending upon the source and destination, based upon the space required for routing information used by the network hardware between them.  Technically, MTU can be as high as about 1500 bytes, but you really shouldn't rely on having more than 1300 bytes in total.  If your class is larger than about 1300 bytes, you'll find that it'll often not be received by the intended recipient, as the routers between you and them will just silently drop the too-large packet.


So what do you do?

Well, you really need to pack your individual pieces of data into a cross-platform blob of data, which can be done relatively easily using functions like memcpy().   You need to do this yourself, to keep your compiler from laying out data in what it considers to be a "smart" way.  Something like this:

void *buffer[120];
int posInBuffer = 0;
memcpy( buffer[posInBuffer], &valueA, sizeof(valueA) );
posInBuffer += valueA;
memcpy( buffer[posInBuffer], &valueB, sizeof(valueB) );
posInBuffer += valueB;

...etc.  And you can copy data out of the buffer and back into your values, to decode a received packet.

Now, that's not quite enough, though;  you also have to deal with endianness, which I mentioned in point 2 above.  The internet standard is to transmit all data in big-endian format, and Berkeley sockets provides a standard set of functions to help convert your data to and from big-endian format. These functions are:

ntohl(uint32_t) // convert a 32-bit number from Network-format (big-endian) to Host-format (big-endian or little-endian, whichever is appropriate for the computer being run on)
htonl(uint32_t) // convert a 32-bit number from Host-format (big-endian or little-endian) to Network-format (big-endian).
ntohs(uint16_t) // convert a 16-bit number from Network-format to Host-format
htons(uint16_t) // convert a 16-bit number from Host-format to Network-format


Hope this explanation about why you can't naively send a class directly over the network makes sense.  You really need to manually convert and pack the specific data you want to synchronise into network-order packets, and then unpack it into native data on the receiving side.  Smiley
Logged
dspencer
Level 3
***


View Profile WWW
« Reply #7 on: January 14, 2010, 01:59:41 AM »

very very helpful! I had thought of some of the issues (pointers, for instance) but there are a lot of things I just didn't think of at all. thank you all!

any additional wisdom is still appreciated Smiley
Logged

increpare
Guest
« Reply #8 on: January 14, 2010, 03:23:13 AM »

I found your considerations educational, mewse : )
Logged
mewse
Level 6
*



View Profile WWW
« Reply #9 on: January 14, 2010, 04:39:51 AM »

Extra little bit:

My engine code for building this type of platform-agnostic blob of data is here:

http://www.vectorstorm.org/svn/repos/VectorStorm/trunk/Memory/MEM_Store.h
http://www.vectorstorm.org/svn/repos/VectorStorm/trunk/Memory/MEM_Store.cpp

I used it for writing out platform-independent save games in MMORPG Tycoon, network packets, and a bunch of other places as well.

Feel free to copy/modify/whatever, if you think it'd be useful for you.  You'll probably need to remove the functions for reading/writing my particular commonly-used classes, and replace them with your own, but otherwise it should just work.  Smiley
Logged
BorisTheBrave
Level 10
*****


View Profile WWW
« Reply #10 on: January 14, 2010, 04:19:47 PM »

No one's mentioned it, but USE A LIBRARY. Like Protocol Buffers or Boost::Serialization. You'll discover converting a class to and from a platform agnostic blob is seriously annoying, as there's no reflection. These libraries will insulate you from that, and from mistakes (it's really amazing what is allowed to vary from platform to platform, i'm pretty sure mewse's excellent post isn't the end of it).
Logged
Pages: [1]
Print
Jump to:  

Theme orange-lt created by panic