Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411501 Posts in 69373 Topics- by 58429 Members - Latest Member: Alternalo

April 25, 2024, 02:52:58 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Storing Game Data Efficeintly
Pages: 1 [2] 3
Print
Author Topic: Storing Game Data Efficeintly  (Read 14972 times)
handCraftedRadio
The Ultimate Samurai
Level 10
*



View Profile WWW
« Reply #20 on: July 15, 2007, 08:23:33 PM »

Wow! Thanks for all the help! I still have a problem though. Whenever I try to read it i'm getting all ones. Here is the code I have:

Writing:
Code:
int zBits;
  ofstream LFile("LevelFile.lvl", ios::out | ios::binary);
  for (int SCheckX = 0; SCheckX < 1024; SCheckX = SCheckX + 8)
  {
   for (int SCheckY = 0; SCheckY < 768; SCheckY++)
   {
    zBits |= (MapCo[SCheckX][SCheckY]<<0);
    zBits |= (MapCo[SCheckX+1][SCheckY]<<1);
    zBits |= (MapCo[SCheckX+2][SCheckY]<<2);
    zBits |= (MapCo[SCheckX+3][SCheckY]<<3);
    zBits |= (MapCo[SCheckX+4][SCheckY]<<4);
    zBits |= (MapCo[SCheckX+5][SCheckY]<<5);
    zBits |= (MapCo[SCheckX+6][SCheckY]<<6);
    zBits |= (MapCo[SCheckX+7][SCheckY]<<7);
   
    LFile.write((char *)(&zBits),sizeof(int));
 
   }
   
  }

Reading:
Code:
int zBit[8];
 unsigned char rNum;
   for (int SCheckX = 0; SCheckX < 1024; SCheckX = SCheckX + 8)
  {
   for (int SCheckY = 0; SCheckY < 768; SCheckY++)
   {
    LFile.read((char *) &rNum, sizeof(rNum));
   
    zBit[0] =  (rNum >> 0) & 1;
    zBit[1] =  (rNum >> 1) & 1;
    zBit[2] =  (rNum >> 2) & 1;
    zBit[3] =  (rNum >> 3) & 1;
    zBit[4] =  (rNum >> 4) & 1;
    zBit[5] =  (rNum >> 5) & 1;
    zBit[6] =  (rNum >> 6) & 1;
    zBit[7] =  (rNum >> 7) & 1;
   
    MapCo[SCheckX][SCheckY] = zBit[0];
    MapCo[SCheckX + 1][SCheckY] = zBit[1];
    MapCo[SCheckX + 2][SCheckY] = zBit[2];
    MapCo[SCheckX + 3][SCheckY] = zBit[3];
    MapCo[SCheckX + 4][SCheckY] = zBit[4];
    MapCo[SCheckX + 5][SCheckY] = zBit[5];
    MapCo[SCheckX + 6][SCheckY] = zBit[6];
    MapCo[SCheckX + 7][SCheckY] = zBit[7];
   
   }
   
  }

I must be doing something wrong but I don't know what it is.
Logged

PoV
Level 5
*****


Click on the eye and something might happen..maybe


View Profile WWW
« Reply #21 on: July 15, 2007, 09:28:54 PM »

You're not clearing your temporary variable each time.  OR'ing will eventually fill it all up with 1's.

Also, you might want to write a byte, and not an integer, else your file will be 4x larger than it needs to be.

Code:
  ofstream LFile("LevelFile.lvl", ios::out | ios::binary);
  for (int SCheckX = 0; SCheckX < 1024; SCheckX = SCheckX + 8)
  {
   for (int SCheckY = 0; SCheckY < 768; SCheckY++)
   {
    unsigned char zBits = 0;
    zBits |= (MapCo[SCheckX][SCheckY]<<0);
    zBits |= (MapCo[SCheckX+1][SCheckY]<<1);
    zBits |= (MapCo[SCheckX+2][SCheckY]<<2);
    zBits |= (MapCo[SCheckX+3][SCheckY]<<3);
    zBits |= (MapCo[SCheckX+4][SCheckY]<<4);
    zBits |= (MapCo[SCheckX+5][SCheckY]<<5);
    zBits |= (MapCo[SCheckX+6][SCheckY]<<6);
    zBits |= (MapCo[SCheckX+7][SCheckY]<<7);
   
    LFile.write((char *)(&zBits),sizeof(unsigned char));
 
   }
  }

And in your read code, you should be able to do away with the zBit array, and set MapCo directly to the values.
Logged

Mike Kasprzak | Sykhronics Entertainment - Smiles (HD), PuffBOMB, towlr, Ludum Dare - Blog
ravuya
Level 7
**


Yip yip yip yip yip


View Profile WWW
« Reply #22 on: July 15, 2007, 09:40:42 PM »

Hey, don't forget about endianness, you jerks.
Logged

PoV
Level 5
*****


Click on the eye and something might happen..maybe


View Profile WWW
« Reply #23 on: July 15, 2007, 10:09:32 PM »

It should be fine.
Logged

Mike Kasprzak | Sykhronics Entertainment - Smiles (HD), PuffBOMB, towlr, Ludum Dare - Blog
inmatarian
Level 0
*


2993 forever


View Profile
« Reply #24 on: July 15, 2007, 10:25:02 PM »

If you want to link against zlib, you could also use their inflate and deflate functions. I have a generic inflate function here, which should be sort of the same with a deflate. (the error codes were removed to make it terse, but you could find them in the documentation)

Code:
int unzip( const char * from, int fromlen, const char * to, int maxlen )
{
  int ret;
  z_stream zstrm;

  /* allocate inflate state */
  zstrm.zalloc    = Z_NULL;
  zstrm.zfree     = Z_NULL;
  zstrm.opaque    = Z_NULL;
  zstrm.avail_in  = fromlen;
  zstrm.next_in   = (Bytef*) from;

  /* Start Stream (Allow GZIP and ZLIB Headers) */
  ret = inflateInit2(&zstrm, 15 + 32);

  /* Set Target */
  zstrm.avail_out = maxlen;
  zstrm.next_out  = (Bytef*) to;

  /* Decompress */
  ret = inflate(&zstrm, Z_FINISH);

  /* Shutdown */
  inflateEnd(&zstrm);
  return 1;
}
« Last Edit: July 15, 2007, 10:27:26 PM by inmatarian » Logged
DrDerekDoctors
THE ARSEHAMMER
Level 8
******



View Profile WWW
« Reply #25 on: July 15, 2007, 10:42:30 PM »

If you can blow chunks out of it in the game ...

Well then, don't hint at "blowing chunks" out of anything, whilst supporting the idea of RLE.  Tongue

Wuh? Wuh? Wuh?

I don't see why we're arguing here. We're talking about saving to disc versus storage in real time. In real time the data sits in an array, you blow holes out of it or otherwise deform it. As you blow holes out of it, it gets more complex to a point, but probably not massively.

THEN you need to save it to disc. At that point you write it out in an RLE format. It takes up a few K at most and it's forgotten about.

THEN you need to load it, you read it in, in RLE format and blow it up to the usual full-sized array.

It's only ever in RLE format when it's on the disc, which is when it's not being updated. How is this a problem?
Logged

Me, David Williamson and Mark Foster do an Indie Games podcast. Give it a listen. And then I'll send you an apology.
http://pigignorant.com/
handCraftedRadio
The Ultimate Samurai
Level 10
*



View Profile WWW
« Reply #26 on: July 16, 2007, 05:28:27 AM »

Thanks a lot PoV! Grin Your method works perfectly. Down to 96.0kb. Problem solved. Thanks for helping me out, everyone.


Logged

PoV
Level 5
*****


Click on the eye and something might happen..maybe


View Profile WWW
« Reply #27 on: July 16, 2007, 11:02:37 AM »

I don't see why we're arguing here. We're talking about saving to disc versus storage in real time. In real time the data sits in an array, you blow holes out of it or otherwise deform it. As you blow holes out of it, it gets more complex to a point, but probably not massively.

Wow!  Oh noes!  I am wrong!   Embarrassed

For some reason I neglected the whole idea that this is static on disk, and not some memory efficient system too.  Sorry Graham.  I'm completely at fault for this stupid debate.

Can we still be friends??  :D :D
Logged

Mike Kasprzak | Sykhronics Entertainment - Smiles (HD), PuffBOMB, towlr, Ludum Dare - Blog
DrDerekDoctors
THE ARSEHAMMER
Level 8
******



View Profile WWW
« Reply #28 on: July 16, 2007, 12:09:12 PM »

Aaah, I assumed you had to be reading it wrong but I was trying not to sound utterly patronising. But if you'd persisted in your vein the next step was writing some code with offensive comments in it. Wink

:D We're still bestest chums. Smiley
Logged

Me, David Williamson and Mark Foster do an Indie Games podcast. Give it a listen. And then I'll send you an apology.
http://pigignorant.com/
Chris Whitman
Sepia Toned
Level 10
*****


A master of karate and friendship for everyone.


View Profile
« Reply #29 on: July 16, 2007, 12:11:06 PM »

Not to flog a dead horse or nothing, but I would still really recommend using run length encoding. 96K per level is pretty wasteful for a two colour level, especially if people are going to be downloading it. I guarantee you could cut that down by a factor of ten, at least.
Logged

Formerly "I Like Cake."
DrDerekDoctors
THE ARSEHAMMER
Level 8
******



View Profile WWW
« Reply #30 on: July 16, 2007, 12:13:42 PM »

Hey, don't forget about endianness, you jerks.

Bloody Mac users. Wink
Logged

Me, David Williamson and Mark Foster do an Indie Games podcast. Give it a listen. And then I'll send you an apology.
http://pigignorant.com/
DrDerekDoctors
THE ARSEHAMMER
Level 8
******



View Profile WWW
« Reply #31 on: July 16, 2007, 12:16:45 PM »

Yuss, assuming you go with a 7-bit counter and 1-bit data configuration, then as long as your runs of colour are longer than 8 pixels on average, you'll save data over a regular 1-bit bitstream. However if it's gonna' have really long areas of solidity, you might want to go a longer 15-bit counter and 1-bit data configuration. Or if you want to be really clever about it, go with an n-bit counter and 1-bit data setup and bitstream that. Although that's possibly overkill.

Obviously the moment you decide that you'd like to have some concept of grades of damage to the landscape, most of the advantage of RLE will go pissing out the window. Wink
Logged

Me, David Williamson and Mark Foster do an Indie Games podcast. Give it a listen. And then I'll send you an apology.
http://pigignorant.com/
Chris Whitman
Sepia Toned
Level 10
*****


A master of karate and friendship for everyone.


View Profile
« Reply #32 on: July 16, 2007, 12:44:17 PM »

If you're using an 8 bit RLE scheme, it would only be inefficient if your average run was shorter than eight. Even if you were taking chunks out of the landscape, you're still preserving average run length because you're reducing black and adding white. Lots of big runs of white are still going to mean very efficient encoding. Remember that, even if it's locally inefficient at one or two points, it could still be globally efficient.

The worst case would be one bit on, one bit off, one bit on, one bit off, etc. Somehow I don't think that is likely.
Logged

Formerly "I Like Cake."
PoV
Level 5
*****


Click on the eye and something might happen..maybe


View Profile WWW
« Reply #33 on: July 16, 2007, 12:47:21 PM »

Screw the 1 bit data part.  Grin.  Just alternate colors A and B (0 and 1), and use that bit for longer runs.  Zero is your magic run length that puts nothing down (255 0 138 for 393 pixels of color A).  And if you really want to be special, make 255 a magic number that notes the 2 bytes following are a 16 bit run.  This is nice because it's the same size as having to throw down a 0 for a longer run, but you gain 16 bit resolution as needed.

And if you will have higher detail short runs, you could use nibbles.  4 bit 0-15 for color A, and 4 bit 0-15 for color B, all packed in to a byte.
Logged

Mike Kasprzak | Sykhronics Entertainment - Smiles (HD), PuffBOMB, towlr, Ludum Dare - Blog
Alec
Level 10
*****



View Profile WWW
« Reply #34 on: July 16, 2007, 02:43:11 PM »

You don't see many games being developed for the Atari 2600 anymore...
Logged

Lancer-X
Level 0
**


internet hate machine


View Profile
« Reply #35 on: July 16, 2007, 06:53:53 PM »

Nowadays, unless you're truly storing an insane amount of very specific data, I wouldn't be too concerned about agonising over your own home-cooked compression methods. Just shove the whole thing through DEFLATE and be done with it. Most byte-wise redundancy will be eliminated by the LZ77-style dictionary compression, and the bit-wise redundancy by the Huffman compression that follows it.
« Last Edit: July 16, 2007, 06:58:50 PM by Lancer-X » Logged
Chris Whitman
Sepia Toned
Level 10
*****


A master of karate and friendship for everyone.


View Profile
« Reply #36 on: July 16, 2007, 10:35:13 PM »

It's not really a huge deal; I just figured that with all that fairly homogeneous data, you could save a little space with relatively little effort using RLE. Alternately, you could just use an image format like PNG which already uses DEFLATE.

I just figured, you know, it's a good learning experience for a beginner to try coding possible solutions yourself and all that.
Logged

Formerly "I Like Cake."
Lancer-X
Level 0
**


internet hate machine


View Profile
« Reply #37 on: July 17, 2007, 02:28:29 AM »

Yes, it is true that you save some data (sometimes vast amounts of data!) using RLE, depending on what the data is like and exactly how you RLE it. (for example, if you have a tile-based game with multiple bytes of data per tile, it may be more efficient to store them planar and compress each plane). However, the point is that you essentially have to spend extra time thinking of something that is a trivial concern. The extra overhead involved in inflating the data is almost unnoticable nowadays (even consoles like the GBA have built-in lz77 and huffman compression code in the BIOS), you don't have to think about the data you're compressing that much to get modest savings and in the end you save a lot of time not worrying about fancy storage methods, and can spend the time actually getting the game made =p
Logged
PoV
Level 5
*****


Click on the eye and something might happen..maybe


View Profile WWW
« Reply #38 on: July 17, 2007, 04:38:30 AM »

You don't see many games being developed for the Atari 2600 anymore...

Well excuse me princess.  Tongue
Logged

Mike Kasprzak | Sykhronics Entertainment - Smiles (HD), PuffBOMB, towlr, Ludum Dare - Blog
Alec
Level 10
*****



View Profile WWW
« Reply #39 on: July 17, 2007, 04:54:27 AM »

Huh?
Logged

Pages: 1 [2] 3
Print
Jump to:  

Theme orange-lt created by panic