next »

[RalenHlaalo]

I'm reading through the book Game Engine Architecture by Jason Gregory, and he talks about providing the illusion of a seamless game world by dividing the game's resources into chunks. However, it doesn't describe the process in enough detail for me to confidently implement (or perhaps I've missed something). I'm hoping you guys can point me in the right direction.

From what I gather, the basic idea is to define a set of overlapping regions on the game world. The resource manager asynchronously loads the chunks within the current region. Because the chunks are all the same size it can allocate the memory from a pool. He states the chief motivation for this as being to avoid memory fragmentation, which could result in the situation where your new/malloc calls fail even though you think there should be enough RAM left (there's also the expense of context-switching).

I have a few questions about this. He says for this to work your resources should be inherently 'chunky'; you can't have a contiguous array spanning multiple chunks, obviously. How would the engine instantiate a chunky mesh or texture? Would it wait for all the chunks to come in, then allocate enough memory for the whole thing and copy the data over? That would surely result in greater memory consumption - and there would have to be a separate pool with chunks big enough for the whole resource. Presumably, the ideal is to instantiate a resource into the same chunk that its data was loaded into?

So would the runtime representations of the resources have to be explicitly chunky, e.g. a mesh class that contains a list of its chunks, a texture class containing a list of its chunks, etc.?

I was rather hoping to ignore complex memory management strategies until later (no premature optimisation). Will I have to develop the engine from the ground up to support chunky assets?

Thanks

[zuzu]

Hi there. I read exactly that part of the book and pondered this for a while. He is commenting from AAA experience with *tons* of art assets, especially from experience with consoles that have very slow disk read speeds. Disk reading was literally a bottleneck for many AAA games not that long ago. Nowadays majority of games have giant on-line downloads and updates, so disk seeking is a non-problem.

Personally I use a fairly naive asset loading system, and I keep track of how much memory I use. I set a hard-RAM limit of 500MB, and I use a least-recently-used cache to evict assets anytime I need some space to load something.

Asset streaming is dead-simple. It only requires 2 threads. The main thread requests and asset to be loaded. If it is loaded, the asset system gives the main thread the asset. If it is not loaded, it gives the main thread a "placeholder". Like an empty sound, or a checkerboard texture. This kicks off the other thread to do actual loading from user disk.

The main thread can use the placeholder, but keeps asking for the real asset. The moment the second thread is finished it will give the asset system the asset. The main thread will soon after request and receive it.

This is all documented quite well on HandmadeHero.

Edit:
Memory fragmentation is also mostly a non-problem for PC. Consoles can have problems with fragmentation if they do not have virtual memory support. Also if you run a server that needs to last many years, fragmentation avoiding strategies should be used. Otherwise it really does not matter much.