next »

[Dacke]

Greetings! I'm currently making a game from the CPB-compo called Homecoming.

I am trying to have the game load textures in a separate thread. Since OpenGL uses a lot of global states, this is quite difficult.

I've found quite a few articles and forum discussions about this problem, but I've been unable to turn their instructions into actual code.

What I think I need to do (but have no idea how to) is to create a new OpenGL context for the loading thread. But as I'm using glut I didn't even have to create a context in the main thread and don't know how to do this in the second one. Can I do it in a platform independent way, possibly using glut?

I'm only compiling for windows at the moment, but have so far managed to avoid windows-specific code.
I am using:

• C++, with standard libraries.
• OpenGL, only using basic stuff.
• glut, which does all the system-specific OpenGL stuff for me. It might be causing some trouble on it's own. Will it allow me to create new OpenGL contexts?
• soil for loading images. I'm not sure if it's thread safe. I think it would work if I made it work in a separate OpenGL context? I am prepared to use another library if anyone can give me good suggestions.
• pthreads-win32 for creating threads. It is basically pthreads but for windows.

Some articles on the subject:
http://hacksoflife.blogspot.com/2008/02/creating-opengl-objects-in-second.html
http://support.microsoft.com/kb/128122
http://www.gamedev.net/community/forums/topic.asp?topic_id=424705
http://www.gamedev.net/community/forums/topic.asp?topic_id=395910&whichpage=1%EF%BF%BD
http://www.gamedev.net/community/forums/topic.asp?topic_id=394701&whichpage=1%EF%BF%BD
http://www.equalizergraphics.com/documentation/parallelOpenGLFAQ.html

Thanks!

[Hideous]

I don't really know anything about this, but can't you mix in SDL threads, perhaps? I'd say those are multiplatform enough, right?

[Hajo]

SDL threads, perhaps? I'd say those are multiplatform enough, right?

SDL uses pthreads, I believe.

[bateleur]

SDL threads, perhaps? I'd say those are multiplatform enough, right?

SDL uses pthreads, I believe.

But that's not really the problem...

It's not that finding/using a thread library is hard, it's that writing code which supports multithreading is hard.

The pThreads stuff will let you create a new Thread and call a function from it, but done incautiously this will just break everything because many libraries do not support concurrent calls from multiple Threads.

(Apologies Dacke, I don't know OpenGL well enough to suggest a solution.)

[Ivan]

I am pretty sure that you'll need the same context even if you do multiple threads.

[Bad Sector]

Although its by no means 'free', sending the data to the video card is by a magniture faster than loading (and usually decompressing it) from disk. So what you need to multithread is the loading part (assuming of course that the OS can take advantage of it and the loading is done from a hard disk - a CD ROM drive contains a single head so it will probably be slower). Once you have everything you need loaded, send them to OpenGL from your "opengl" thread (that is, the thread that made the context).

As the http://www.equalizergraphics.com/documentation/parallelOpenGLFAQ.html FAQ says, its recommended to use a single thread which takes care of communications with the video card and do the CPU-intensive parts in other threads. In fact this should be more a requirement than a recommendation because so far no graphics driver supports multithreaded OpenGL applications and such a program might produce unexpected behavior (like freezing the whole system, if your vendor is ATI :-P).

[LtJax]

You have two options. Either do what Bad Sector said and only use OpenGL from one thread. This is actually what I recommend. Or you need to implement your own locking facilities for your OpenGL code - and only issue GL commands if you get the lock.
On a side note, it's very likely that your driver is already doing multithreading for the real GL work, and just scheduling your commands really. However, loading is indeed something this cannot account for, as the input for the TexImage calls etc must be read when the call returns. Your best option if you want interactivity during long loading procedures is to use something like a coroutine, that only does some partial loading (preferably while the CPU is waiting for the GPU to catch up). This is a bitch with C++, since you don't have yield, but it can be done (and has been done)!

[Dacke]

Thanks for all the answers!

To begin with I'll try to clarify some things.

What I am able to do:
I can create new threads. I can share objects between these threads and put locks on them when needed.

What I want to do:
Load textures from image files without having the game freeze during the loading process. Loading a texture takes approximately one second.

How I'm trying to do it:
Putting the loading process in a separate thread.
(Note: I do not intend to do any drawing from the loading thread. I just want to handle the loading of the image files from the secondary memory (hard drive) to OpenGL textures in the primary memory (RAM)).

How I think it should be done:
According to this article http://hacksoflife.blogspot.com/2008/02/creating-opengl-objects-in-second.html (and the others I have found) there is one way to go with this. You create two separate OpenGL contexts, one for the thread that loads and creates the objects (like textures, vertex buffer object etc.) and one for the rendering.. Once the loader-thread has created the objects it "sends" them to the main thread, where they are used.

How platform independent code for this could look:
eerrrrr... eehhh... ummmm.. ?
As soon as I try to use the image loading code in the new thread I get a segfault
I don't know how to create a new OpenGL context in the new thread either, glut doesn't seem to allow it..?
Help!


You have given me suggestion for some alternative solutions:

Hajo suggested that I could use SDL. It's quite possible that SDL has support for multi-threaded loading. Have you used SDL for loading OpenGL texture objects or do you know of a tutorial dealing with this?

LtJax came with a very interesting suggestion. I just realized that I can use photoshop to cut up the images in tiny pieces (using the slice tool). This way I can load a tiny piece of the texture every frame, preventing the game to completely freeze. In no way an ideal solution, but it's currently my main back-up-plan.

[LtJax]

Just to clarify that, you don't need to cut up the textures manually. You can just read a few rows of the image and upload them (via the TexSubImage calls or PBOs), and then update your screen.

[Dacke]

Indeed. I did understand your suggestion but I have no idea how to actually do it your way.

I'm using the soil library which loads one image at a time. Splitting the image in small pieces takes about 11 button presses in Photoshop. Relatively speaking, keeping track of hundreds of small texture pieces will be a breeze too.

But if you can tell me how to read stuff in pieces that would be really cool.

[LtJax]

Oh, maybe your library doesn't support it. You either need to write the code for that yourself, or use a library that supports it. For example, libpng supports this, but then it's only loading png files, no uploading or anything. If you want this really fast, you should probably write it yourself, since there's a few ways to make this really fast now - but those are specific and I don't think there's a good library out there that does the top-notch stuff.
Personally, I really wouldn't want to split the images up manually. It just seems wrong.

[muku]

What I want to do:
Load textures from image files without having the game freeze during the loading process. Loading a texture takes approximately one second.

One second? That sounds excessive. How large are your textures?

I think Bad Sector has made a very good suggestion. It's quite possible that the file I/O and decompressing take the major amount of time when loading a texture (measure it!), so if that's the case, try doing that in a separate thread first. That should be quite easy to implement and you don't have to worry about multiple OpenGL contexts.