next »

[Klaim]

I think Rob Lach was only suggesting we WRITE the tool in python, like in PyQt (http://www.riverbankcomputing.co.uk/software/pyqt/intro). That would be a really good idea. The point it to write it easily than in C++ but get an executable binary in the end.
The user don't need python, it needs the final executable only.

For example, TortoiseHg 2.x is now using Qt  (instead of Gtk). I totally wouldn't make it with Gtk. I'm using it at my dayjob and it's just hell.

I agree that using PyQt or something similar would make life simpler.

Possible, although we'd need to agree on a small standard set of functionality; is there a portable library for Lua that will suffice?

Errr... Lua implementation IS a library, some C files in fact, that you can easily embedd in any C-compatible language. It's so lean I used it a lot on NintendoDS and other embedded devices.

The point is that most game developers would know (or want to know) about lua for some reasons, and even some build systems (premake and bam) does use lua for a lot of good reasons (related to the game developpement).

So I think that would be a good choice too. Python+Qt for the tool code, lua for the user scripts.


For the file compression format, forcing your users to use what you provide is a realy bad idea, even if you use standard files. You should provide  default format, yes, implicitely used until the user wants the tool to use another format. Then we have to provide a way to let him plug something to convert the files for him. No extension point means alienating your users.

[technogothica]

I have some questions regarding Qt...

1) Can Qt be statically linked? This question is especially relevant to Python + PyQt

2) Can you provide me/us with sufficient proof that using a Qt based solution in a commercial game project will not incur a licensing fee?

3) Why Lua when you are using Python? Python is a script engine, yes?

[Klaim]

1) Can Qt be statically linked? This question is especially relevant to Python + PyQt

Yes but it requires another licence, if I understand correctly. Or maybe that have changed. PyQt I don't know if it can produce just one exe.

2) Can you provide me/us with sufficient proof that using a Qt based solution in a commercial game project will not incur a licensing fee?

Well the licence is clear enough, see the table on this page : http://qt.nokia.com/downloads
Or better : http://qt.nokia.com/products/licensing

So as far as Qt is a separate non-modified binary from your game, it's gratis.

The problem is that indeed suppose that this LGPL licence is available ONLY if Qt is in separate binaries, so dlls.

That said, there are ways to embedd executables and their dlls into only one thin executable, (I don't remember the name of the free tool I think about).

3) Why Lua when you are using Python? Python is a script engine, yes?

That would work too but embedding Python in a binary have a bigger cost than lua on the binary size. And also on execution speed but I guess we don't care here.

That said, I don't know if using PyQt allow easier embedding of Python in your final binary.

[Klaim]

By the way, I think Qt is best fit only because it allow flexible customization and is more clean than other GUI libraries I've used so far (Gtk/mm, MFC and others) (in C++).

If there is a very customizable simpler thinner alternative, I thing it would be even better. Maybe FLTK? Not sure, it really looks like "too" simple.

[mcc]

Oh, I didn't realize Qt had moved from GPL to LGPL. When did that happen?

[technogothica]

The problem is that indeed suppose that this LGPL licence is available ONLY if Qt is in separate binaries, so dlls.

There's the rub This question needs a definitive answer before proceeding down the Qt road.

BTW, I did some research on py2exe and the resulting executable requires python??.dll to run. Not suitable

http://www.py2exe.org/index.cgi/FAQ

Still looking around...

[mcc]

The problem is that indeed suppose that this LGPL licence is available ONLY if Qt is in separate binaries, so dlls.

There's the rub This question needs a definitive answer before proceeding down the Qt road.

BTW, I did some research on py2exe and the resulting executable requires python??.dll to run. Not suitable

http://www.py2exe.org/index.cgi/FAQ

Still looking around...

What about cx_Freeze?

Also does this help?

http://www.pyside.org/

I unfortunately am not quite familiar enough with the python/qt/lgpl problem to comment intelligently here :|

My distaste for .tag.gz is that the entire archive has to be decompressed into an additional temporary file before you can get to the files inside.

One thing worth noting, libarchive can read and write both tgz and zip and it's really easy to integrate.

[mcc]

As OS X goes, RPMs are used for everything, so an app package in an RPM that can be dragged onto the desktop is an acceptable way of doing things.

*scratches head* Is this a typo?

[Evan Balster]

I suspect the python/Qt solution may be a can of worms we don't want to open, especially for a small-scale project which can be easily written in native code.  And to state it more clearly, I think it's best if we write these launchers as separate applications; their functionality is very platform-dependent.

For GUIs, we might as well use platform APIs if we have experts in them.  Failing that, wxWidgets is a decent and very lightweight library.  I've heard a few negative things about it, but nothing that would ought to make a simple progress bar problematic to implement.

Incidentally, if no one else is spearheading the Linux version, I will.  I know enough about Linux dev, but I'm not very well-versed in matters of compatibility or distribution, so again there may be those more qualified than I.


Klaim:

For the file compression format, forcing your users to use what you provide is a realy bad idea, even if you use standard files. You should provide  default format, yes, implicitely used until the user wants the tool to use another format. Then we have to provide a way to let him plug something to convert the files for him. No extension point means alienating your users.

I think that since the 'users' of the tool in question are technically capable folk (game and application developers) restrictions on compression format are acceptable.  That said, if there's a good library available, zip might be a good choice for superior accessibility.


Technogothica:

If we are to support multiple file resources, it would probably be good to version them separately.  So maybe a format like:

Code:

win32:
binary.pak 0.3.0 d6f89cf7489a349f http://www.server.com/game/download/binary-win-030.pak
osx:
binary.pak 0.3.0 d6f89cf7489a349f http://www.server.com/game/download/binary-osx-030.pak
extraThing.pak 0.2.3 c468f9b94684e798 http://www.server.com/game/osx/extrathing-023.pak
all:
graphics.pak 0.1.0 c468f9b94684e798 http://www.server.com/game/download/graphics-010.pak

I'm still inclined towards keeping the specification small, so we might want to forgo the hash (not that it's a huge requirement) or the ability to have separate archives.

For the most initial version, we might even hold off on the lua scripting in favor of something as simple as a system() call, which could go to a shell script if desired.

As OS X goes, RPMs are used for everything, so an app package in an RPM that can be dragged onto the desktop is an acceptable way of doing things.

*scratches head* Is this a typo?

RPMs are the standard way of distributing applications on OS X, correct?  In any case, the downloading and running of the launcher should be as simple as possible.

[jmp]

BTW, I did some research on py2exe and the resulting executable requires python??.dll to run. Not suitable

It doesn't if you set the bundle_files option to 1 ("bundle everything"):
http://www.py2exe.org/index.cgi/SingleFileExecutable

This includes the Python interpreter in the resulting executable.

[eclectocrat]

RPMs are the standard way of distributing applications on OS X, correct?  In any case, the downloading and running of the launcher should be as simple as possible.

You're thinking of DMG's, but whatever.

Looking at the requirements it seems to me that the only platform code is:
Networking/HTTP requests. < Lots of cross platform stuff available.
Filesystem management. < Easy as pie.
Splash screen/Progress indicator < Tiny bit of platform specific GUI mucking.

I still think that Python is a good way to go because it's batteries included, ie: the standard python distro has everything needed built in (you can say the same thing about QT, but I'm paranoid of Nokia's legal dept.). Also, one source trunk is easier to maintain than 3. Double plus I am a hundred and thirty percent sure that you can package a no external requirements executable.

So I'm still going ahead with my prototype, which I have written between bouts of frustration over mysterious castle, and consists of less than 20 lines of code that downloads a file over HTTP and saves it in a temp directory. Haven't used Python in over a year, and trying to update my skills to 3.x version, but it'll be a couple of days (depends on how pissed of I get with my game!) before I post it on GITHUB.

Either way, I can contribute the OSX native code version if that is the consensus. I've already looked into it and it's really simple.

[JobLeonard]

This seems like a great idea for a project, and something that could be used more widely than just games if done properly.

So uhm... a suggestion by someone who doesn't actually really know what he's talking about: have any of you looked into delta encoding?

Delta encoding is a way of storing or transmitting data in the form of differences between sequential data rather than complete files; more generally this is known as data differencing. Delta encoding is sometimes called delta compression, particularly where archival histories of changes are required (e.g., in software projects).

I don't know much about the subject, I recently learned about this through a reddit post about Chrome Courgette and never had a reason to look more deeply into it myself. However, using a smart algorithm to make patches only a fraction of the full binary size seems like a good idea for multiple reasons (faster to download, indy publishers save money because of the reduced data bandwith).

Now, Courgette is optimised for Google Chrome, but the bsdiff tool appears relatively straightforward, and it's cross-platform on Linux and Windows:
http://www.daemonology.net/bsdiff/

For those who are interested, the Courgette source:
http://src.chromium.org/viewvc/chrome/trunk/src/courgette/

[technogothica]

I suspect the python/Qt solution may be a can of worms we don't want to open, especially for a small-scale project which can be easily written in native code.  And to state it more clearly, I think it's best if we write these launchers as separate applications; their functionality is very platform-dependent.

I'm behind you 100% on that one. Window toolkits are overkill, especially for a single modal dialog box.

But the guys who want to try Python should still do so if they feel inclined. I'm interested to see what they come up with.

I'll contribute to a native Win32 version.

If we are to support multiple file resources, it would probably be good to version them separately.  So maybe a format like:

Agreed.

I'm still inclined towards keeping the specification small, so we might want to forgo the hash (not that it's a huge requirement) or the ability to have separate archives.

md5 hashes are a reliable and efficient way of comparing files without having to download them and do a byte-for-byte comparison. Do an md5 hash of the local file, and compare the hash string to the one in the text file. Comparing two 32-character strings is trivial. The algorithm doesn't have to be md5, but it's available everywhere and relatively reliable in this scenario.

For the most initial version, we might even hold off on the lua scripting in favor of something as simple as a system() call, which could go to a shell script if desired.

Agreed.

[technogothica]

So uhm... a suggestion by someone who doesn't actually really know what he's talking about: have any of you looked into delta encoding?

A good feature to put on the version 2.0 to-do list.

[Evan Balster]

Eclectocrat -- As long as the python EXE doesn't barf DLLs all over the user's desktop when it's run, I'm chill with it and it's probably a good thing to move forward with.  You did, for whatever it's worth, forget one last important platform-dependent item: the location of the application data directory.  It varies between versions of windows, and is in strange places on OSX.  That's the one I worry about, though it shouldn't be too much of a problem.

If it's relatively simple and won't take long, I'm game to try out this Python version.


Technogothica -- The MD5 is probably fine; you're right.  I'm just very wary of us making more than the bare minimum for this tool.

[drChengele]

This project is a great idea. Python and network programming are not my strong points, but I would like to lend a hand somehow, at least with testing the thing.

[technogothica]

Yesterday my game development ground to a halt due to a combination of artist's block and a speed limited DSL. So I decided to have a go at a prototype for the native Win32 version.

I went with the WinInet functions rather than Winsock for the following reasons:

(1) It takes far less code
(2) InternetOpenUrl() transparently supports HTTP, HTTPS and FTP
(3) InternetOpenUrl() also transparently handles HTTP 3xx redirects.
(4) WinInet is available in all versions of Windows since 95/NT4

So far my prototype downloads files and does md5 hashes on them. My next job is to add support for that version info text file.

[Evan Balster]

Keep up the good work!

[eclectocrat]

Work in progress. Parses text file, downloads latest version to temporary file, checks hash, shows progress indicator while downloading. I'm literally learning python 3 as I write this, so this is definitely not done in the one true way(TM), but it's really clear and concise code, very small easily understandable file.

https://github.com/eclectocrat/AutoUpdate

Anyways, I feel really guilty about spending the morning reading python docs instead of working on my game so, I'm gonna get back to work, and return to this when I need a holiday from my f*xing gorramn sh*t bastard filter persistence code.

Aaargh.

[mcc]

A warning about the python+qt concept is it seems like it will tend to produce very big binaries just for the updater. However it still sounds really useful as a way of getting a rapid prototype together.

One thing I never quite got. Do you really need a gui as complicated as qt? If you need lots of stuff like displaying webpages and patch notes then sure, but if all you do is throw up an "UPDATING..." progress bar at startup sometime then integrating qt might be more work than just making simple OS-specific gui window things.