next »

[technogothica]

How does MineCraft works in that case? Put everything in ~/.MineCraft ?

I've no idea. I need to sit down and have play of Minecraft! There's just not enough hours in the day

Anyone had a chance to muck around with the launcher/updater? Any thoughts?

[Klaim]

I took a look at the Minecraft download page, it's simply a .jar (java archive) file so the user can install it wherever he want I guess? or running the .jar will download the data somwhere safe.

I'll try it and tell you.


Another interesting thing : the minecraft launcher is the minecraft game window. I think it's not a good "general" example so a lot of other examples would help better.

[Jase]

You download the launcher java app from the download page and put that wherever you want. When you run it, it downloads the actual game files and puts that in AppData/roaming/.minecraft

It looks seamless from the users point of view, in fact, I didn't realise minecraft even used a launcher until Notch (minecraft creator) released an updated one and my launcher didn't change.

[Klaim]

Jase> We did discuss about it in the last pages of this thread
What we don't know is where it put files on Linux.

[Jase]

oops :S

That'll teach me to skip pages.

[dredogol]

*edit* WOW... that is one ugly MANBABY,  

Hey guys, I'm new to the forums, but I've been reading this entire thread just now; wanted to give my developer and end-user input:

I think it's great we're trying to make a "generic" updater for others to use for whatever application (program and/or game updates).

I'm going to try to address all of Cellulose's OP design wishes, as well as those proposed up until this post.


=== Multi-platform Design ===
The best way to go about this is have a different updater for each OS.
Not only will the overall file size be smaller, but less prone to bugs/issues/resigns if future OS structure designs change.
Would also be kind of dumb, if I had to update my updater, because a new MacOS/Linux came out,when I'm using Windows.


=== Minimal Design ... Necessary Files ===
If we're talking about the core files, we'll need:
1) Launcher.exe
2) Config.ini (whatever extension)

The reason we need a config file, is because we never know if/when the client needs to change the http address, or whatever. If you don't want users to tamper with it, you can keep the file hidden during launch so that only the developers know about it.
The Config file should also contain the last update version.
The file update list (manifest) should be downloaded from the server, and later deleted after update is complete.

The program name and those kind of details need to be edited by the developers before they build the solution.

Icons can be built into the binary file, so no need for (.ico) files.

As far as staying invisible after finishing updates...
Option 1:
The updater should be the parent process, stay launched in an IDLE state, and launch the program in question as a child process. The developer will be responsible for proving the launcher an "exit exception", so the launcher knows the main program ended, and it too needs to close.
Option 2:
The updater should be the parent process, launch the program in question as another parent process, then close itself. (I THINK this option is possible...)



=== Archives / File Compression ===
For a FREE one... I too believe (.zip) files are the way to go. They are universal across all platforms, are archives, as well as file compressors. You can also password encrypt the files, in case you believe they should be... though I see no reason to do so.


=== Launcher & Installer (combined) proposal ===
This was a discussion early in April, but)I believe there should be 3 completely different modules to this entire project idea:
1) Game exe
2) Launcher exe
3) Install/Uninstall exe

The reason I believe the launcher and installer should be separate, is because not all program developers may like the installer we provide.
If anything, the launcher should have an "Uninstall Button", and nothing more.
What type of uninstaller the developer chooses (custom or standard) is up to them.
This keeps the entire design concept more modular, less prone to bugs, and smaller in size.
Less work on our end, and gives us a broader client base if this open source patcher/updater is launched.


=== Client / Server updater (http) ===
I just saw your program technogothica... great work!  
I too believe the MD5 checksum IS the way to go in terms of checking for data integrity when patching files.
I also agree that writing the update list (manifest) in XML would be better than standard Text format. I too experienced the same issues technogothica had in the past with standard text files.
XML files may be larger (not by a lot), is well used in today's industry, and IS robust enough for this situation.

There was also talk earlier about keeping the command line on the server end... I honestly think that is a bad idea, unless you're making a browser based game, or a game that requires connection to the website to launch; which is very client specific (small client base). The thing would break if offline.


=== Auto Updater? ===
The updater should ONLY be launched when the user wants to launch the program.
I honestly HATE those auto updaters that run in the background or at startup 24/7. I also edit my registry and disable those evil things.
Now, about the auto updates...
Instead of always checking for updates and auto updating, I think we should give the developer (not client) "options" to allow this "feature" or not. (IE. Firefox auto updates features).
The developer may want to force an auto update (like MMORPG games), make it periodical optional (like browsers), or completely optional (offline programs in which the client/user has to force an update).


=== Update Directories (folders) & User Security Issues ===
These are the toughest questions...
I understand that Cellulose doesn't want to have an updater that deals with the varying file directory structures, as well as deal with user security levels.
The "easiest" solution, is to allow the developer to make their product, include the updater program in their INSTALLER, and let the installer modify the registry to allow all users to access the updater / files.
That way, there are no issues with the updater having to deal with user permissions when adding/removing files.

However, if the developer has no installer, and the product they have is only a few binary files, but still requires access to an updater... now we have problems.
This is more of a client/developer issue, because they will be 2 completely separate programs with no common directory. Actually, nothing we can do about this problem.
The (end-user) will need to copy all the files where the UPDATER developers mandate the files to go for everything to work properly, not the actual client/program-developers.


=== GUIs ===
This is completely in the air...
Most game developers want a "nice looking" updater with a lot of "bling bling", so their clients can see/do something during the updates, and not be stuck staring at a boring progress bar.
Commercial programs on the other hand are looking for more professional (streamlined) updaters that use as little memory/space as possible.

We could give the client here, again, the option for a simple progress bar, or a fancy GUI they can customize.
Unfortunately, this means more coding, and a much larger binary file... even if the client wants a simple updater.

I know Cellulose said he wanted this updater to be as small and lean as possible, but let's be honest... updaters are NOT that large in file size, and 1-2MB extra due to a better GUI option won't hurt. I honestly believe giving the client 2 options (advanced graphic GUI / simple progress bar and cancel button) is the best way to go if you want the broadest client base.
It's not like we're making something for a mobile device where very BYTE counts.

==============================================

I think that's all the major points I wanted to address.
If there are a few I missed, I'll make another post.

Oh... is there anything I can do left?  
I think technogothica pretty much finished the first Alpha of this updater.
I'm not fluent enough with Linux OSes, and nothing for MacOS.

[technogothica]

Awesome summary dredogol Very helpful too - will assist with developing the docs and working out what changes are needed for the next version. Keep up the good work!

The reason we need a config file, is because we never know if/when the client needs to change the http address, or whatever.

Typically, you will want to use a dedicated IP address for the lifetime of the product. You can set up Apache to redirect to another URL, and the XML manifest file can also point to files on another server, so you have plenty of options available.

One option that could be added, though, is the ability to provide a directory of alternative IP addresses in the XML manifest file. All new builds of the updater will use the hardcoded address, but maintain a local database of alternate addresses that it collects on each update. If one address fails, it can try and contact the other addresses.

I've actually been toying with the idea of introducing an extra tag into the XML where you can specify a message box to show the user on startup:

Code:

<message>
A new version of the launcher/updater is available. Do you want to download it?
</message>

There was also talk earlier about keeping the command line on the server end... I honestly think that is a bad idea, unless you're making a browser based game, or a game that requires connection to the website to launch; which is very client specific (small client base). The thing would break if offline.

Not sure what you mean, but you may be confusing the role of the update manifest XML file generator (umgen). The current implementation is a command line tool which allows it to be used in batch scripts - allowing you to automate the build process. It was also an ad-hock thing that I slapped together because generating MD5 hashes by hand is tedious. You could easily replace this with a GUI version. I just didn't bother because it would have taken too much time on something that wasn't a high priority at this early stage.

=== Update Directories (folders) & User Security Issues ===
These are the toughest questions...
I understand that Cellulose doesn't want to have an updater that deals with the varying file directory structures, as well as deal with user security levels.
That way, there are no issues with the updater having to deal with user permissions when adding/removing files.

I've been putting some serious thought into this over the past few weeks. I'm very much in Klaim's camp on this issue, and I'm hell-bent on solving it.

For now, the AppData directory is a convenient solution, and Minecraft is a strong proof of concept supporting the idea.

The "easiest" solution, is to allow the developer to make their product, include the updater program in their INSTALLER, and let the installer modify the registry to allow all users to access the updater / files.

Unlocking the Program Files folder for non administrator access is a very bad idea. It defeats the whole reason for that lock in the first place (protecting essential program files from viruses and idiots).

Oh... is there anything I can do left? 

Absolutely! This Win32 beta is seriously in need of a decent tutorial, and I'm looking forward to some feedback on testing. Plus, do you know anything about SourceForge? SVN? It would be good to have someone who could deal with the day-to-day administration of that side. As much as I would like to do it, I only have a limited amount of time to devote to this project, and that would be best spent on coding.
 

[dredogol]

There was also talk earlier about keeping the command line on the server end... I honestly think that is a bad idea, unless you're making a browser based game, or a game that requires connection to the website to launch; which is very client specific (small client base). The thing would break if offline.

Not sure what you mean, but you may be confusing the role of the update manifest XML file generator (umgen). The current implementation is a command line tool which allows it to be used in batch scripts - allowing you to automate the build process. It was also an ad-hock thing that I slapped together because generating MD5 hashes by hand is tedious. You could easily replace this with a GUI version. I just didn't bother because it would have taken too much time on something that wasn't a high priority at this early stage.

Yeah, I guess I misunderstood the meaning on this subject.

The "easiest" solution, is to allow the developer to make their product, include the updater program in their INSTALLER, and let the installer modify the registry to allow all users to access the updater / files.

Unlocking the Program Files folder for non administrator access is a very bad idea. It defeats the whole reason for that lock in the first place (protecting essential program files from viruses and idiots).

A lot of games actually unlock that specific folder within the Program Files directory, and I believe the upper tier folders, including the parent Program Files directory, is locked to the limited user. Again, I'm not 100% sure about this.
I know some games actually require you to play as an Administrator... which is retarded, because you need to give the other users to Admin PW to play them.

Oh... is there anything I can do left?  

Absolutely! This Win32 beta is seriously in need of a decent tutorial, and I'm looking forward to some feedback on testing. Plus, do you know anything about SourceForge? SVN? It would be good to have someone who could deal with the day-to-day administration of that side. As much as I would like to do it, I only have a limited amount of time to devote to this project, and that would be best spent on coding.

I've used TortoiseSVN before for my game programming capstone back in my university days.
Haven't used SourceForge for uploading personally (except for file DLs by others).
What about a Wiki... anyone going to maintain that?
Also, what about Source Code Documentation, are you going to write some of that up, or are you going to allow someone else to do that? I always use Doxygen to do my code documentation, and I usually document AS I write my larger code projects... time consuming, but helpful in the long run when I need to look back at them after not seeing them for months/years.

Lastly, for the WINDOWS version, what are we using if we're doing SVN?
C-Make... MSVC++... Eclipse...

Anyone else have their take on the things I pointed out?

[technogothica]

A lot of games actually unlock that specific folder within the Program Files directory, and I believe the upper tier folders, including the parent Program Files directory, is locked to the limited user. Again, I'm not 100% sure about this.
I know some games actually require you to play as an Administrator... which is retarded, because you need to give the other users to Admin PW to play them.

I've not seen recent games do that (admittedly I've played far fewer recent games than I should like), but User Account Control has a mechanism that allows pre-Vista software to write to Program Files and other protected folders and registry keys without error.

In reality, those programs are not writing to Program Files or protected registry keys. What happens is that Windows automatically redirects all file and registry operations to another directory/registry hive in the current user's profile.

http://windowsteamblog.com/windows/b/developers/archive/2009/08/04/user-account-control-data-redirection.aspx

This helps Windows 95/98 applications run in Vista without too many problems, but it's not foolproof. New software should not be doing this and will probably fail if the EXE contains an assembly manifest (I believe this is how Windows differentiates old and new applications).

[Evan Balster]

Hey all.  I've been away for a while; pardon that.


dredogol -- You seem to have a pretty solid grip on the idea other than one important point: ideally, the user would be able to click a download link (to the .exe updater, in Windows' case) and open the downloaded file to immediately launch the game.  I've been bringing up simplicity as a design goal over and over throughout the course of this thread, and don't think the program should even be visible unless it's downloading an update.  (In which case the operating system's provided loading bar graphic should do)

An installer or separate configuration file is a bit of a can of worms; it's a place for the technically illiterate to fumble helplessly with zip files and dialogs when a simple double-click should work.  Really, more than anything I'm infatuated with the way Minecraft manages to give the false impression that its updater *is* the game.

Regarding levels of professionalism, this thing will most likely be used primarily by lower-budget games and software; larger teams with more technical capacity have the capacity to build more customized updater software themselves, employing tactics such as housing game code in dynamically-loaded object files and such.


Technogothica and I were talking about setting up a website (and sourceforge account) for this little project, and I think I'll be doing so soon.

[dredogol]

I've actually started to code up an Updater for a game I'll be making here soon.

It won't be using the windows built-in GUI system for a simple updater look, like the one you're after Cellulose.

Instead, mine'll be done in C++, CEGUI (CrazyEddie's GUI), and GLUT.
I'm aiming for a cross-platform patch updater in a single project, rather than a different one for each OS. Only a few things will be different between each OS version (I think)... so might as well add them in. If not, I can always easily separate them based on the way my framework is designed...

Anyways, It's going to be a generic patch updater, like those found for MMORPGs, with connection to an HTTP server and all. Also a transparent (non-window bordered) background, so you can add whatever transparent image for your updater's borders.

When I get a sample up and running, I'll post some samples.

[technogothica]

Hi Guys,

I've been watching, with great interest, the porting of Desura to Linux. They seem to be tackling some of the same technical challenges as us with regard to user accounts:

http://www.desura.com/groups/desura/news/desura-linux-development-we-have-ui

(See the comments section after the main article)

[Klaim]

It looks like a lot of people get the same ideas at the same time ....

(source : http://www.ogre3d.org/forums/viewtopic.php?f=16&t=65134&view=unread#unread)

https://github.com/amireh/Karazeh
https://github.com/amireh/Karazeh/wiki

[amireh]

Hi guys, I've also had the need for such a tool and sadly I didn't come across this topic until today. Anyway, I've released my first version of the tools which I'll describe below.

What's done

Generally, I made 2 tools, one is a launcher that a) validates the application/game, b) patches it, and 3) launches the game client. The other tool is one for creating the patch scripts and deploying them.

Karazeh: the launcher, here's a breakdown of its "features":

• uses boost::filesystem for handling file paths and file management
• does not tie you to any front-end/rendering library: I've shipped 3 "Renderers", one is Ogre3d-based, another is Qt-based, and a vanilla CLI one.. creating a GTK/wxWidgets/any other toolkit based renderer is very straightforward and possible
• uses either boost::thread or Qt threads for threading support while patching so the launcher can stay responsive and inject feedback for the user
• could also be used in threadless mode, and with any other thread provider. Creating your own thread wrapper using your own thread provider is simple
• Karazeh can patch itself! This was tested on Linux and Windows XP, might be UAC problems on 7/Vista
• is runtime path agnostic; you can run it from anywhere and it will work granted that the user doesn't manually move the binary to a different directory
• uses CURL for fetching the patch data, and supports for retries in downloading; if a file could not be fetched, or didn't pass the integrity check, you can specify how many times it can be redownloaded
• all downloaded files are MD5-summed and verified for integrity
• in-memory binary patching using bsdiff so users won't have to waste bandwidth.. send the partial changes instead of huge files in one chunk
• diff files are (de)compressed using BZip2
• supports 4 operation types (CRUD): create new files, modify, rename/move, and delete existing files
• patching is staged, transaction is not committed until all operations were verified, should never corrupt your tree
• support for hard-coded patch server URLs and an external patch mirrors list
• gracefully falls back if there was any problem carrying out any operation, or if there's no internet connectivity, or if all registered hosts can not be resolved

For more info please refer to its github page/wiki.

Now the 2nd tool, Kiwi, is built with Qt and lets you do a few things:

• generate the patch script to be consumed by Kiwi
• create a bz2 tarball of all your patch data (newly created files and diffs) for instant deployment on your server
• generate binary diff/md5 checksums of files from within the toolkit

What I want to do better
As per my question on StackOverflow, here, I had to find a way to encode the application's version.. so I can use it for validation and well, the whole patching process. I was looking for a solution to embed such a signature *inside* the binary itself, but that seemed to go against recommendations. So what I ended up doing is using an external plain text resource in which the version is written out. In the case the user spoofs with that file... it's very easy to fix it again since it's a text file. However, if Karazeh suspects the file has been tampered with, it resorts to the version string #defined in its binary. So by updating that string everytime you update the game, this problem would be moot.

But I just don't like the solution, so if anyone has a suggestion please do tell.

What I want to do
1) Most importantly, I'd like the patches to be self-contained and portable. I'd like clients to be able to patch their game using the patch data they downloaded on their desktop, or somewhere on the network. That could be done either by allowing people to host their own patch servers (ie over the network) or by feeding Karazeh with the patch data and it would know how to deal with it. The latter is just as good and it's less complicated IMO.

2) I want Karazeh to support integration *within* an application like a library. Just call on to it if you'd like to patch (like HoN's patcher) instead of a full-fledged outside of the client launcher.

3) I'm using CEGUI for my next game so I'll be making a CEGUI renderer... Qt is kinda too heavy and has some licensing "meh"s.

4) I'd like to make use of each platform's home directory for staging patch data. Windows use the AppData of the user's home, linux and OS X use the home directory of the user (~/.XXX). However, this will still not permit the launcher not to need write permissions as it will technically *have* to modify the game data, so clients will still need such permissions to use it. Frankly, there's nothing to do about that unless the game is installed in each user's folder, which is up to them.

Edit: fixed some typos and forgot to add an entry in the last section.

----
Anyway I didn't know how many people would find this useful, but I did design the launcher in a way to impose as few constraints as possible on the user. It should not tie you to how *it* works. Please if you have any feedback, I'm all ears!

Finally, I'll happily join my efforts with any of the existing/solutions being developed. Just send a tell or reply here! =D

Salam guys,
amireh

[technogothica]

Hi amireh,

You've done a fantastic job, well done!

• does not tie you to any front-end/rendering library: I've shipped 3 "Renderers", one is Ogre3d-based, another is Qt-based, and a vanilla CLI one.. creating a GTK/wxWidgets/any other toolkit based renderer is very straightforward and possible

This is actually a great design. We've been trying to avoid the likes of Qt both because of licensing issues and the issue of having to pre-install runtimes before the launcher/updater. But separating the interface from implementation to the extent that the end developer can use their favorite user interface package is excellent.

• in-memory binary patching using bsdiff so users won't have to waste bandwidth.. send the partial changes instead of huge files in one chunk
• diff files are (de)compressed using BZip2

That's great. We've been working towards using diff, but haven't quite got there. You've provided us with a proof of concept!

• create a bz2 tarball of all your patch data (newly created files and diffs) for instant deployment on your server

That's one of our goals as well - suggested by Cellulose's original question.

• generate binary diff/md5 checksums of files from within the toolkit

I put together a command line tool called umgen which generates an "update manifest XML file" which sits on the web server. This tool scans the current working directory and (optionally) all directories below, adding filenames to the XML output, complete with MD5 hashes. You could expand this to include diff.

As per my question on StackOverflow, here, I had to find a way to encode the application's version.. so I can use it for validation and well, the whole patching process. I was looking for a solution to embed such a signature *inside* the binary itself, but that seemed to go against recommendations. So what I ended up doing is using an external plain text resource in which the version is written out. In the case the user spoofs with that file... it's very easy to fix it again since it's a text file. However, if Karazeh suspects the file has been tampered with, it resorts to the version string #defined in its binary. So by updating that string everytime you update the game, this problem would be moot.

But I just don't like the solution, so if anyone has a suggestion please do tell.

With my current implementation (Win32) I pretty much ignored a version numbering scheme (though I may need to add one later on). What I did was have the launcher/updater rely on the server-side update manifest XML file to describe what the application should be. The XML file lists all the files that the application requires along with their MD5 hashes and the URL from where to download it. So the launcher/updater downloads the XML file, checks MD5 hashes of all local files, and if any files don't match, they are downloaded again.

So the XML file serves as a kind of bench mark for what the current version should be, without actually using a version number. It works fairly well, though my current implementation does have a small hole in it which I'm about to plug (that hole is due to broken downloads, but the result is no worse than Steam - you just can't play the game until all files are fully downloaded).

1) Most importantly, I'd like the patches to be self-contained and portable. I'd like clients to be able to patch their game using the patch data they downloaded on their desktop, or somewhere on the network. That could be done either by allowing people to host their own patch servers (ie over the network) or by feeding Karazeh with the patch data and it would know how to deal with it. The latter is just as good and it's less complicated IMO.

One important consideration that has been raised by several contributors is the issue of security. Allowing people to host their own patching servers is a perfect opportunity to spread malware. But it does depend on the needs of the audience and the application itself. Our target audience with this project is gamers, particularly those who are not tech savvy. We are also targetting games that roll out regular updates (e.g. on a daily or even hourly basis). But you may be targetting a different audience which may prefer or even require such a configuration?

That said, I'm totally with you on being able to download patch data and manually patch. Such capabilities would be good for the advanced users who want more fine control.

4) I'd like to make use of each platform's home directory for staging patch data. Windows use the AppData of the user's home, linux and OS X use the home directory of the user (~/.XXX). However, this will still not permit the launcher not to need write permissions as it will technically *have* to modify the game data, so clients will still need such permissions to use it. Frankly, there's nothing to do about that unless the game is installed in each user's folder, which is up to them.

Our number one topic of discussion right now

The problem with developing a multi-platform solution is that each platform deals with security in different ways. Putting files into the user's home directory is convenient but not ideal. In the case of Windows, EXE files should really be stored in the UAC-protected Program Files folder, but that would require the launcher/updater to elevate to administrator every time the game needs to update. Similar issues exist for Linux.

I'm only just starting to learn to use Mac OS X, so I have no useful remarks on that.

Finally, I'll happily join my efforts with any of the existing/solutions being developed. Just send a tell or reply here! =D

Likewise, assuming you are using an MIT-compatible (permissive) license.

At the moment we are working on several independent solutions, sharing ideas and experiences in the hope that we can produce a unified solution for all platforms down the road.

[amireh]

You've done a fantastic job, well done! Grin

Thank you good sir, only makes me happy :D

This is actually a great design. We've been trying to avoid the likes of Qt both because of licensing issues and the issue of having to pre-install runtimes before the launcher/updater. But separating the interface from implementation to the extent that the end developer can use their favorite user interface package is excellent.

My point of making it a generic launcher was that it adapts to *your* environment and application, not the other way around. For all it cares, you should be able to launch it without a renderer (directly inside your application). And my real plan is to eventually create a Win32 renderer, a Cocoa one for mac, and a GTK one for Linux.

That's one of our goals as well - suggested by Cellulose's original question.

In v0.2 of my patcher, all downloadable patch data will be compressed as one tarball for the following reasons: a) faster to download and possibly add resume support, b) less complicated technical design, c) users will use less bandwidth but more CPU

I put together a command line tool called umgen which generates an "update manifest XML file" which sits on the web server. This tool scans the current working directory and (optionally) all directories below, adding filenames to the XML output, complete with MD5 hashes. You could expand this to include diff.

I considered checksumming the whole directory tree, but that conflicted with my design. IMO, keeping track of the whole directory tree/its files will give you a lot of pain to maintain and even implement in a cross-platform manner. What if my Mac tree has 17 more files than Windows? How do you account for environment-specific files, like logs, configuration files etc? You will have to include "exclude lists" in your manifest parser which just adds to the complexity.

The way I saw it when I designed the launcher is that patches/repositories should represent the *changes* in the application tree, not the state of the whole deal. But then again, your way sounds more fool-proof and integral. I like it.

With my current implementation (Win32) I pretty much ignored a version numbering scheme (though I may need to add one later on). What I did was have the launcher/updater rely on the server-side update manifest XML file to describe what the application should be. The XML file lists all the files that the application requires along with their MD5 hashes and the URL from where to download it. So the launcher/updater downloads the XML file, checks MD5 hashes of all local files, and if any files don't match, they are downloaded again.

So the XML file serves as a kind of bench mark for what the current version should be, without actually using a version number. It works fairly well, though my current implementation does have a small hole in it which I'm about to plug (that hole is due to broken downloads, but the result is no worse than Steam - you just can't play the game until all files are fully downloaded).

That's very interesting, but is related to my point above. I'm just against hashing all the files, this could be very costly time-wise, for example on my quad core it took 2m21s to md5 sum a 4.4gig file, that's just bad.

I've been thinking alot about this, and here's my proposed solution, and why:

Case1: rely on the Launcher to represent your game's version

In this case, it's a simple matter of re-compiling the launcher with the new version signature embedded in the binary whenever you patch the game, and since Karazeh can already patch itself, just patch itself with the new version. Use no external resource, rely solely on this embedded signature.

Fail scenarios: the user edits the binary and changes the version signature, then the Launcher will have an invalid version signature.

Solution: everytime the Launcher runs, it calculates a checksum of *itself* and diff that against a checksum written in the patch script (from the server) which represents itself when it was shipped. If the user did indeed spoof it, the checksums won't match, and we can tell that it was tampered with

Upsides: 1) no use of external resource to track version, 2) integrity can always be verified, 3) reduced checksum calculations down from N files to 1, where N is the number of files your tree contains

Downsides: 1) you need to always re-compiled the launcher whenever you patch the game (not a biggie), 2) you rely on the patcher to verify the game, which could be a problem if the game was launched directly.. this could be assuaged by doing the same thing in the game's binary

Case 2: rely on the game binary itself

The mechanism is similar to case 1 but instead of using the version embedded in the launcher, we can modify the game's binary to parse ARGV and return *its* version. That version is checked and verified just as with case 1, and it has the same downfalls and solution.

Upsides:
1) No need to recompile/update the launcher when you update your *game*
2) the Launcher and your game can be two separate entities with two separate versions, unlinked

Downsides:: you need to make your game compatible by returning its version if asked.

---

I like Case 2 better, as it will allow me to update the launcher whenever I want, and update the game whenever I want. The patch script (generated by Kiwi) will have to adapt to these changes, or better yet, make the launcher look for updates for itself as well as the game.

Please give me your input on these solutions and tell me if you see something I don't.

The problem with developing a multi-platform solution is that each platform deals with security in different ways. Putting files into the user's home directory is convenient but not ideal. In the case of Windows, EXE files should really be stored in the UAC-protected Program Files folder, but that would require the launcher/updater to elevate to administrator every time the game needs to update. Similar issues exist for Linux.

I'm only just starting to learn to use Mac OS X, so I have no useful remarks on that.

When it comes down to it, no matter how you do it, you *will* need permission to modify files. Really, that's just the way OSes work, there's not much we can do about that. If the game was installed to a local directory, that's the user's decision and they do know that they have to duplicate the game across all users for that to work. All that really matters on *our* side is to handle the cases where we don't have enough permission, and respond with kindness and love.

Likewise, assuming you are using an MIT-compatible (permissive) license.

At the moment we are working on several independent solutions, sharing ideas and experiences in the hope that we can produce a unified solution for all platforms down the road.

Both Karazeh & Kiwi are MIT-licensed. I myself will be using these tools for my current game and my upcoming one, so it will be heavily tested and modified probably. I'd gladly offer my help if you think I can help you with anything.

Cheers guys,
amireh

[technogothica]

That's very interesting, but is related to my point above. I'm just against hashing all the files, this could be very costly time-wise, for example on my quad core it took 2m21s to md5 sum a 4.4gig file, that's just bad.

Quite so. I suppose if you are using very large files, and hashing every time the game starts, then that would be an unreasonable amount of time wasting when no update is required.

You could probably counter that by hashing only once, and caching the result so that the file only needs to be hashed again if the file timestamps change. Or something to that effect...

That's one for the to-do list

[jack_norton]

Just wanted to bump this thread to know if there have been any progress on this?

[amireh]

Just wanted to bump this thread to know if there have been any progress on this?

Hi! Regarding my application, I added a few things, but for now I'm going to let it rest as I develop the main app for which it will be used, and in the meanwhile I'll be adding features as I need.

I added native support to Mac OS X and Gnome users (Cocoa and GTK3 front-ends), a Windows one is still missing. There's now support for archived patches (you can tarball all the patch data to save bandwidth and in the future add resume support for downloads). Anyway you can check/request features on Karazeh's github page or reply to this post and I'll see to them :-)

I didn't get to implement the integrity check we discussed here yet, as it's not crucial to me at the moment, but I'll update this post when something interesting is done.

Cheers,
amireh

[technogothica]

Hi guys,

I'm now working two jobs with no time to spare, so I'm going to bow out of this project for a few months.

I'll still be watching this thread and chime in once in a while with my thoughts on what you guys are up to.

Keep up the good work