Welcome, Guest. Please login or register.

Login with username, password and session length

 
Advanced search

1411196 Posts in 69314 Topics- by 58380 Members - Latest Member: feakk

March 18, 2024, 08:29:01 PM

Need hosting? Check out Digital Ocean
(more details in this thread)
TIGSource ForumsDeveloperTechnical (Moderator: ThemsAllTook)Project: free autoupdater/launcher for games
Pages: [1] 2 3 ... 8
Print
Author Topic: Project: free autoupdater/launcher for games  (Read 26945 times)
Evan Balster
Level 10
*****


I live in this head.


View Profile WWW
« on: March 27, 2011, 11:10:16 PM »

Hey, all.

So with Infinite Blank, I've run into a need.  A need I expect many other developers who release frequent builds of in-dev projects encounter:  autoupdates.  Some systems (like Java) let us replace program information while our code runs, with enough work.  Others, like Game Maker or old-fashioned native code (for Windows, anyway) don't.  Mojang's illustrious Minecraft solves this problem very gracefully with a simple piece of software which installs the game to an OS-specific location and runs it.  In short:  I want one.

I would like to make a similar piece of software, but for free use by game and developers of all sorts.  That is, a compact and configurable program, available for as many platforms as possible, which downloads a program's binaries and resources from the web (with a small graphical display indicating download progress) installs them to an application data folder, and runs the main binary of the program.  In subsequent runs, the launcher checks the developer's website for updates and possibly downloads and installs them.

However, my platform-specific programming know-how is limited (I typically stick to crossplatform abstraction layers) -- I talked with mcc last month at GDC and we concluded this might make a good collaborative project for the programmers of TIGSource!

After settling on a simple (see: handwritten textfile) protocol for the web side of all this, we would create a launcher for each targeted platform.

The major requirements would be these:
  • Minimal filesize
  • Single-file download
  • Trivial to user (eg, an RPM'ed app package for OS X and a simple EXE [no DLLs] for Windows)
  • Configurable web source, icon, and program name
  • Configurable launching routine for software (preferably a system call)
  • Invisible when not updating the program
  • Able to perform HTTP downloads, and HTTP-1.1 compliant
  • Able to extract some kind of compressed archive format and perform filesystem operations
  • Installs to a reliably-accessible application data folder (OS-specific)
  • Functions properly when not connected to internet

A simple initial scheme for things would be to have a textfile available on an HTTP (web) server which indicates a version number for the program and locations for binaries for each platform.  Something like this:

Code:
0.3.0
win32: http://infiniteblank.com/download/infiniteblank-0.3.0-win.pack
osx: http://infiniteblank.com/download/infiniteblank-0.3.0-mac.pack
linux32: http://infiniteblank.com/download/infiniteblank-0.3.0-lin.pack

The archive (here called '.pack' for lack of a decided format) would be extracted and its files would replace equivalent ones in the application directory.

More elegant and bandwidth-efficient versions which support partial updates or more sophisticated file handling should probably be held beyond the scope of this project for the time being.


I'm interested to hear who's game for helping out with this.  I think I can handle the Windows version, though there are probably many more qualified than I; that leaves a need for experts in Linux and OS X, for starters.


And though it might be jumping the gun, I like the name 'Launchy'.  Smiley
Logged

Creativity births expression.  Curiosity births exploration.
Our work is as soil to these seeds; our art is what grows from them...


Wreath, SoundSelf, Infinite Blank, Cave Story+, <plaid/audio>
mcc
Level 10
*****


glitch


View Profile WWW
« Reply #1 on: March 27, 2011, 11:23:09 PM »

I am interested in aiding with an OS X or Linux version if this is helpful, but my schedule is very, VERY full right now.
Logged

My projects:<br />Games: Jumpman Retro-futuristic platforming iJumpman iPhone version Drumcircle PC+smartphone music toy<br />More: RUN HELLO
Evan Balster
Level 10
*****


I live in this head.


View Profile WWW
« Reply #2 on: March 27, 2011, 11:32:19 PM »

Mine is too, to some extent.  Two active projects, yeesh.  >_>
Logged

Creativity births expression.  Curiosity births exploration.
Our work is as soil to these seeds; our art is what grows from them...


Wreath, SoundSelf, Infinite Blank, Cave Story+, <plaid/audio>
Tycho Brahe
Level 10
*****

λx.x


View Profile
« Reply #3 on: March 28, 2011, 03:02:37 AM »

I'd be happy to help with what I can, like hunting bugs and stuff.
Logged
eclectocrat
Level 5
*****


Most of your personality is unconscious.


View Profile
« Reply #4 on: March 28, 2011, 03:33:48 AM »

I'll do OS X.
Logged

I make Mysterious Castle, a Tactics-Roguelike
Cheezmeister
Level 3
***



View Profile
« Reply #5 on: March 28, 2011, 04:08:46 AM »

I feel like this exists already in several incarnations, though perhaps not as portable and modular as what you're describing.

I'll hop in with everyone lending moral support, but can't commit any dev time. Sad
Logged

෴Me෴ @chzmstr | www.luchenlabs.com ቒMadeቓ RA | Nextris | Chromathud   ᙍMakingᙌCheezus II (Devlog)
technogothica
Level 1
*



View Profile WWW
« Reply #6 on: March 28, 2011, 04:39:17 AM »

Sounds like a great idea.

I am proficient in C/C++ and Win32 API. I can do user interfaces with proper font scaling and theme support. I have experience with Winsock, and zlib. Actually, as far as the windows version is concerned, this is all pretty easy stuff for me.

I've never worked on a collaborative project like this before, but I'd like to offer whatever help I can.

I'm working full time on my game, but since I need a tool like this myself, I have no problems devoting time to it.

Any thoughts on DRM or DLC?
Logged

Klaim
Level 10
*****



View Profile WWW
« Reply #7 on: March 28, 2011, 05:44:23 AM »

I'm insterested and would like to help ... in few months.

I'll follow the project anyway.

Could you just explain more extensibly the features? I'm not sure I understand all of them and some kind of feel overkill.


Anyway, looks like something based on Qt might fit the need as it would then provide gui, html rendering, easy customization, networking, etc.
The thing is : it have a cost on the binary size. That said it can be optimized.
Logged

Evan Balster
Level 10
*****


I live in this head.


View Profile WWW
« Reply #8 on: March 28, 2011, 09:06:13 AM »

Technogothica -- You sound well-qualified for the Windows version; have you ever programmed HTTP?

eclectocrat -- Likewise.

And even if you haven't, it's a very simple protocol other than a few nitpicky things.


These will probably be completely separate programs, unified only in the formatting of the web file they use and some basic customization parameters.  Additionally, a separate tool (possibly web-based for maximal compatibility?) for adding configuration information to each version of the launcher would be a good thing.  As that goes, I know a standard technique on Windows and Linux is simply to append data to the end of the binary.  On OS X, there's the benefit of an RPM package.


Any thoughts on DRM or DLC?

Supporting some basic kind of DRM would be a nice feature, but probably one best left to developers of locked software; if the source for each version is released, they could build it into the launcher(s) themselves.  However, if we can find a way to allow for a simple scheme with minimal work, it could be a nice feature.

The same (more or less) goes for DLC.


I feel like this exists already in several incarnations, though perhaps not as portable and modular as what you're describing.

There are things resembling simplified Steam clients which do this stuff, but I for one would rather just let my user start the program without having to navigate another interface.  Additionally, these (as I understand) aren't single-file distibutions.


Could you just explain more extensibly the features? I'm not sure I understand all of them and some kind of feel overkill.

A minimal program that can be made to look like it *is* the game the user is playing (which is actually installed to an application data directory, and autoupdated in subsequent runs), is extremely simple to use (eg, not packaged in a zip-file for Windows users), is small in size, and is configurable so developers can host builds of their games and versioning information on webservers.
Logged

Creativity births expression.  Curiosity births exploration.
Our work is as soil to these seeds; our art is what grows from them...


Wreath, SoundSelf, Infinite Blank, Cave Story+, <plaid/audio>
mcc
Level 10
*****


glitch


View Profile WWW
« Reply #9 on: March 28, 2011, 09:22:32 AM »

My worry with qt is there could be license issues. I'm not sure whether you'd want to link this into the program or not...
Logged

My projects:<br />Games: Jumpman Retro-futuristic platforming iJumpman iPhone version Drumcircle PC+smartphone music toy<br />More: RUN HELLO
Klaim
Level 10
*****



View Profile WWW
« Reply #10 on: March 28, 2011, 01:30:29 PM »

A minimal program that can be made to look like it *is* the game the user is playing (which is actually installed to an application data directory, and autoupdated in subsequent runs), is extremely simple to use (eg, not packaged in a zip-file for Windows users), is small in size, and is configurable so developers can host builds of their games and versioning information on webservers.

Yes, I understand all that but it's more a per-point more extensive explaination I was asking :

Quote
Minimal filesize

Ok so you mean, size of the tool itself, not a kind of compression.

Quote
Single-file download

There you mean the ability to setup a new version with only specific files to updates, and the tool will update only those files, correct?

Quote
Trivial to user (eg, an RPM'ed app package for OS X and a simple EXE [no DLLs] for Windows)

That's fuzzy, you're talking about the generated tool? The whole game package? The package of the tool only?

Quote
Configurable web source, icon, and program name

Yeah, configuration is key there.

Allowing the user to set some configuration options that the game will use would be generic excellence too.

Quote
Configurable launching routine for software (preferably a system call)

So, a script system would be nice. A lua based scripting system would help match the lean binaries requirements.

Quote
Invisible when not updating the program.

Ok so it should just launch the game and quit (after update), that's what you mean?

Quote
Able to perform HTTP downloads, and HTTP-1.1 compliant

OK... that said it's not really a "big" requirement, more like a "would be nice to have", no? Anyway Qt would help with displaying html pages.

Quote
Able to extract some kind of compressed archive format and perform filesystem operations

Ok, but then you'll have to allow an extension point : allow the user to provide it's own compression plugin. The thing is we shouldn't assume we'll provide all the compression, just ways to add more.

Quote
Installs to a reliably-accessible application data folder (OS-specific)

Must have.

Quote
Functions properly when not connected to internet

Yes, must have too.

Logged

technogothica
Level 1
*



View Profile WWW
« Reply #11 on: March 28, 2011, 02:09:04 PM »

Technogothica -- You sound well-qualified for the Windows version; have you ever programmed HTTP?

Yep, I've had experience sending raw HTTP requests over a TCP socket. I haven't done partial file downloads before, but I do know the HTTP commands to do it.

These will probably be completely separate programs, unified only in the formatting of the web file they use and some basic customization parameters.

Agreed. I like the idea of doing separate programs. Reduce the dependencies as much as possible. Make each version lean and mean.

Please don't use XML for the config file. I like the plain text format in your example. Keep it simple Smiley
Logged

technogothica
Level 1
*



View Profile WWW
« Reply #12 on: March 28, 2011, 02:23:40 PM »

My worry with qt is there could be license issues. I'm not sure whether you'd want to link this into the program or not...

Agreed Hand Thumbs Up Left

If a window toolkit is needed I would advocate GTK. But if the program isn't doing anything more sophisticated than showing a download progress meter and a cancel button, then perhaps just a simple X window? The fewer dependencies on external libraries and toolkits the better.
Logged

Rob Lach
Level 10
*****



View Profile WWW
« Reply #13 on: March 28, 2011, 02:25:06 PM »

I'd hop on for any platform but ill be busy until IGF deadline lol.

This project screams python btw.
Logged

technogothica
Level 1
*



View Profile WWW
« Reply #14 on: March 28, 2011, 04:17:23 PM »

This project screams python btw.

Please elaborate.
Logged

eclectocrat
Level 5
*****


Most of your personality is unconscious.


View Profile
« Reply #15 on: March 28, 2011, 05:49:03 PM »

This project screams python btw.

DUDE! Seconded. I'm just going to go ahead and get something working in python and I'll get back to y'all in a few days.
Logged

I make Mysterious Castle, a Tactics-Roguelike
Evan Balster
Level 10
*****


I live in this head.


View Profile WWW
« Reply #16 on: March 28, 2011, 07:47:19 PM »

Rob Lach and eclectocrat:  That's a great solution, but only if it doesn't require the user to have Python installed.  Would it?


Klaim -- You've got it, for the most part.  I'll clarify a few points, though.

Quote
Minimal filesize
Ok so you mean, size of the tool itself, not a kind of compression.

Quote
Single-file download
There you mean the ability to setup a new version with only specific files to updates, and the tool will update only those files, correct?

Quote
Trivial to user (eg, an RPM'ed app package for OS X and a simple EXE [no DLLs] for Windows)
That's fuzzy, you're talking about the generated tool? The whole game package? The package of the tool only?

I mean that the launcher itself will be, for instance, downloaded as an EXE on windows.  No DLLs or other files, so as to avoid requiring the user to extract a ZIP archive or similar.  For the same reason, a Python autoupdater (if it requires Python installed on user machines) won't work.  It should be something that can be downloaded and run, and will "just work".

As OS X goes, RPMs are used for everything, so an app package in an RPM that can be dragged onto the desktop is an acceptable way of doing things.  Again, user-friendliness is the biggest priority in my mind.


Quote
Quote
Configurable launching routine for software (preferably a system call)
So, a script system would be nice. A lua based scripting system would help match the lean binaries requirements.

Possible, although we'd need to agree on a small standard set of functionality; is there a portable library for Lua that will suffice?


Quote
Quote
Invisible when not updating the program.
Ok so it should just launch the game and quit (after update), that's what you mean?

Yep!  (Or it could run the game as a child process and do so invisibly, terminating after the game quits.)


Quote
Quote
Able to perform HTTP downloads, and HTTP-1.1 compliant
OK... that said it's not really a "big" requirement, more like a "would be nice to have", no? Anyway Qt would help with displaying html pages.

I mean downloading files, not displaying webpages.  That would be a huge increase in requirements.  Visually, all this would need would be a progress bar and/or text indicating updating status.


Quote
Quote
Able to extract some kind of compressed archive format and perform filesystem operations
Ok, but then you'll have to allow an extension point : allow the user to provide it's own compression plugin. The thing is we shouldn't assume we'll provide all the compression, just ways to add more.

I was thinking we could agree on a common opensource format like tar.gz.
Logged

Creativity births expression.  Curiosity births exploration.
Our work is as soil to these seeds; our art is what grows from them...


Wreath, SoundSelf, Infinite Blank, Cave Story+, <plaid/audio>
technogothica
Level 1
*



View Profile WWW
« Reply #17 on: March 28, 2011, 10:21:56 PM »

Here's the design as I see it - Windows example:

Say a game consists of three files:

launch.exe
engine.exe
data.pak

Start the game with launch.exe.

First thing launch.exe does is connect to a HTTP server and download a plain text file containing a list of game files, MD5 checksums for each file, and URLs where the latest version can be downloaded from.

If any MD5 checksum doesn't match the corresponding file, that file is downloaded again from the web server. In this example, engine.exe and data.pak are checked and newer versions downloaded if required.

Once all files are determined to be up-to-date, launch.exe runs engine.exe, supplying appropriate parameters. The game begins and fun times are had.

launch.exe need not show itself to the user unless files need to be downloaded. In that case, it shows a dialog box with a message saying "Updating", a progress meter, and a cancel button.

Updates are rolled out simply by modifying the plain text file on the web server.
Logged

Evan Balster
Level 10
*****


I live in this head.


View Profile WWW
« Reply #18 on: March 28, 2011, 10:44:23 PM »

I was envisioning pretty much exactly that, except with all game files including the program rolled into a single compressed archive, which is of some agreed-upon format.  (Again, I'm in favor of .tar.gz.)  New versions are simply extracted over old ones, so as to update relevant files and leave save information intact.


Does anyone have any thoughts on archive format (which could potentially be different for different platforms) or the format of the versioning file?  If not, we could get this show on the road rather quickly.  I think this is something that could do a lot of devs (myself selfishly included) a lot of good!   Grin
Logged

Creativity births expression.  Curiosity births exploration.
Our work is as soil to these seeds; our art is what grows from them...


Wreath, SoundSelf, Infinite Blank, Cave Story+, <plaid/audio>
technogothica
Level 1
*



View Profile WWW
« Reply #19 on: March 28, 2011, 11:05:17 PM »

Quote
I was envisioning pretty much exactly that, except with all game files including the program rolled into a single compressed archive...

Wouldn't be difficult to support both cases.

Quote
Does anyone have any thoughts on archive format (which could potentially be different for different platforms) or the format of the versioning file?

For the archive, I would favor ZIP, but if the launcher uses zlib, there's no reason why you couldn't support both. My distaste for .tag.gz is that the entire archive has to be decompressed into an additional temporary file before you can get to the files inside.

For the versioning file, I would recommend "MIME text/plain". XML is a possibility, but it takes too much effort to parse. List each file on a line, with name of file, md5 checksum and URL separated by whitespace:

Code:
package1.pak d6f89cf7489a349f http://www.server.com/game/download/package1.pak
package2.pak c468f9b94684e798 http://www.server.com/game/download/package2.pak


File names with whitespace should be urlencoded (e.g. %20 for space).
Logged

Pages: [1] 2 3 ... 8
Print
Jump to:  

Theme orange-lt created by panic