next »

[wg/funstorm]

For an upcoming AS3 game I need to store multiple files online per user so that they can be downloaded by other users. Each file could be between 10 and 1000KB so storing that data in a database is out of the question. I've done everything I can to simplify and compress the data and bring file size down already and that's what I'm left with. The files need to be uploaded by a user, and then downloaded by other users. The users need to be authenticated. And I also need a database to match users<->files.

Does anyone know of any existing services that would be suitable? I've looked into things like PlayerIO, Playtomic, Parse, but they're all focused on storing small bits of data and don't offer what I need.

I'm a crappy self-taught programmer so writing my own using some sort of cloud offering like amazon or heroku is a very daunting proposition especially since there's security stuff involved when it comes to authentication and so on.

Suggestions, please?

[SFBTom]

Take a look at some of the CMS systems out there. I know there's quite a bit about using Drupal with flash, so that might be a good starting point. Writing your own would be a huge endeavour, and most likely full of security holes.

[raigan]

>Each file could be between 10 and 1000KB so storing that data in a database is out of the question.

I'm a database noob, but why would this be a problem in a database?

Could you possibly store a UID/file-name in the database, which could be used to address the file? i.e don't store the file contents, just a handle which lets you access the file.

[phi6]

I would recommend MongoDB as they have a very good blob store (GridFS). It's also really easy to set up and use and has a variety of examples and tutorials for all the major server side languages. It's pretty much built for exactly what you are trying to do!

[Sean A.]

Have you looked into encoding the data into an image? I think is one of the easiest (and coolest) ways of transferring data. Once you have that system in place you can look for more broad online storage systems that arent focused on games but on managing images instead and I think you'll find more options. http://www.gallerycms.com/ is an example, after a quick google for image cms's that one came up and it has user log-in and each user can have many different pictures. It also has an xml feed/api so you can integrate it with flash.

[wg/funstorm]

Thanks for all the suggestions everybody.

Question about CMS'es.. aren't they more focused on allowing users to browse the files through a web interface? I just want to upload->save->download the files all from within flash. I mean I guess maybe I could find a way to hook into their up/download feature like Sean suggest with the xml feed / api and just not use the frontend features but my gut says this feels like quite a heavy solution if I'm not using most of the features.

My own research has led me to something like what raigan suggests. I could use swfupload to upload files to S3. And then use playerIO to give me the security I need with user authentication, gameplay server, and database to save which user uploaded which files.

[Dacke]

Have you looked into encoding the data into an image? I think is one of the easiest (and coolest) ways of transferring data.

Storing stuff as images is highly volatile. Image hosting sites often encode images to save space wihtout any warning.

Also, a point worth repeating:
Databases can easily store 1MB blobs.

For example, the MySQL LONGBLOB datatype can store up to 4.3GB per entry (if I'm not mistaken)
http://kimbriggs.com/computers/computer-notes/mysql-notes/mysql-data-types-50.file

[JMickle]

4.3GB per entry

STILL NOT ENOUGH 

[Dacke]

Then you won't like sqlite, which has a default limit of 1GB per entry.

Fortunately, there are ways of automatically splitting big files into smaller parts. For example, MongoDB appears to use 16MB chunks. But it can seamlessly split arbitrarily large files into such 16MB pieces. Which allows you to seamlessly stream a 74YB video. Theoretically. If I'm not mistaken.

[JMickle]

ok, now we're getting there.

[Sean A.]

Have you looked into encoding the data into an image? I think is one of the easiest (and coolest) ways of transferring data.

Storing stuff as images is highly volatile. Image hosting sites often encode images to save space wihtout any warning.

Also, a point worth repeating:
Databases can easily store 1MB blobs.

For example, the MySQL LONGBLOB datatype can store up to 4.3GB per entry (if I'm not mistaken)
http://kimbriggs.com/computers/computer-notes/mysql-notes/mysql-data-types-50.file

Well I was under the impression that the OP had his own site and could choose his own CMS and control what he uses store images rather than using external image hosting.

[Dacke]

Sure, it would totally work. I just wanted to point out the risks with using images in creative ways.

[Sean A.]

Yes absolutely there are some risks with this method.

http://www.co-optimus.com/article/4722/monaco-s-levels-are-images-of-the-levels.html Really interesting thing that monaco did where the image preview of the level also contains the level data. There is a more detailed explanation on the facebook page but when I read about it, it blew my mind.

[wg/funstorm]

Actually I would much much prefer to pay a 3rd party to take care of the hosting headache for me so I can focus on the gameplay and just using a simple API to up/download files. Either through an existing service, or at least using some cloud offering like S3. Any time I spend on setting up my own, is time I can't spend on making the game better.

The services I've looked at so far (like playerIO) have database file size limits of 500KB or less (yes, even for paying customers). If you know of any that are suitable I'd love a link

[JMickle]

http://bayfiles.com/ has an API for uploading.