next »

[Jonathan Burroughs]

Hey all!

I think this is my debut post, so I think I'll head to the Townhall after this and introduce myself properly.

So I'm in week 4 of my first foray into the world of independent game development. And I woke up this morning in cold sweats that I don't have a very satisfactory way of backing up my work.

I'm using Google Drive / Docs as much as possible for keeping my documentation and artwork backed up. I'm currently playing around in Unity for my software prototyping. A cursory scan of the Unity forums has pointed out the many ways Unity doesn't play nice with solutions like Dropbox, although I understand there are ways of solving these problems.

I was just interested in opening up a discussion on the subject of backing up work and good versioning etiquette.

Doubly interested in good practice for working with backups when working collaboratively. My partner / comrade in chief is in Dublin and I'm based in London, so we're figuring out how best to share work and collaborate effectively. I figure we're way too small to be worried about Unity Pro and proper version control software.

Anyway, any tips, advice or anecdotes greatly appreciated. And again hello to everybody! And please don't be afraid to just drop a link to a thread which already serves this purpose. I did a search before writing this, but I acknowledge I may not have been thorough enough. Eek!

[Bandreus]

Welcome to TIGS!

When it comes to backing up my work I'm a bit paranoid about cloud-services. Maybe it's only me though.

That being said, I think Google Drive is a good solution for sharing doc files and collaborative work across long distances.

If we're talking backup specifically though, I wouldn't put my trust on online services alone. Suppose you need to recover your work and you don't have access to the internet for whatever reason. I reckon nowaday that's kind of a remote possibility, but still something worth keeping in mind.

Here's my thoughts on how to set up a simple yet effective back up strategy which keeps in mind different factors.

First of all, you want to have a local copy of ALL your work which is readily available at all the times. This is as simple as purchasing an external hdd (those are ridicolously cheap in this days and age) and copying your gamedev folder as is on a regular basis (weekly might suffice, but this can be varied effortlessly depending on your needs). If you do all your work on a desktop PC (as opposed to a laptop), this can be as effortless as leaving the external unit hooked up all the time and scheduling a cron-job for backing up your work folder automatically, say every saturday at midnight or w/e.

Oh, if you can afford the disk space (this might quickly build up) never delete older copies of your buckup: at times you realize you accidentaly deleted an old file from months ago, being able to fire up a previous copy of your back up can be invaluable in such an instance. I will mention some software can achieve this while saving disk space by storing diff data (as opposed to multiple, distinct copies of the same files) such as, for instance, Apple's Time Machine. The back-up software topic is vast and I'll leave the task of researching all the different soultions/technologies to you. Unless you need a very specific solution though, simply coping over the entire working folder should suffice.

You can supplement your master hdd back-up by burning all of your stuff on DVDs, though less frequently for obvious reasons. Say, you burn your latest back up once a month (or even less frequently if you find that's too much of a chore). External hdds can fail too, trust me. Better safe than sorry.

Keeping docs on google drive is okay, mainly because of the collaborative-work capabilities. You can store other resource files (art/audio/whatever) in there too if that works for you. Or possibly use Dropbox.

Oh, if you can afford it, consider a pay-for online backup service. Hundreds of those are available in various price-ranges and with different value of service, so you might have to do some research in order to find the one which suits your needs best. This might seem like an odd advice in the era of "everything is free on the internet", but think about it this way: as an indie dev, your files are the most valuable asset to your business. If you can get great value and increased safeness in exchange for a few bucks, don't even think about it twice.

Last but not least, you should really think about using source control. This is not necessarely limited to source files only (SVN/GIT/whatever are great tools for managing your docs and assets too, for instance).

I feel like every developer should use source control regardless of back-up needs, and even when you're the only person working on your project.

That being said, setting up proper source control can be a bit hard for a first-timer, and setting up a dedicated machine as a master repository (esp useful for team work) might be a bit prohibitive.

In all of this cases, my best advice would be using an online service. You can get 5 private repositories for 7$/mo only. That's great value. Just like with online back-up services, a number of source control providers come at different price ranges and offer different services, so you do have choice. For commercial products though, you do want a private repository as a minimum, and those don't come free (as far as I'm aware of, at least. But I might be wrong).

That's that. Sorry for the long and poorly thought out post. I'm not pretending this is a complete guide on the subject of secure back-up (i.e. I didn't even mention raid configurations) but I wanted to keep it simple and offer solutions which almost everyone could set up in an easy fashion and without having to dish out a ton of money.

Hope it helps!

[Jonathan Burroughs]

Wow. Thank you, Bandreus. Not poorly thought out at all. This is very thorough and entirely useful. If nothing else you've impressed upon me the importance of making multiple backups, with plenty of redundancy.

On a previous project we used Google Drive for docs, Dropbox Pro for art source assets and Unity Pro's source control for game assets and code. But this was within a larger company, with robust policies and infrastructure for keeping backups.

I think my first business investment is going to be an HDD caddy and some high capacity drives.

[Garthy]

Welcome.

I figure we're way too small to be worried about Unity Pro and proper version control software.

Re version control software, give this a closer look. There are countless benefits to using proper version control even on a solo project, and if you're working collaboratively, you'll be doing unnecessary work performing manual merges and chasing up "lost" code/assets if you don't use something.

About the only time I wouldn't use version control software of some sort would be very early on in development on a project where you know very little about the environment and are essentially putting together rapid-fire experiments to get your bearings. Once you find your feet, version control is extremely useful. Even the ability to instantly get a visual diff against your last checkin without having to track which backup of your source is the most recent is reason enough to start using such a system, and that's just the tip of the iceberg as far as benefits go.

Having said that, I'm not terribly experienced with Unity, and have heard it can be a bit fiddly to get working with external version control. This might be something that influences your decision.

Best of luck.

[Bandreus]

On a previous project we used Google Drive for docs, Dropbox Pro for art source assets and Unity Pro's source control for game assets and code. But this was within a larger company, with robust policies and infrastructure for keeping backups.

Whatever works best for you! Obviously your development pipeline, working environment and needs are all to be kept into account when it comes to planning out setting up your back up policies.

Any sort of back up is better than no back up at all, and reduntant back ups are even better.

It all depends on your needs and the amount of resources you're willing to spare. I.e. NAS units are available for only a few hundred bucks which are perfectly capable of handling back ups with the added safety of a raid set-up, and can often double up as an actual multi-purpose server.

For a small indie studio working on a small-to-medium sized project though, I feel like a simple in-house solution is more than enough. A local copy on your workstation plus backups on an external device supplemented with in-the-cloud copies, while not entirely safe, can easily save you from most disaster-scenarios.

I will reiterate the importance of source control though. If I had to pick between a proper source control solution and additional back up infrastructure I would go for the former without a doubt. I have no experience with Unity, so I'm not the best person for giving advice about source control set-ups in that environment though.

[Rusk]

When you are planning a backup solution, it is helpful to start with listing everything you are protecting yourself against. If you want a "complete" solution, it can become very complicated.

Also, make sure you know exactly how to use the backup. The easiest and most time consuming is to reinstall all software from scratch and then copy your data from a backup. But if you have some smarter recovery tool, make sure you know how it is used before you need it, otherwise you might not use it correctly.

* Hard drive failure
      Raid for immediate recovery
      Local backup for quick recovery
      Remote backup for slow recovery
* Accidental deletions, file history
      Multiple snapshots
      Rotating backups
* Bit rot (is this a real thing?)
      Checksums
      Special file systems (ZFS, Btrfs)
* Theft (would a burglar take an external hd from your desk?)
      Remote backup
* Fire and other catastrophes
      Remote backup
      Keep the password for the email and backup service somewhere safe!
* Ransomware, encrypting valuable files
      Rotating backup
      Checksums
      Keep older snapshots read-only or inaccessible

[Fallsburg]

Unity has gotten a lot better about version control (it was actually the reason I didn't use it for the longest time), you just need to make sure to set the project settings to do asset serialization and text only mode.

[Jonathan Burroughs]

Currently taking a crash course in Git:

http://git-scm.com/book/en/Getting-Started-Git-Basics


Is Git most people's preferred version control? I hear a lot of positive things for SVN, particularly on the Unity forums.

[Julien]

Last but not least, you should really think about using source control. This is not necessarely limited to source files only (SVN/GIT/whatever are great tools for managing your docs and assets too, for instance).

I feel like every developer should use source control regardless of back-up needs, and even when you're the only person working on your project.

Totally agree on this one !
I personally use dropbox with copies on several computers, some using ntfs, others ext4, including some that are rarelly turned on.
Beside the privacy issue (wich I don't really mind since my work will eventually be open source) there is another issue. Some viruses can encrypt your file system and ransom you. I suppose it would affect the dropbox directory as well, that's why I took care to put my files on less used computers.

[Fallsburg]

SVN is horrible and shouldn't be used anymore.  The world has turned and the sun has set on the empire of SVN.  git or mercurial are both fine and I have no strong opinions one way or another.

[apookapus]

Is Git most people's preferred version control? I hear a lot of positive things for SVN, particularly on the Unity forums.

I like both, we use subversion at work and I've used it exclusively for personal projects, though just recently I've started using git. For me, at least, pretty much any version control program would fit the bill -- I check out, commit, diff, revert, branch and tag, and so long as the VCS can cope with those operations, is reliable and works in both Windows and Linux, I'm set.

The only reason why I started looking at git was because there's a lot of talk about it. So far, I find myself falling in love with the local branch and merge support... makes it super easy to create a local branch to try something out and then merge in my changes if they work out. I can switch between branches or go back to the master any time that I want.

I do use an old Linux machine as a server, so I have the working copy on my machine, the repository is hosted on the Linux box, and then a cron job kicks off once a week to back up all my repositories to dropbox (the backup file is encrypted with GPG before it's copied to the dropbox folder). I guess it won't stop the NSA, but I figure that it's good enough.

[Jonathan Burroughs]

This is all very exciting and illuminating to hear. Equal parts essential knowledge and slightly terrifying information overload.

I'm very much the newbie when it comes to setting up these kinds of services, so I'm going to immerse myself in whatever documentation I can find.

It sounds like an ideal setup would be that I convert one of my old, unused PCs to a server which I can use to host the depot and both myself and my collaborator in Ireland can sync to and from that via our work PCs. And periodically I take an image of the depot as an additional backup.

Based on a cursory Google, I'm coming down slightly in favour of trying Git above SVN. It seems like there's wider support and learning resources for it.

I can but try!

Thanks so much for all your feedback and suggestions by the way. This is all very encouraging.

[clockwrk_routine]

I use Sourcetree a gui frontend for Git/mercurial.  here's a quick tutorial I wrote: http://mekongswells.tumblr.com/private/75923479402/tumblr_n0n7xsBBLX1r68dk0

[Jonathan Burroughs]

This is awesome. Thank you!

[oodavid]

Version Control

When a programmer learns how to properly use version control he evolves from a smelly caterpillar into an enlightened butterfly.

Seriously, there's life before version control and life after.

For me, it's GIT through-and-through although I have used SVN and Mercurial before, both a step in the right direction but for me GIT is a pleasure to use.

[jgrams]

I've heard good things about this Git tutorial/doc thing: http://www-cs-students.stanford.edu/~blynn/gitmagic/ch01.html

[ThemsAllTook]

SVN is horrible and shouldn't be used anymore.  The world has turned and the sun has set on the empire of SVN.  git or mercurial are both fine and I have no strong opinions one way or another.

git may be trendy right now, but svn is still a perfectly viable choice. If you're exploring options, there's no reason not to consider it. It's not obsolete by any stretch of the imagination.

[Garthy]

I've been using Subversion since before Git was a gleam in Linus's eye. It is a very capable version control system and an essential development tool in my work. Cold dead hands and all.

Which is better, Subversion or Git? Myself, I don't know, I've not used Git enough to be able to offer an informed opinion of their comparative merits. There are plenty of opinions out there though, a search term of "git versus subversion" provides plenty of bedtime reading.

Early on in the piece Git was pretty flaky, had minimal tool support, etc, etc. Point me to a project that isn't like this in its early days though. However, it has been in use in one of the world's largest distributed software projects for close to a decade, and has had plenty of time to mature. And from what I have read, it has.

To someone starting out, I'd probably suggesting experimenting with both. Of the two, I'd probably start with Git first nowadays, despite my personal unfamiliarity with it, because from what I have read it is the superior tool for distributed and offline development. However, if it turned out to be far more difficult than Subversion to set up, I'd use Subversion instead, as you can always pick up Git later.

Oh- on a related note, there is an Eclipse survey that shows a sampling of version control systems in use. From 2010 to 2013, Subversion use dropped from 58.3% to 37.8%, whilst Git use grew from 6.8% to 36.3% (*). Given the direction of the trend, knowing Git could well be the more valuable skill in a few years.

Source:
http://www.slideshare.net/IanSkerrett/eclipse-survey-2013-report-final
(skip to page 14)

(*) I combined Git and Github scores as being Git.

Also, this is interesting:

http://www.ohloh.net/repositories/compare

All IMHO. Well, except for the cited stats of course.

[Jonathan Burroughs]

At the risk thoroughly exposing myself as the newb I am, can anybody recommend a good GUI for Git? I've taken a look at the Git Wiki and it appears I'm spoiled for choice!

https://git.wiki.kernel.org/index.php/InterfacesFrontendsAndTools#Graphical_Interfaces

Conscious I'm branching (forking?) us into a more specific topic, so it might warrant its own thread.

[Fallsburg]

git may be trendy right now, but svn is still a perfectly viable choice. If you're exploring options, there's no reason not to consider it. It's not obsolete by any stretch of the imagination.

A person could also use CVS.  It's not like it has stopped working.  But why would you want to?  If I was a person coming to version control, with no prior knowledge, and no legacy systems tying me down, why would I choose the older, less capable system?  And it is obsolete.  By definition, newer, better things exist, making it obsolete.