Version control isn't just for programmers

No readers like this yet.
Open art versioning

Opensource.com

Working with other artists and creatives, I'm constantly amazed—and, to be frank, a little horrified—when I look at their project directories. So frequently, they're riddled with files that start with the same name, but with numbered with suffices like -v1, -v2, -v3-FINAL, -v3-FINAL3, v3-FINAL3-real, -v3-FINAL5-please_will_it_ever_end, and so on.

And even more amazing, often these are artists are involved in collaborative projects where more than one person may touch the same file (or set of files), often over a long period of time. I would often ask these artists, "Have you ever thought about using any kind of version control for your projects?"

Invariably, I get the response, "What's version control?"

What's version control for an artist?

We live in a modern time, with these fancy computer things, that are supposed to help make us more efficient. Why in the world are so many of us still "versioning" our files by tacking a progression of numbers and letters to the end of them?

This problem has been solved multiple times and in a variety of interesting ways by our friends in the software development world. In short, version control can most easily be compared to "save states" in video games. You get to a point where you're comfortable with your progress and you create a save state (a commit in version control parlance). From there, you can either continue down your current path or test an idea by checking out a different path (a branch). Regardless of which way you go, you can always get back to any previous commit and start over from there. It's confusing why these tools haven't seen greater adoption among those of us who produce creative content. To that end, I'm going to give a brief introduction to how I've been doing it for a number of years now, using Mercurial.

Mercurial

The first question from anyone who already knows anything about version control software (VCS) is probably, "Why would you choose Mercurial?" It's true that there are a variety of other tools out there, in both open and closed ecosystems. On just the open source side of things, there's Subversion, Fossil (it has a built-in wiki!), Bazaar, and—arguably the most popular VCS—Git. On the proprietary side of things, there are products like Perforce and Alienbrain. Even Dropbox gives you the ability to roll back to earlier saves of a file. So why Mercurial?

It's a personal decision, so I'm sure other people might choose a different tool. However, here's a quick overview of my reasoning:

  • It doesn't require a centralized server. Mercurial is what's known as a distributed version control system (DVCS). Among other things, that means all versioning can be done locally on your own computer. So unlike Subversion or even Dropbox, you're not reliant on having a separate server set up for storing the version history of your project files.
  • It's multi-platform. Sadly, not all of the other artists and creatives that I collaborate with are on open source operating systems. And while most of the other DVCS programs have ports with command line interfaces, Mercurial has a very mature graphical front end in TortoiseHg. I recommend that folks have a good understanding of Mercurial from the command line, but a lot of artists do appreciate a good clicky interface... and the history view in TortoiseHg is quite pleasant to work with.
  • It has a simple, clear-to-understand workflow. Mercurial doesn't have the popularity or near ubiquity that Git has attained. However, for me at least, the workflow is a lot more straight-forward. For the type of work that most creatives do, such features as a staging area and history mutability don't tend to be worth the additional procedural overhead. Remembering to do an explicit commit with a meaningful commit message (don't worry, we'll cover that soon) is already more effort than many creatives want to expend. Staging files or, in the case of Fossil, making sure the repository database is open, is more than we want (or need) to think about.

There's also some anecdotal evidence that Mercurial might handle binary files (like images and other media files, a mainstay of most creative work) more efficiently. Plus, one of the best primers I've ever read on distributed version control (Joel Spolky's Hg Init) was written with a focus on Mercurial. Granted, nearly all version control systems weren't necessarily designed to support the kinds of files typical in creative production (large binary files), and there are a few systems for that (known as digital asset management software, or DAM). However, they tend to be overly specific to a specific medium or workflow style for my tastes.

So that's why I've personally chosen Mercurial. That said, there's an analogous process in most of these other systems for what I'm going to describe here. So if you'd prefer to use Git or Fossil, I say that's great. At least you're using something. That puts you a step ahead of most other creatives.

Before you get started

I'll leave the process of installing Mercurial and TortoiseHg to their documentation and your understanding of your own operating system. However, there is an imperative configuration step that you need to do before creating your first project repository. You need to identify yourself. Remember that Mercurial is a distributed version control system, designed to allow teams of people to collaborate on the same project. When you have collaboration, you need some level of accountability. You need to know who made what change. That way, you can contact the person separately for more clarification on the change.

Setting your identity is pretty easy. From a terminal window, you can type hg config -e. This will bring up a text editor with your Mercurial config file (typically .hgrc or hgrc in your home directory. In that text file, enter the following (or edit it if it's already there):

[ui]
username = Your Name 

Save and close the file and you're set to go. Of course, there's a graphical way to do this from TortoiseHg (FileSettingsGlobal SettingsCommit), but like I said, I recommend having an understanding of how this works from the command line. It makes you more effective when working from the graphical side.

Getting started with Mercurial

OK, so all that said, how does one get rolling with this version control business? Well, the first step is to create your repository. A repository ("repo" for short) is where all of the relevant files for your project are kept (and versioned). While it's possible to have a single master repository that's broken into multiple sub-repos, the recommended practice is to have a separate repo for each project you work on. The only exception I have to this is for small, single-file projects. For instance, I keep a single repository for all of my Opensource.com articles, since each article is only one file.

To start your repository (or initialize it), the process is much like starting any large project. First create a directory for the project. Then you go into that directory and initialize your repo. From the command line, the steps looks like this:

$ mkdir myproject
$ cd myproject
$ hg init

That last command, hg init, is the Mercurial command (all Mercurial commands start with hg; Hg is the chemical symbol for mercury) that initializes your repo. If you're doing this from TortoiseHg, you go to FileNew Repository and use the pop-up dialog box to choose the path to your project directory. It then runs the hg init command for you.

Of course, just having the repository doesn't do you a whole lot of good if you don't actually have any of your project files in there. And unfortunately (but with good reason), it's not as simple as copying or creating files in your repo's directory. It's not automatic like that. Any file that you want Mercurial to track (that is, keep versions of) needs to be explicitly added to the repo. From the command line (assuming you're still in your project directory):

$ hg add totally_sweet_painting.ora

This lets Mercurial know that you wish for it to track the changes to your file—in this case, totally_sweet_painting.ora. From within TortoiseHg, you can do this from the lower left pane. You should see your file listed in there with a check box and a question mark to the left of it. You can right-click the file name and choose Add from the context menu that appears.

Mercurial now knows to track changes on your file, but it doesn't yet know that any changes have actually been made. You need to explicitly commit your changes and updates to the repository. And with that commit, you need to include a note, or message that describes the changes that you've made. This may seem to be an unnecessary hassle, but trust me, when you come back to your project months later (or, in my case, the next day), it's really useful to have that extra bit of information about your changes so you get a better idea of where you were going. To paraphrase a cliché, those who don't know their version history are doomed to repeat past mistakes in their projects.

In the simplest kind of commit, you include the message about your change all in the same line, like so:

$ hg commit -m "Added my totally sweet painting to the repo."

To do the same from within TortoiseHg, use the Commit button on the far right of the interface. When you click that button, a dialog box appears. That dialog shows the changes in the commit (or changeset) and gives you a text area for entering your commit message.

And that's it. From here, you continue working on your totally sweet painting. As you continue you working, periodically make commits to your repository, noting the changes that you've made by modifying your file or adding new ones. You don't need to do this every time you save your file (some of us have Ctrl+S built in to our work style so much that hitting it almost seems like a muscle tic). However, it's a good idea to do commits whenever you make big changes or before you're about to step away from your work for a while. Again, use the "save state" analogy from video games as your rule of thumb for when it's appropriate to commit.

As you work, you can always ask Mercurial what files it thinks has changed since your last commit. You just need to issue the following command:

$ hg status

Mercurial will then list out which files have changed, which have been added, and which ones aren't being tracked at all. In TortoiseHg, these changes show up in the file list pane on the left side (you may need to click the Refresh button to get a current view).

For me, I typically work with both a terminal window and TortoiseHg up at the same time. I enter all of my Mercurial commands from the terminal and I can get a quick (and attractive) log my commit history from TortoiseHg. Technically, I suppose I could do all of my command line work from TortoiseHg's integrated Console pane (ViewShow Console), but I typically have a terminal open anyway, so it's not that big of a deal for me.

Knowing what should be versioned

Once you've figured out how versioning works, it's going to be tempting to use it on everything in your project. While there's some value in doing that, it's not necessarily the most efficient way of managing your project repo. For example, if you tend to collect reference files (video, photos, mood board illustrations, etc.), those files don't tend to change very much and aren't really part of your project proper, so they don't really need to be versioned.

That's on the input side of things. On the output side, I tend to use a similar methodology to how software developers use version control. That is, I only version my "source" files, but not the output files derived from that source. For example, if I'm working on a design in Inkscape, I'll include my SVG file in the repository. However, if I need to deliver an exported PNG or PDF or EPS file as the finished work, I don't version those files with Mercurial. I can always (with relative ease) re-export from the source SVG, so it doesn't matter as much if those export files get lost. Therefore, I don't need to clutter my repo by tracking them.

This can be a bit muddy on bigger projects like 3D animation where regenerating frames of animation is a computationally intensive and time-consuming process that you don't want to repeat if you don't have to. For those kinds of projects, though, I still prefer to keep the output out of my main repository. If versioning those files is really critical, I may opt to do a secondary repo or a sub-repo just for holding that output, but that's a bit more of an advanced topic.

Of course, on the topic of version control, there's a lot more involved than I've given in this little article, stuff like cloning repositories to other computers for collaborating or working remotely, creating branches to safely experiment with ideas and variations, and tagging commits as important milestones. However, this should be enough to get you started down the path of proper versioning and project directories forever de-cluttered of those abominable v2, v3-FINAL, v13-FINAL8 (and so on) files.

If you don't mind, let me know what you think of this kind of article in the comments. If there's interest, I can certainly continue talking about this in future Open Art columns.

Have fun creating cool stuff!

User profile image.
Jason van Gumster mostly makes stuff up. He writes, animates, and occasionally teaches, all using open source tools. He's run a small, independent animation studio, wrote Blender For Dummies and GIMP Bible, and continues to blurt out his experiences during a [sometimes] weekly podcast, the Open Source Creative Podcast. Adventures (and lies) at @monsterjavaguns.

16 Comments

Great article! I've been meaning to set something up for years but have never come around to actually do it. I'll give it another go now.

I do the same but with Subversion and Tortoise. I use the SVN for Blender files and Python/C

Nice! Subversion is still a good choice for creative projects (particularly collaborative ones). It's the VCS I started with and used for years. What happened to me is that I got lazy and didn't want to set up a server just for Subversion. :)

In reply to by antonioya (not verified)

Just to throw it out there, I use subversion on my solo projects on the same machine as the "server" without any issue. At least on Windows and with Tortoise, it's fairly painless. Might consider Hg, though.

In reply to by Jason van Gumster

Thanks for pointing this out, Tony. You're absolutely right. I probably didn't adequately explain how incredibly lazy I am (one "hg init" command vs. just 2-3 simple steps for a local Subversion repo).

In reply to by Tony (not verified)

That's totally fair, and probably the reason I'll try Mercurial soon. It's often not so much the time savings, as it is lowering the barrier / removing roadblocks. The mental burden of extra steps tends to add up, especially with something like art where one might be spinning up new repos fairly routinely.

I've stuck with SVN because to the best of my knowledge, it handles binaries better than GIT - including the ability to do deltas. But I've also heard it said those deltas don't really save much. I'm certainly open to looking around.

While I'm thinking about it, might be a good time to remind people that version control by itself should probably not be your backup solution. :)

In reply to by Jason van Gumster

I remember some time ago that there was an article for using version control for documents. I know that in my shop it is used.

This is what I found using a search engine.

http://www.makeuseof.com/tag/not-just-for-coders-top-version-control-sy…

I am not surprised that it can be used for artist that collaborate each other. I think it is a good idea being that it keeps everything organized and provide the ability to back track in case that something goes horribly wrong.

Version control is not the only technology that can be used for other thing. For example, I keep track of my house chores and projects using Mantis (www.mantisbt.org). It is better than Hi and Lois' chore jar. :)

At home I have a cvs repository for personal work. The reason why I use cvs is just familiarity with it.

Wow, CVS? Really? I did *not* expect to see that. But you're right. It's familiar and and it still works. I can certainly see it being useful in writing. And I like your use of Mantis... I do something similar with Redmine.

Also, thanks for sharing that link... that's a really nice rundown of choices for people with a writing focus.

In reply to by Papo Anaya

my tip of the day:
use secret branches and use graft to insert the cherry picket versions into the main branch

Oh, how I do love graft. I've only run into problems with it when I want to graft changes from a single file in a commit that included more than one file.That's pretty easy to work around, though.

Thanks for sharing!

In reply to by SaltatorMortis (not verified)

I prefer GIT. Used SVN before, but GIT is lightyears ahead.

Jason, nice article. I started my career as a software entrepreneur by founding and operating a version control company back in the mid-80's. I remember trying to make this argument back then and people's eyes just glazed over. (Of course, I also remember one engineering manager explaining to that they had no need for our product because "they were a small shop with only 15 programmers." Times have changed.)

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.