Python file-management tricks for digital artists

Read Part 1: Automating repetitive tasks for digital artists with Python

If you've done digital artwork for any appreciable amount of time, then the importance of good file management should be apparent to you. This is even more true if you're collaborating with other artists. Everyone has their own favorite naming conventions and project directory structures. It can be pretty frustrating when you try to find files that are supposed to be named a certain way, but one of your collaboration partners thinks it'd be more interesting to name each after a Three Stooges pun. (Hey, it happens!)

This frustration gets amplified once you start automating parts of your process with scripts. Now it's your code, rather than you, that can't find the right files. Worse still, most of those scripts won't look for a workaround for a slight naming change. They simply won't work at all.

Fortunately, with a few relatively simple lines of code, you can help alleviate issues like this. Let's work with an example not caused by self-amused collaborators. Sometimes the problem could be your own fault. I personally never (ahem) make mistakes. But occasionally, the programs that I use do exactly what I tell them, rather than what I intend them to do.

Case in point

Animation is a big part of what I do. When creating animation or visual effects, to output (render) each frame of your animated work as a separate image file is good practice. (Sometimes you render multiple images per frame, but allow me to simplify to start.) Typically, those frames of animation are all put in their own directory on your hard drive.

Now, I'm not sure if this has happened to anyone else or if it's just a quirk of mine, but there have been times (in multiple software packages) in which I've picked the directory where my render files should go, but then there's a glitch. (Some may say "user error", but remember, I never make mistakes.) Rather than saving my renders to project/render/, the last slash gets left off and each rendered image file starts with the word "render" rather than going into the render directory. That is, I want frame 1 of my animation to be project/render/frame0001.png, but instead the program creates project/renderframe0001.png. Now my main project directory is flooded with thousands of render files. Gross.

I have a couple options at my disposal. The simple solution would be to move all of those render files to the correct directory and simply tolerate the poor naming. However, this can be problematic if my post-production steps expect a naming structure without that render word at the start of each file name. It could get even more complicated if I need to re-render.

Speaking of re-rendering, that could be another option. I could just delete the improperly named files, fix the output path in my animation program, and redo all of the rendering. The problem there, however, is that rendering frames of animation can sometimes be very time-consuming. For complex scenes, it may take well over an hour for a single frame. Multiply that by 24-30 frames per second of animation and we can quickly see how re-endering is not the quick and easy solution we wish it could be.

Of course, there's always the manual option: Move all of the render files to the proper directory and then go through and change the name of each one. Sure, if you only have a couple dozen frames that might not be such a hassle. If you have thousands of frames of animation, it's very much a hassle.

Python script solution

So what's left? That's right: Write a script! As with the Part 1 in this series, we'll be doing this in Python. In Part 1, we used the subprocess module. This example doesn't require that, but it does use another one of Python's built-in modules, os. The os module provides a means of doing tasks—such as moving and renaming files—handled by your operating system. And because Python is multi-platform, the os module works everywhere Python does, regardless of the actual operating system you're using.

So the quick and dirty version of your script might look like this:

import os
    
for filename in os.listdir('./'):
    if filename.startswith('renderframe'):
        os.rename(filename, filename[:6]+'/'+filename[6:])

If you've never coded before, this script has a few things with which you might not be familiar. Let's start with what you do know. The first line, import os, makes your script aware of Python's built-in os module (similar to how you imported subprocess in the previous article).

The next line of code (extra line break notwithstanding) indicates the start of a loop. Loops are one of the primary time-savers in scripting and programming. Basically, if there's a process that you need to do over and over (such as renaming a bunch of files), then loops are there to save the day (and your sanity).

In this particular example, you're making use of a for loop, which is a particular kind of loop that you use to iterate through a series of things. In this case, you're iterating through the name of all of the files in your current directory. How are you doing that? Let me introduce you to os.listdir.

The Python os module has a function named listdir. That function will take any directory path as input and give you back a list of all the files in that directory. In this example, you're using os.listdir('./'). The './' bit is a string of text (hence the quotes) that's shorthand for "the current directory I'm in right now."

"Great," you might be saying, "So os.listdir('./') makes a list of all the files in my current directory, but what does that have to do with a for loop?"

Good question! You don't want to work on the whole list of files at once. You need to address each one, one at a time. The for loop iterates through the list of files provided by os.listdir('./'). Because your for loop needs a generic name to call each file while it's being worked on, we're using the variable filename as a stand-in.

Knowing this, have a look at the full line that's setting up your for loop: for filename in os.listdir('./'):. An English translation of this line would be, "Make a list of every file in my current directory. Then loop through the list of file names. To keep things simple, just call every file filename while it's being worked on."

On the next line of code in your script, you're in the loop (you can tell, because that line is indented). Because you're in the loop, everything you do here will be repeated for every single file in your current directory. Remember that we started this because my rendered files ended up in the wrong place—likely there are other files here that aren't my render frames. We need to make sure that this script isn't renaming and moving just any file; it needs to limit itself to only those rendered frames.

Filtering and renaming

Fortunately, we have a good way of doing this. We can filter based on the horrible mistaken naming that I told my animation software to produce. All of the poorly placed, improperly named files in the current directory start with renderframe.

For each file that the script loops through, it needs to check and see whether that file starts with renderframe. That's exactly what this line of code does: if filename.startswith('renderframe'):. This line of code uses another common construct in scripting and programming, the if statement, or conditional. It starts with the word if and is then followed by a test condition. That test condition must be either true or false. If the test condition is true, then the script can do a specific bit of code stipulated by the if statement. If the test condition is false, then that bit of code gets skipped.

In this example, the test condition uses the startswith function that's built-in to all strings in Python. As its name indicates, if the string starts with whatever bit of text you give as input, the startswith function returns true. Otherwise, it returns false. So, to translate if filename.startswith('renderframe'): to English, it would read, "If the current file name in our list of file names starts with the text, 'renderframe', then do the next bit of code."

Alright. You've got a list of files in the current directory and you've narrowed that list down to just our misplaced render files. Now for the actual work of renaming and moving those files to where they're supposed to be. Fortunately, this renaming and moving step can happen in a single line of code, using the os module's rename function.

The os.rename function takes two input parameters: the file you want to rename and what you want to rename it to. The cool part, though, is that those input parameters treat the file's path like it's part of its name. So if you include a different path as part of the second input, you can rename and move your file in a single go. Hooray for less typing!

Now, looking at that second line of text (os.rename(filename, filename[:6]+'/'+filename[6:])), the first half is pretty straightforward. The second half—after the comma—that's another little bit of funkiness that you may not have encountered before. It's not difficult to unpack, though. It's just a matter of understanding what you want to do.

Let's say your script has started and it's working on the file renderframe0001.png. To rename and move your file, you just need to add a / character after the word render. That word, render, is six letters long. With that little tidbit of information, you can construct a new path for your file using what you already have in your filename variable. You just need the right notation.

I've always liked Python's notation for getting a subset of a string of text. It's [start:end], where start is the first character of your subset and end is the character that's after last character in your subset. And like every sane programming language, Python starts counting at the number zero. So in our example where we're working with a filename variable that has the text renderframe0001.png, you could use filename[3:7] and Python would give you derf as the result.

You might notice that your code not only uses this notation twice, but in each case, it's either missing the value for the start character or the end character. This is a cool little convenience trick. If you only provide a start character, but keep the colon in the notation, Python assumes you want every character in the string after that point. Likewise, if you only include the end character value, Python will give you all of the characters in the string that are before that. In our example, filename[:6] gets you render. The filename[6:] notation gets you frame0001.png.

Using this technique, you split your filename in half after the word render. Now all you have to do is reassemble it with that additional slash (/). So, pulling it all together, this line of code (os.rename(filename, filename[:6]+'/'+filename[6:])) translates to, "Rename the my file by inserting a slash after the sixth character in the file name."

Adding user feedback

That's it for describing your quick and dirty script for moving and renaming a whole bunch of misplaced files. The only thing that might be worth adding to it is a bit of user feedback. If you're moving and renaming thousands of files, it might take a minute or so. It would useful to know the file that your script is working on. You can do that with a little print statement just before your rename. Your finished script might look like this:

import os
    
for filename in os.listdir('./'):
    if filename.startswith('renderframe'):
        print('Moving and renaming:', filename)
        os.rename(filename, filename[:6]+'/'+filename[6:])

And there you go! This little handful of lines of code can save you an immense amount of time if, like me, your software makes the mistake of doing exactly what you asked of it.

This block of code also serves as a great starting point for other useful file-management scripts. For instance, I love the File Output node in Blender's compositor. I even use it when rendering multiple passes of a still (non-animated) frame. The downside, however, is that the File Output node always tacks the current frame number to the end of every file it generates. That's fine for animations, but it's a bit annoying when I'm just rendering stills. I end up with a bunch of files that end in 0001.png.

Fortunately, with a couple minor modifications to my move and rename script, I can easily trim off those 0001s in one go:

import os
    
for filename in os.listdir('./'):
    if filename.endswith('0001.png'):
        print('Renaming:', filename)
        os.rename(filename, filename[:-8]+filename[-4:])

The differences between this script and the previous one are pretty minimal. Instead of dealing with the start of the file name, this script works from the back of it. So rather than filename.startswith, this script uses filename.endswith as its filtering mechanism. And instead of inserting a slash after the sixth character, this script modifies the characters before 0001.png (that is, every character until the eighth from the end). Notice the negative number in filename[:-8]. That negative value tells Python to start at the end of the string instead of the beginning.

And there you go! Now you have a way to change the beginning or end (or middle!) of a bunch of files all at once. You save time and avoid doing boring, repetitive tasks so you can focus your energy on doing more interesting creative work.