Catch up on the series, Python tricks for artists:
- Part 1: Automating repetitive tasks for digital artists with Python
- Part 2: Python file-management tricks for digital artists
- Part 3: Using Python to find corrupted images
Scripts are a great way to automate the tedious parts of your creative work and allow you to focus the bulk of your energy on the fun and interesting parts. There comes a time, though, when a quick one-off script gets used frequently enough that it becomes a utility. When that happens, it's often not sufficient for the script to just be this quick thing that you spit out that runs on its own without any interaction with you. The script will need to take input arguments, ask you meaningful questions, and then act upon them.
Let's take the script from the previous article of this series, Part 3: Using Python to find corrupted images. It was a handy little thing for listing out any corrupt image files in a directory. That's great, but not having to copy that script into every directory you want to test for bad images would certainly be more convenient. Also, if your script could delete the corrupt images after finding them for you would be even nicer. Of course, deleting files can always be a little scary. You want to be sure you're only deleting the bad ones, so having a bit of prompting and confirmation from your script would be good, so let's do that.
Next is the script as it existed at the end of the last article:
from os import listdir
from PIL import Image
for filename in listdir('./'):
if filename.endswith('.png'):
try:
img = Image.open('./'+filename) # open the image file
img.verify() # verify that it is, in fact an image
except (IOError, SyntaxError) as e:
print('Bad file:', filename) # print out the names of corrupt filesAccepting command line arguments
The first useful bit of convenience to add is to get your script to work in any arbitrary directory that you choose. You could have the script prompt you for the directory name, but I prefer to make this an input argument when actually running the script. There are justifications either way, but for me, it's primarily about laziness. If I specify the directory path when I run the script:
python remove-corrupt-pngs.py /path/to/directoryI can take advantage of my shell's built-in tab completion feature and, as such, type less (well, type the same amount by pressing fewer buttons on my keyboard).
To take input (or arguments) from the command line, the simplest way is to use another one of Python's default modules: sys. Specifically, you're interested in sys.argv, a list of words (separated by space) that's entered at the same time that you launch the script. The list is a fundamental data structure used frequently in Python scripts. Think of them as a series of variables all grouped under the same name.
Next is a quick little test example. Make a new Python script called list_test.py and populate it with the following code:
import sys
print('Length of list:', len(sys.argv))
print(sys.arv)Save your script. Now when you run it from the command line (python list_test.py), the output from your script should be as follows:
Length of list: 1
['list_test.py']Try adding an argument or two on the command line when you launch the script (e.g., python list_test.py blahblah blah). The script will output something like this:
Length of list: 3
['list_test.py, 'blahblah', 'blah']The first thing to notice is how to find the length of a list, which is as simple as using the len() function on the list. The next thing to notice is the notation of the list itself. This was covered a bit in the first article of this series, Part 1: Automating repetitive tasks for digital artists with Python, but for a quick refresher, a list is surrounded by square brackets ([ ]) and each list item is separated by a comma. In the case of the sys.argv list, the name of your script is always the first item in that list. Each word after that is a subsequent item in the list. You can access each item in the list individually by specifying its index, which is an integer number, within square brackets after the list name (e.g., sys.argv[0]).
A quick digression about that index
It's worth it now to take a quick digression and talk about indices. Whereas most people (and some oddball languages—I'm looking at you Lua and MATLAB) start counting at 1, Python and most modern programming languages don't; their counting starts at zero. Therefore, even though the name of your script is the first item in the sys.argv list, you access it with sys.argv[0]. If you use sys.argv[1], you'll get the second item in the list (in the previous example, that would be blahblah).
And now back to the script
Now that you know how to check the length of a list and access individual elements in it, you can make your corrupt image-finding script a little bit smarter. First, assume that the first argument after your script name is the path to the image directory you want to check. It would be wise to assign that path to its own variable. I'll call it imgdir. To assign your path argument to the imgdir variable, you can just use imgdir = sys.argv[1].
But there's a catch: What if you forget to specify the directory name when you launch the script? It would be nice if your script would check for that. Fortunately, you can check the length the sys.argv list. If the sys.argv list isn't 2, then you know a path hasn't been specified. The block of code would look something like this:
if len(sys.argv) == 2:
imgdir = sys.argv[1]
else:
print('You need to include a path to the directory you want to check.')
exit(2)This little block of text checks to see if there are two items in the sys.argv list and assigns the second time (index 1) to the imgdir variable. Otherwise, it prints an error message and quits (exit(2)). As a bonus, this also quits if your directory path has spaces in it, but you didn't properly wrap it in quotes. (Spaces in file names can be a pain for scripting. Avoid making such atrocities in your file naming conventions.)
Bolt this block of code to the top of your script. Then, you'll need to make your script take advantage of that imgdir directory, which is pretty straightforward because it is mostly a matter of swapping './' with imgdir. After doing that, your script should look something like this:
import sys
from os import listdir
from PIL import Image
if len(sys.argv) == 2:
imgdir = sys.argv[1]
else:
print('You need to include a path to the directory you want to check.')
exit(2)
for filename in listdir(imgdir):
if filename.endswith('.png'):
try:
img = Image.open(imgdir+'/'+filename) # open the image file
img.verify() # verify that it is, in fact an image
except (IOError, SyntaxError) as e:
print('Bad file:', filename) # print out the names of corrupt filesNow you can run your script from anywhere and check any directory your computer can access; no more copying the same script all over the place.
Prompting for user input
To get your script to delete those bad files for you, is actually pretty easy. You just need to use the remove() function that's built into the os module. However, don't go doing that just yet.
It always pays to be extra careful when you're deleting files. When they're gone, they're gone. For that reason, you should have the script that is deleting files first tell you exactly which files it wants to remove and then ask you for confirmation, so you'll add that part to the script first.
The simplest way to prompt for input in a Python script is to use the raw_input() function. At face value, it looks a lot like the print() function. You put a string between the parentheses and Python spits that out to the screen. The difference, however, is that raw_input() will then wait for you to type something and press Enter. Anything that you typed is stored in a variable. Your script can then read the content of that variable and decide how to act from there.
In the case of this specific script, you want it to ask if it should delete a specific file. The line of code for that might look something like this:
delete_files = raw_input('Would you like to remove this bad image? (y/N)')Whatever the user enters will be stored as a string in the delete_files variable. For deleting files, the way I like to handle it is to use "no" as the default response. Only if the user types some variation of "yes," will the script proceed to delete anything. This chunk of code looks something like this:
if delete_files in ['Y', 'y', 'Yes', 'yes', 'YES']:
print('Removing bad image.')
# code for deleting image goes here (we'll get to it, promise)
else:
print('Leaving bad image in place.')Pulling all the pieces together so far, your script should look something like this:
import sys
from os import listdir
from PIL import Image
if len(sys.argv) == 2:
imgdir = sys.argv[1]
else:
print('You need to include a path to the directory you want to check.')
exit(2)
for filename in listdir(imgdir):
if filename.endswith('.png'):
try:
img = Image.open(imgdir+'/'+filename) # open the image file
img.verify() # verify that it is, in fact an image
except (IOError, SyntaxError) as e:
print('Bad file:', filename) # print out the names of corrupt files
delete_files = raw_input('Would you like to remove this bad image? (y/N)')
if delete_files in ['Y', 'y', 'Yes', 'yes', 'YES']:
print('Removing bad image.')
# code for deleting image goes here (we'll get to it, promise)
else:
print('Leaving bad image in place.')You have one problem with this setup, though. What if your script finds a lot of corrupt images? Having to confirm deleting each one individually would be pretty tedious and annoying. Having your script tell you all of the bad images and ask you to delete them in one go would be much nicer.
Although that's a good idea, you're going to need to rethink a little bit of the logic in your script. Don't worry, it's not a huge thing, code-wise. Just a minor "choreography" adjustment. Up to this point, your script has been working with bad files immediately, printing their names right when it finds them. However, if your script is to work on all bad files at the same time, then you're going to need some kind of mechanism to store the names of each of those bad files. You're going to need another list.
Instead of trying to delete a file right when it finds it, your script will first store the bad file's name in a list, which you could name something like badfiles. You create an empty list in Python like this:
badfiles = []You use just an equal sign and a pair of square brackets, and you're done. But you might find yourself asking, "Why do I need an empty list? I thought this was supposed to be a list of corrupt images, not a list of nothing."
Good question. The short answer is that you can't add things to a list if it doesn't exist first. Collecting food for a picnic is difficult if you don't start with an empty basket. With your empty list in place, you need a mechanism for adding the names of each bad file to the list. Fortunately, Python's built-in lists have a very nice way of handling this. Every list has an append() function, which you can use to add items to the list. Its usage looks something like this:
badfile.append('corrupt.png')This assumes the bad image's name is "corrupt.png." Of course, you don't need to type the actual file name; you have a variable for that already. In your script, instead of printing the name of the bad file when it finds one, it instead appends the name of that file to a list. Then, once the script is finished checking the directory for bad images, it can print that list of files to screen and then ask if you want to delete them.
When you do that, your script starts to look something like this:
import sys
from os import listdir
from PIL import Image
if len(sys.argv) == 2:
imgdir = sys.argv[1]
else:
print('You need to include a path to the directory you want to check.')
exit(2)
badfiles = [] # Empty list to store names of corrupt images
for filename in listdir(imgdir):
if filename.endswith('.png'):
try:
img = Image.open(imgdir+'/'+filename) # open the image file
img.verify() # verify that it is, in fact an image
except (IOError, SyntaxError) as e:
badfiles.append(filename) # Add bad file name to list
print('List of bad images:')
for filename in badfiles:
print(imgdir+'/'+filename) # Print each corrupted file's name with full path
print('There are,' len(badfiles), 'bad images that need to be deleted and replaced.'
delete_files = raw_input('Would you like to remove these bad images? (y/N)')
if delete_files in ['Y', 'y', 'Yes', 'yes', 'YES']:
print('Removing bad images.')
# code for deleting image goes here (we'll get to it, promise)
else:
print('Leaving bad images in place.')Deleting files
At this point, your script is happily creating a list of corrupted images and printing that list to your screen when it's done checking the whole directory. Now let's talk about actually deleting those files. As I mentioned in the previous section, deleting is as simple as using the remove() function in the os module. Of course, you need to make your script is aware of that function. Remember that right now your script doesn't know about the whole os module; you're only importing listdir at the top of your script.
You could just slap from os import remove at the top of your script and be done with it, but let's be lazy. You're already importing one function from os, so you may as well try to reuse that line for the remove() function, which, in fact, you can. Each function you want to import from a module just needs to be listed in that line, separated by commas. Your import line should look like this:
from os import listdir, removeNow your script knows how to delete files. Hooray for less typing!
As for actually using the remove() function, it's very easy. You just feed it the full path to the file you want to delete and the file is gone with nary a puff of smoke. You just need to do that for every file in your badfiles list. The code for that process just uses another for loop:
for filename in badfiles:
remove(imgdir+'/'+filename)Plug that into the code you've already written (right where the comment about promising to delete files is), and your finished script should look something like this:
import sys
from os import listdir, remove
from PIL import Image
if len(sys.argv) == 2:
imgdir = sys.argv[1]
else:
print('You need to include a path to the directory you want to check.')
exit(2)
badfiles = [] # Empty list to store names of corrupt images
for filename in listdir(imgdir):
if filename.endswith('.png'):
try:
img = Image.open(imgdir+'/'+filename) # open the image file
img.verify() # verify that it is, in fact an image
except (IOError, SyntaxError) as e:
badfiles.append(filename) # Add bad file name to list
print('List of bad images:')
for filename in badfiles:
print(imgdir+'/'+filename) # Print each corrupted file's name with full path
print('There are,' len(badfiles), 'bad images that need to be deleted and replaced.'
delete_files = raw_input('Would you like to remove these bad images? (y/N)')
if delete_files in ['Y', 'y', 'Yes', 'yes', 'YES']:
print('Removing bad images.')
for filename in badfiles:
remove(imgdir+'/'+filename) # Permanently delete files
else:
print('Leaving bad images in place.')Now you have a script that can run from anywhere because it takes user input at the command line. More importantly, your script asks for permission before doing anything dangerous (like deleting files). Furthermore, now you know how to take arguments from the command line and add some simple user interaction to your scripts.
You can use the techniques I've covered to add interactivity to any Python script that you write, which means you don't have to have random scripts floating around your work directories. You can keep your scripts all in one location and use them from there. Fewer steps, easier maintenance, and you're still automating away tedium so you can focus on your creative work—what's not to love?
1 Comment