Tuesday, January 25, 2011

Photo Workflow, Part 1

I often get questions like "Is that Photoshopped?" when people see some of my photos. That's a loaded question in my view because the general perception is that somehow Photoshop is used for altering "reality." It certainly can be, but that's not generally the way it's used by photographers. So I thought I would lay out, in pieces, what I actually do with my photos once I take them.

First, I shoot everything in RAW mode, both on my Canon DSLRs and with my Canon G10 that I use for underwater.

The first thing I do is quickly import my photos into Adobe Lightroom. I use a setting under "Catalog Settings" to "Always write changes into XMP." Many people don't recommend this, but it's crucial to how I work:
  • I'm not relying on Lightroom's database. If it gets corrupted all the important metadata is in a little .xmp file that rides along with the image files
  • I can use other programs to manipulate the xmp file's metadata and import the changes back into Lightroom
  • I use Lightroom on two computers and this is an easy way, along with Unison, to keep things in sync between the computers
After I have imported my photos, I now have an XMP along with each RAW file. Then I quit Lightroom.

Recently I've gotten into geotagging my photos (adding information about where they were taken), so that's the next step in my process. If I remember, I track where I am with my iPhone. (When I'm scuba diving, the iPhone tracks the boat, and not me, of course.) The best application I've found for this kind of tracking is EveryTrail, but there are a number of apps for the iPhone and other devices. The important thing is that they output a GPX file that can be used in the next step.

Next, I use a program called Geotag that will load up all the XMP files and the GPX file(s) and correlate them based on the time of when the photo was taken and where I was. Again, there are a number of programs that do this, but Geotag is the only one I found that runs on any computer (it's Java), is free, and works with XMP files (most like to use JPGs instead).

If I don't have a GPX file describing exactly where I was at any time, I use Google Maps to take a guess and fill in requisite fields in Geotag.

The last step, before I'm ready to begin filtering and working with my images, is to rename them all. Every camera maker seems to have a slightly different naming scheme and things like IMG_1234.JPG have a couple of problems. First, they can overlap if you take more than 10,000 photos (which I have). Second, I often use more than one camera on any given trip or gallery, so one camera may be at 1234 and the other at 4321. That means that alphabetically and chronologically things are in different orders. So I rename all my files. I use the script below, which renames things like 4c64de23. It's sort of gibberish, but each second of the day since 1970 has it's own code, so unless more than one photo was taken in the same second, it's guaranteed to be unique (it actually deals with that contingency too). The script renames all the RAW, XMP and JPG files, etc. that share a common root name. It's got options to work only with JPG files (like some of my galleries), to keep a backup JPG, and to just explain what it will do but not actually do anything.

Once all the files are renamed, the last step in this part of the workflow is to go back into Lightroom and choose "Synchronize Folder" for the folder containing all the photos. This is the same step I take if I change something on one computer and want to work on it on another. This reads all the files in fresh, taking all the metadata and development settings from the XMP files.

In the next post I'll start to answer the question I posed at the beginning: "Do I Photoshop my photos?" (At this point I haven't actually done anything to the images themselves, just done a bit of organization.)

Update: With the release of Lightroom 4, which includes geotagging, the first part of my workflow can be simplified a lot. Basically I can rename the photos as soon as I put them on my computer, import them into Lightroom, and do the same sort of geotagging with the GPX file directly in Lightroom's Map module. Or just drag and drop the photos onto the map if I don't have a GPX file.

#! /usr/bin/env python
# -*- coding: utf-8 -*-

import commands
import os
import sys
import optparse
import re
import time

outMap = {'xmp' : 'x', 'jpg' : 'j', 'cr2' : 'c', 'crw' : 'w', 'mov' : 'm'}

parser = optparse.OptionParser(usage="usage: %prog [options]")

        action="store_true", dest="test",
        help="Don't change any files, print what would happen")

parser.add_option("-j", "--jpeg",
                  action="store_true", dest="jpeg",
                  help="Stamp JPG and MOV files instead of RAW")

parser.add_option("-b", "--backup",
                  action="store_true", dest="backup",
                  help="Create a backup copy of the JPEG designated with '-b'")

args = sys.argv
(options, args) = parser.parse_args(args)

filenames = []
# Patterns for RAW files and already re-stamped RAW files
rawFile = re.compile('\w+\.(cr2|crw|CR2|CRW)')
stampedRaw = re.compile('([0-9a-f]){8}\.(cr2|crw|CR2|CRW)')
if options.jpeg:
    print "Doing jpeg files instead"
    rawFile = re.compile('\w+\.(jpg|JPG|MOV|mov)$')
    stampedRaw = re.compile('([0-9a-f]){8}\.(jpg|JPG|MOV|mov)')

def exivTS(fileName):
    seconds = 0
    cmd =  'exiv2 pr "%s" | grep "Image timestamp"' % (fileName)
        status,output = commands.getstatusoutput(cmd)
        for line in output.split('\n'):
            if line.find("Image timestamp") > -1:
                timeString = line.split(' : ')[1]
                #print "Time is ",timeString
                dt = time.strptime(timeString, '%Y:%m:%d %H:%M:%S')
                seconds = int(time.mktime(dt))
    return seconds

# Collect and categorize filenames
for fileName in os.listdir('.'):
    if not rawFile.match(fileName):
    if stampedRaw.match(fileName):

existingTimes = {}
bumpedShots = 0
for infile in sorted(filenames):
    bumped = False
    seconds = 0
    outString = ''
    outList = []

    inString, extension = infile.split('.')

    # Find time from XMP
    testFileName = inString + '.xmp'
    seconds = exivTS(testFileName)
    if seconds:
        timeSource = 'X'

    # Find time from CR2/CRW
    if not seconds:
        seconds = exivTS(infile)
        if seconds:
            timeSource = 'R'

    # Find time from JPG
    # Take timestamp of CR2/CRW
    if not seconds:
        seconds = int(os.path.getmtime(infile))
        timeSource = 'T'

    while seconds in existingTimes:
        bumped = True
        seconds += 1
    existingTimes[seconds] = True
    if bumped:
        bumpedShots += 1

    newFilename = ("%x.%s" % (seconds,extension)).lower()
    outString = "%x" % seconds

    if options.test:
        cmd = 'mv "%s" %s' % (infile, newFilename)
        status,output = commands.getstatusoutput(cmd)
        if status:
    sideExts = []
    if not options.jpeg:
        sideExts = ['xmp','jpg','JPG']
    fullExt = '.' + extension
    for ext in sideExts:
        # This may be too risky (screwed up once already) Try with part/ext directly?
        #newExt = '.' + ext
        #print "Constructing",infile,newFilename,fullExt,newExt
        oldXMP = inString  + '.' + ext
        newXMP = outString + '.' + ext.lower()
        if os.path.exists(oldXMP):
            if options.test:
                #print "Rename %s to %s" % (oldXMP, newXMP)
                cmd =  'mv "%s" %s' % (oldXMP, newXMP)
                status,output = commands.getstatusoutput(cmd)
                if status:
                if options.backup and ext in ['jpg','JPG']:
                    backup = outString + '-b.' + ext.lower()
                    cmd =  'cp "%s" %s' % (newXMP, backup)
                    status,output = commands.getstatusoutput(cmd)

    print inString,outString,timeSource,''.join(outList),seconds
print "Number of photos:             ",len(filenames)
print "Total number of shots bumped: ",bumpedShots


No comments:

Post a Comment