Tuesday, January 25, 2011

Photo Workflow, Part 1

I often get questions like "Is that Photoshopped?" when people see some of my photos. That's a loaded question in my view because the general perception is that somehow Photoshop is used for altering "reality." It certainly can be, but that's not generally the way it's used by photographers. So I thought I would lay out, in pieces, what I actually do with my photos once I take them.

First, I shoot everything in RAW mode, both on my Canon DSLRs and with my Canon G10 that I use for underwater.

The first thing I do is quickly import my photos into Adobe Lightroom. I use a setting under "Catalog Settings" to "Always write changes into XMP." Many people don't recommend this, but it's crucial to how I work:
  • I'm not relying on Lightroom's database. If it gets corrupted all the important metadata is in a little .xmp file that rides along with the image files
  • I can use other programs to manipulate the xmp file's metadata and import the changes back into Lightroom
  • I use Lightroom on two computers and this is an easy way, along with Unison, to keep things in sync between the computers
After I have imported my photos, I now have an XMP along with each RAW file. Then I quit Lightroom.

Recently I've gotten into geotagging my photos (adding information about where they were taken), so that's the next step in my process. If I remember, I track where I am with my iPhone. (When I'm scuba diving, the iPhone tracks the boat, and not me, of course.) The best application I've found for this kind of tracking is EveryTrail, but there are a number of apps for the iPhone and other devices. The important thing is that they output a GPX file that can be used in the next step.

Next, I use a program called Geotag that will load up all the XMP files and the GPX file(s) and correlate them based on the time of when the photo was taken and where I was. Again, there are a number of programs that do this, but Geotag is the only one I found that runs on any computer (it's Java), is free, and works with XMP files (most like to use JPGs instead).

If I don't have a GPX file describing exactly where I was at any time, I use Google Maps to take a guess and fill in requisite fields in Geotag.

The last step, before I'm ready to begin filtering and working with my images, is to rename them all. Every camera maker seems to have a slightly different naming scheme and things like IMG_1234.JPG have a couple of problems. First, they can overlap if you take more than 10,000 photos (which I have). Second, I often use more than one camera on any given trip or gallery, so one camera may be at 1234 and the other at 4321. That means that alphabetically and chronologically things are in different orders. So I rename all my files. I use the script below, which renames things like 4c64de23. It's sort of gibberish, but each second of the day since 1970 has it's own code, so unless more than one photo was taken in the same second, it's guaranteed to be unique (it actually deals with that contingency too). The script renames all the RAW, XMP and JPG files, etc. that share a common root name. It's got options to work only with JPG files (like some of my galleries), to keep a backup JPG, and to just explain what it will do but not actually do anything.

Once all the files are renamed, the last step in this part of the workflow is to go back into Lightroom and choose "Synchronize Folder" for the folder containing all the photos. This is the same step I take if I change something on one computer and want to work on it on another. This reads all the files in fresh, taking all the metadata and development settings from the XMP files.

In the next post I'll start to answer the question I posed at the beginning: "Do I Photoshop my photos?" (At this point I haven't actually done anything to the images themselves, just done a bit of organization.)

Update: With the release of Lightroom 4, which includes geotagging, the first part of my workflow can be simplified a lot. Basically I can rename the photos as soon as I put them on my computer, import them into Lightroom, and do the same sort of geotagging with the GPX file directly in Lightroom's Map module. Or just drag and drop the photos onto the map if I don't have a GPX file.

Friday, January 21, 2011

Further thoughts on the TV Live Plus

After a couple of weeks of playing with my WD TV Live Plus media player, I've got more experience with it. It's definitely a neat little device that I'll be keeping.

In my first post, my major concern was the functionality of the music interface. I had thought that I would have to do some hacking on my Samba server to mimic playlists of all my music and artists, etc. But it turns out that directly attached hard drives or network shares (Samba or Windows network sharing) are not the only options. The other possibility is a UPnP server which is specifically set up for media sharing. Basically, the server knows all the meta-data like the artist and genre of a song, etc. and presents that to the player.

After an upgrade to my server from Ubuntu 6.06 to 10.10, I was able to install Mediatomb, a UPnP server for linux. After turning on the UI and turning off data storage caching (avoided database errors for me) in the config file, I had a very functional set up. Now the music interface is much better. I get a browseable menu with "All music", "Artists", "Genre" etc that I can get a much better view of my music.

The WD TV Live is not without it's issues though. First, an inexcusable in my view, is the lack of gapless playback. Every modern music player does this and that it does not is puzzling. Basically what this means is that even when playing two tracks from the same album, there is a slight (fraction of a second) gap between one song ending and the next starting. This is really irritating on live albums or "album rock" like Pink Floyd's "The Wall" where one track runs right into the next. Worse is that not only is there silence, the digital output is momentarily turned off, so sometimes there is an additional delay (and missing sound) while my digital receiver re-syncs to the output. They've known about this problem for a long time and claim to be working on a fix. We will see.

The second annoyance, which I have solved, is that the photo browser (also using UPnP) was showing most of my photos as very small tumbnails instead of filling the screen. Strangely my portrait photos were filling the screen. The solution, as it turned out, was to resize all the photos. As long as the photos are not too wide, they show correctly. I resized everything to 1080 pixels tall (HD resolution) and it's fine now. This was no problem for me since I already have a reduced size group of photos that I keep on my laptop rather than carting around the full sized collection.

The third thing, also purportedly being worked on, is that the Netflix streaming doesn't output the nice Dolby Digital 5.1 audio and only supports standard stereo. Supposedly a fix is coming for this later this year.

Saturday, January 15, 2011

Downloading all of South By Southwest

One of my annual traditions is to download all the preview music from the South by Southwest (SXSW) conference in Austin, TX. Then I can listen to it at my leisure and find new artists I like. Every year, they change the website slightly and make it a bit harder to do. In fact, last year they did this shortly after I downloaded everything, so maybe they are on to me.

Music for SXSW 2011 is already appearing on their site (the festival is in March). Over the next few months I'll update my collection and figure out what I like and don't like. So here's the quick and dirty script I worked out to grab everything. This should work on any unix-like OS with python and wget.

Friday, January 14, 2011

Amarok 2 or how to break a program

In an earlier post I made a quick reference to Amarok 2. This program is a case study in what not to do. Amarok 1 was truly an amazing piece of software the people raved about. It was the best music player/manager out there and people were actually switching to Linux to use it. It was that good. To draw an analogy to another of my hobbies, it was the Lightroom of music players.

But apparently it was hard to maintain under the hood, so the developers decided to scrap it and rewrite something they called Amarok 2. But when it was released it still had a ton of problems. A quick web search for Amarok sucks will bring up a lot of this history.

Basically the authors seemed to decide that it was very important that your music player talk to a bunch of online services like telling people on Facebook, as if they cared, what you were listening to right that minute. What was less important apparently was actually playing music and especially being able to organize your collection.

This was all 3-4 years ago. In the meantime, Amarok 2 is getting better, but it's lost its buzz and no one today ever raves about how good it is. Every time I use it to try to accomplish something, I get frustrated very quickly. Occasionally I will evaluate it to replace the original Amarok on my media PC and I quickly find something it won't do. For instance, I have a fair bit of music, so I'd like a playlist that would play a rotating collection, but not play things I've rated poorly and not play classical music. I'd like another playlist that would play things tagged "night" so when I'm trying to fall asleep, I'm not jarred awake by a Metallica song. Piece of cake with Amarok 1.4. Amarok 2 couldn't do it. It looks like finally in 2011 it may be able to do this. Maybe.

Which brings me to today's frustration. I'm trying to organize some of my ripped CDs. I noticed that a title that is set incorrectly, so I changed it in Amarok 2. Immediately, for some reason, this CD goes from being listed under various artists to being listed under 15 different people. I click on it and find Amarok thinks it's a compilation because there is a menu item that says "Don't show under various artists." So if I click on that, what would you think would happen?

  1. Nothing
  2. Apparently nothing but now "Show under various artists" is back and clicking that gets things back the way they were
  3. It shows immediately under various artists
  4. The album totally disappears, it's album cover is assigned to another album, AND a scan of my collection won't bring it back
If you answered #4, you would of course be right. This kind of thing is why I was still running a three year old operating system on my Media PC. And it's why I doubt I will ever like or really use this new version of Amarok.

Sunday, January 9, 2011

Western Digital TV Live Plus, first impressions

I recently picked up a $100 device that I understand is being called a "Digital Media Receiver". I'm not sure that's a great name since it's very different than a home theater "receiver," but it does receive and play digital media, so I guess it's not too bad. There are lots of reviews of these types of devices on the web, but the basic features are:
  • Plays music, video, and photos
  • May be able to access data on a locally attached hard drive or a network share
  • Access to a number of internet services through plugins
    • Most noticeably this device connects to Netflix, Pandora, Blockbuster, and Facebook
My aim is to replace a Linux based media PC that I've had in one form or another for 3-4 years. At one point I had a really nice setup on the Linux box with remote control and an LCD display of what was playing. After a catastrophic failure of the power supply, I eventually rebuilt the hardware, but stopped messing with the software when I reconstructed the ability to do HD output and digital audio. Getting the rest working again was just too much work.

So anyhow, seeing little boxes like this made me think maybe I could get rid of the big PC with something roughly 1% of the volume. I read a few reviews and the consensus was that the WD box was pretty good, but not as good as a similar device from ViewSonic. The Boxee Box looks pretty cool too and I played with the software on my Mac. But unfortunately both these devices require an HDMI output and I have an old TV without only component inputs. An Apple TV is more limited, requiring me to run iTunes somewhere else in the house. So WD it was for me.

So here are my initial thoughts:

First, the music interface is really not that great. I was hoping for something as good as iTunes on a real computer. It's not even close. And iTunes isn't as good for me as my beloved Amarok 1.4 on a linux box. (Amarok 2 was junk when it came out and version 2.3 is getting to the point I might consider it).

Second, while it can access data by a network share (I use a samba server), it's less functional than a directly attached hard drive (which I've only been able to test with a USB thumb drive). Basically the interface on a network share is just slightly glorified file interface so it really has no knowledge of my extensive music collection until I drill down into the folders. I think I can organize my way out of this though.

Third, ocassionally accessing my network share doesn't work and you've got to try again.

Fourth, Pandora, Netflix, and Mediafly (a podcast aggregator as best I can tell) access is pretty cool. Netflix doesn't deliver 5.1 sound yet, from what I understand. But an upgrade to a new API for their service is in the works.

The interface to get to all of this is pretty nice though and uses a very simple remote control. The device also works well with a USB keyboard.

So, I'm going to keep it and play around with ways of working around it's issues. It doesn't do the one thing I used to use my media PC for, playing music, as well or flexibly. But it does a lot more. And at $100, it's the kind of thing I replace every couple of years as I need to.

I just wish I'd found this before I spent all the time, effort, and money to repair my media PC after the blow-up.

Saturday, January 8, 2011


This is my core dump.

I intend this blog to be a collection of seemingly random information on how I use technology to solve the problems I'm interested in. My hope is that someone who's interested in the same problems I am may find it useful too. Here's some of my relevant interests:
  • Photography, especially post-processing
  • Home theater and digital media management
  • Geotagging and GPS tracking
  • iPhone software and it's cousins
  • SCUBA diving
So, that's a pretty diverse collection, I reckon. To organize all the data I've collected over the years related to these activities, I've written a lot of simple little scripts. I'll post those along the way too.

What you're not going to find is my musings on life, the universe, or anything. I've got other outlets for that.