15
Apr/12
5

Migrating F-Spot to Darktable

F-Spot was my favorite photo management program for the past years. In connection with GIMP as photo editor it was a good team. Again and again I came accross Windows users using Lightroom which seemed to make everything so easy. After some other photo editing session I reached the point where the limit was reached. I had to change something! But what? Switch to Windows? Nogo! After searching for alternative solutions I cam accross darktable. This piece of software seemed to be exactly what I was searching for. After some test it was clear: I need to move from F-Spot to Darktable.

The main problem: There is no import function for migrating the photo library from F-Spot to Darktable. At the moment I have like 20k photos in my database and all are well tagged. I would never throw that work away! After some searching I realized that there is no solution out there. I had to create a solution for myself.

I knew that F-Spot stores it’s information about images in an SQLite database. It should be no problem to extract those information. After a short chat at the #darktable channel in freenode IRC network I knew that darktable reads the *.xmp files during import of the images. Seems to be a straight forward solution: Write a script which reads the photos and tags from the F-Spot SQLite database and create an .xmp file for each image. Then import the images to Darktable. And be happy.

The plan was a good one. It worked like a charm. Don’t know if some others might trap into this situation but I decided to release the script. Here it is:

#!/usr/bin/python
# encoding: utf-8
#
# Copyright (C) 2012 Lars Michelsen <lm@larsmichelsen.com>,
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307,
#
# GNU General Public License: http://www.gnu.org/licenses/gpl-2.0.txt
#
# Report bugs to: lm@larsmichelsen.com
 
import os, sys, urllib2
 
fspot_db = '%s/.config/f-spot/photos.db' % os.getenv('HOME')
 
opt_verbose = '-v' in sys.argv or '--verbose' in sys.argv
opt_help    = '-h' in sys.argv or '--help' in sys.argv
 
def help():
    sys.stderr.write('''
 fspot2darktable
 
 This script helps migrating your photo library from F-Spot to Darktable.
 It extracts assigned tags from the F-Spot SQLite database and creates
 .xmp files for each image with the tag information. The script cares
 about and preserves the tag hierarchy structure.
 
 The script must be executed as user which F-Spot DB should be migrated.
 
 The script will query the SQLite database of F-Spot and extract all tags
 and hierarchical tag information from the database for images which do
 exist on your harddrive and do not have an associated xmp file yet. It
 will write those information in a basic xmp file.
 
 This script has been developed to be executed only once for a photo
 library. The cleanest way is to execute it before you import your images
 into Darktable. This way you can ensure all the new tags are really loaded
 correctly into Darktable.
 
 During development I removed the images again and again from the Darktable
 database and also removed all *.xmp files in my photo folders to have a
 clean start. After cleaning up those things I ran fspot2darktable, then
 started Darktable and imported all the fotos.
 
 After executing the script you can import single images or the whole
 photo library to Darktable. You should see your tag definitions in
 Darktable now.
 
 The script has been developed with
   - F-Spot 0.8.2
   - Darktable 1.0
 
 Please report bugs to <lm@larsmichelsen.com>.
''')
 
def err(s):
    sys.stderr.write('%s\n' % s)
    sys.exit(1)
 
def log(s):
    sys.stdout.write('%s\n' % s)
 
def verbose(s):
    if opt_verbose:
        sys.stdout.write('%s\n' % s)
 
if opt_help:
    help()
    sys.exit(3)
 
try:
    import sqlite3
except Exception, e:
    err('Unable to import sqlite3 module (%s)' % e)
 
if not os.path.exists(fspot_db):
    err('The F-Spot database at %s does not exist.' % fspot_db)
 
conn = sqlite3.connect(fspot_db)
cur  = conn.cursor()
 
# Loop all images, get all tags for each image
cur.execute('SELECT id, base_uri, filename FROM photos')
num_files = 0
num_not_existing = 0
num_xmp_existing = 0
num_created = 0
for id, base_uri, filename in cur:
    num_files += 1
    # F-Spot URLs are url encoded. Decode them here. There seem to be
    # some encoding mixups possible. Damn. Try simple utf-8 then latin-1
    # vs utf-8. This works for me but might not for others... let me know
    # if you got a better way solving this
    path = urllib2.unquote(base_uri.replace('file://', ''))
    try:
        path = path.decode('utf8')
    except UnicodeEncodeError:
        path = path.encode('latin-1').decode('utf-8')
    path += '/' + filename
    xmp_path = path + '.xmp'
 
    if not os.path.exists(path):
        verbose('Skipping non existant image (%s)' % path)
        num_not_existing += 1
        continue
 
    if os.path.exists(xmp_path):
        verbose('Skipping because of existing XMP file (%s)' % path)
        num_xmp_existing += 1
        continue
 
    # Walks the tag categories upwards to find all the parent tags to
    # build a list of parent tags. This will be used later to build
    # hierarchical tags in the XMP files instead simple tags
    def parent_tags(category_id):
        cur = conn.cursor()
        cur.execute(
            'SELECT id, name, is_category, category_id '
            'FROM tags WHERE id=\'%d\'' % category_id
        )
        tag_id, tag, is_category, category_id = cur.fetchone()
 
        parent_tags_list = []
        if category_id != 0:
            parent_tags_list += parent_tags(category_id)
 
        return parent_tags_list + [ tag ]
 
    hierarchical_tags = []
    simple_tags       = []
 
    cur2 = conn.cursor()
    cur2.execute(
        'SELECT tag_id, name, is_category, category_id '
        'FROM tags, photo_tags WHERE photo_id=\'%d\' AND id=tag_id' % id)
    for tag_id, tag, is_category, category_id in cur2:
        if category_id:
            hierarchical_tags.append('|'.join(parent_tags(category_id) + [ tag ]))
        else:
            simple_tags.append(tag)
 
    def xml_fmt(tags):
        return ''.join([ '      <rdf:li>%s</rdf:li>' % \
                        t.encode('utf-8') for t in tags ])
 
    # Now really create the xmp file
    file(xmp_path, 'w').write('''<?xpacket begin="<feff>" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:xmp="http://ns.adobe.com/xap/1.0/"
    xmlns:darktable="http://darktable.sf.net/"
    xmlns:lr="http://ns.adobe.com/lightroom/1.0/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
   xmp:Rating="1"
   darktable:xmp_version="1"
   darktable:raw_params="0">
   <darktable:colorlabels>
    <rdf:Seq/>
   </darktable:colorlabels>
   <darktable:history_modversion>
    <rdf:Bag/>
   </darktable:history_modversion>
   <darktable:history_enabled>
    <rdf:Bag/>
   </darktable:history_enabled>
   <darktable:history_operation>
    <rdf:Bag/>
   </darktable:history_operation>
   <darktable:history_params>
    <rdf:Bag/>
   </darktable:history_params>
   <darktable:blendop_params>
    <rdf:Bag/>
   </darktable:blendop_params>
   <lr:hierarchicalSubject>
    <rdf:Seq>
%s
    </rdf:Seq>
   </lr:hierarchicalSubject>
   <dc:subject>
    <rdf:Seq>
%s
    </rdf:Seq>
   </dc:subject>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>''' % (xml_fmt(hierarchical_tags),
                          xml_fmt(simple_tags)))
 
    num_created += 1
 
log('FINISHED! - Summary:')
log(' %-10s images in total' % num_files)
log(' %-10s images do not exist' % num_not_existing)
log(' %-10s images already have xmp files' % num_xmp_existing)
log(' %-10s created xmp files' % num_created)

Simply copy the script to your system, name it for example fspot2darktable, make it executable (chmod +x fspot2darktable) and execute it with ./fspot2darktable -h. The help output should give you enough information about how the script is working and what it is doing. To let the script do it’s work execute it without parameters like this: ./fspot2darktable.

Filed under: Open Source
3
Nov/11
6

Photo autocopy from SD card

Copying photos from the SD card of my camera is a very common task. And always the same pattern. Take the card out of the camera, put it into the sd card reader, wait for the automount popup, browse to the picture directory, select all images, cut all images, browse to my photo directory, create a new folder, name the folder, move to the folder, paste the pictures into it, wait for the finished job.

Sound boring? Sure, it is!

Now, how to make this one a little smarter? Maybe scripting some of those steps is a cool idea. So how would this work in a perfect world?

19
Oct/11
4

Fixing “show desktop” shortcut (hotkey) in Ubuntu 11.10.

After updating my ubuntu 10.04 installation I had to recognize that the shortcut for minimizing all windows to show the desktop has changed (It seem to be + + d after update). I don’t like that combination + d would be way better. So how to fix this?

Solving this is a real quickie:

Launch gconf-editor and browse to /apps/metacity/global_keybindings

Change the value of show_desktop from d to d.

Press enter, close gconf-editor and enjoy the results of pressing d ;-).

Filed under: Open Source
16
Oct/11
2

Ubuntu 11.10: A lot of problems …

I decided to update my notebook from Ubuntu 11.04 to the newest release Ubuntu 11.10. I did not expect any big problems since I had no real problem with the last updates. But it was different this time. The update went smooth, some questions about changed config files but no real problems.

One positive point: The login screen looks nice.

After the first login:

WTF, Unity desktop manager only? grml! I don’t like the unity stuff. I just want to have my console, browser, mail program open and switch between them fast. I don’t need that blingblinghoverfadesmoothscrollpop stuff. I tried the gnome fallback mode but it seems like a real fallback mode which appears like a stripped of version of the old gnome desktop. Also not very comfortable. So I’ll try unity now for some days but will switch to another window manager, maybe xfce.

The second and more annoying thing: When connecting to my home wlan the router crashes and refuses connections on WLAN/LAN. Yay! The new denial of service tool: Ubuntu 11.10. I have a buffalo router with OpenWRT firmware installed. The router is capable of broadband wifi (N). This seems to be a part of the problem. After disabling the N features in the intel driver of my lenovo w500 (03:00.0 Network controller: Intel Corporation Ultimate N WiFi Link 5300) it works as expected. For details take a look here.

Why are the folder (sort, threading, …) settings of my thunderbird gone? I have no clue.

That is the score after three hours using the new version …

Filed under: Open Source
17
May/11
0

Simply amazing: Running Linux in JavaScript

I am really, really, really impressed: JSLinux

2
Aug/10
21

German Umlauts on US Keyboard in X (Ubuntu 10.04)

The US keyboard layout is better for programmers since many of the needed special chars like brackets, semicolons and so on are easier to type than, for example, with the German keyboard layout which I used so far. Since I am programming a lot on my notebook I decided to buy a new keyboard with the US layout to make it easier to learn using the US layout.

Besides programming I often write German documentations, mails and other kinds of texts. When writing things in German the umlaut chars like ä, ö and so on are important. So I would miss these chars a lot when using the US keyboard layout. In fact there are some key combinations in linux and/or X11 to type those chars but they way too complicated to be used while writing a longer text.

After some research I found a solution by using the user specific Xmodmap file. See how I modified that configuration file to fit my needs…

21
Jun/10
0

Firefox offline mode vs. NetworkManager (Ubuntu 10.04)

Sometimes I need to set manual network configuration on my Ubuntu driven laptop. For this cases I ignore the network configuration of the NetworkManager. But I don’t want to disable the network manager at all. So I wait for the NetworkManager to finish it’s tasks for setting the IP configuration manually afterwards.

One annoying thing in this case is that the NetworkManager puts the Firefox in offline mode. I searched my system to turn off this functionality in NetworkManager. But since I did not found the configuration option I searched the Firefox configuration. And I found an option to turn off the NetworkManager influence to the online/offline state of Firefox.

To disable that behavior enter the configuration of Firefox browsing to about:config. Then search for the option

toolkit.networkmanager.disable

in using the filter field and set the value to true.

After restarting Firefox the option should be used and the NetworkManager will never put the Firefox in offline mode again.

15
Feb/10
3

Disable Trash/Wastebaket in Ubuntu Karmic

By default Ubuntu Karmic or more detailed the file manager Nautilus moves files which are deleted to a trash or wastebin like it is called in Ubuntu. Using external HDDs which are plugged in to other systems with different operating systems is kind of annoying cause the “deleted” files are moved to a folder named .Trash-. This was the reason for me to check out how to disable the wastebaket in Ubuntu Karmic for at least external HDDs.

23
Dec/09
1

Git merging with removal of one commit

Code Git grew up to my favorite version control system, I use Git on a daily base now. I like git because it is powerful but also simple for the daily usage. Git is a complex system so there are commands and steps which I do not perform every day. So I write about some more complex steps for later reference.

Today I needed to merge two git branches but remove the changes from only one commit where some files were deleted which should be left untouched in the target branch. I realized that using a temporary local merge branch where I removed the unwanted commit to later merge it with the target branch.

27
Nov/09
1

tar: Extract specific files from a large tarball

tux Unpacking single files from big tar archives which have been compressed with gzip (Often named .tar.gz or tgz) is annoying. For example you got a backup of a big filesystem packed in a tarball and just need to restore a single file which size is just a couple of kilobytes. Extracting the whole archive often takes a lot of time. This is time you can really save. Digging a bit into the man pages you will see it is possible to get single files from large tarballs easily.

You will see it is possible to extract single files based on the path, complete directories and even on wildcard based selections. I give you a brief overview about the ways I extract single files from large tar archives using GNU tar on linux / based systems.