This article describes my migration from Lightroom to open source alternatives, such as Darktable, using xmp sidecar files.
I will explain the required migration steps, my photography workflow as well as a bit about the technical background.
I used Adobe Lightroom for the past ten years. Although I am in no way a professional photographer, I fell in love with this tool. Lightroom 5 was one of the few programs, I actually bought as a student – and felt happy with.
Unfortunately, with the introduction of a subscription based payment model by Adobe, I started to worry if spending 120€ per year on Lightroom is reasonable for my use-case.
Although Lightroom Classic is an option, it is not really usable on a Linux ecosystem. It is actually the last tool which ties me to a Windows machine.
I am currently using Lightroom 5.7.1 in my hardware accelerated Windows 10 Virtual Machine (VM), which I have initially created for gaming purpose only (get enlightenment here). It is working, but not as smooth as it could be. As I had to create a connection between the Windows VM – and the Linux file system, in order to access the pictures from both systems. This makes it cumbersome to use.
Okay I leave – from Lightroom to open source
In order to find the new perfect match, to fill the empty raw-file-developing-shaped hole, in my software heart, I had to understand what Lightroom was doing for me.
I came up with this list:
Lightroom develops and processes (RAW-) image files – non-destructive ~when I make changes to an image, the original is still unchanged.
Lightroom acts as digital asset manager (DAM). ~it allows me to have a folder based photo library structure.
Lightroom reads and writes metadata. ~I can apply tags, color labels and image ratings during my development workflow.
All information (development steps, sorting, meta data etc.) are stored in the Lightroom catalog file.
Additionally requirement for the new software is that it should run on a Linux platform, and is preferably open-source.
During my search for alternatives, some programs keep poping up: , and
~Very long story short we will do the following:
Darktable will make most of the heavy lifting as it ticks all the boxes. Unfortunately, the DAM part of Darktable is not as advanced as I was used to by Lightroom. Thus, I will use digiKam as support for the digital asset management tasks (see upcoming article section).
The destination is set, what do we need?
I guess before we can continue a little technical background is required.
Metadata in general are additional information about files. These information can be directly embedded in the file.
When it comes to photography typical metadata information are:
- Capture date and time
- GPS coordinates
- Copyright information
- Color labels
- Picked / reject flags
These metadata can be used to organize your library, search for pictures or can support your workflow.
Extensible Metadata Platform (XMP)
Directly from Wikipedia:
The Extensible Metadata Platform (XMP) is an ISO standard, originally created by Adobe Systems Inc., for the creation, processing and interchange of standardized and custom metadata for digital documents and data sets.
XMP Sidecar files
Some file formats (e.g. jpeg, png, dng, tiff) directly have some xmp fields embedded in their file types. Others, especially proprietary RAW formats not (or Lightroom/Darktable is not able to write them correctly)
Per default Lightroom stores every image related processing step, every metadata change (e.g. color label, image flag, etc.) in its catalog file (in order to keep the non-destructive image processing promise).
- Version used in this guide: Adobe Photoshop Lightroom Classic (5.7.1)
- Sidecar files in Lightroom are optional
- The naming pattern of the xmp sidecar files is <filename>.xmp
- Sidecar files are created when metadata for an image is saved
- Lightroom allows only one color labels per image
- “Picked” and “Rejected” flags are supported
- Lightroom supports “Star ratings”
The thing is Darktable works a little different. The image information, processing steps etc. are stored in so called xmp sidecar files. Each image file has its own xmp file, which is stored right next to the image in the folder, and created when Darktable.
- Version used in this guide: Darktable (2.6.0)
- Sidecar files in Darktable are necessary.
- The naming pattern of the xmp sidecar files is <filename>.<original extension>.xmp
- Sidecar files are created during the image import process
- Darktable allows more than one color label per image
- “Picked” flag is not support
- “Rejected” flag is supported
- Darktable supports “Star ratings”
My workflow in Lightroom
The following part might not be necessary, but I think it can help to show where we are heading to – *drum roll*
My photo library structure
My library is structured by years, months and events
/photography/album/ / ... / 2012 - 01 - Berlin 2012 - 03 - Groningen 2012 - 07 - Copenhagen / ...
and so on.
Image formats and file handling
I shoot in RAW – but I keep only originals of images I have edited.
Finished edits are also exported to 100% JPEGs, so that I can share or present my edits quickly without the need to open Lightroom first.
Image Sorting and promising picture selection
I run through newly imported photos several times.
During the first run I try get rid of all the obvious broken shots (not in focus, over or under exposed etc.). I use the pick and reject flag for this.
During the second run I delete redundant pictures. Lets say I took 6 shots of the same motive with different settings. I try to keep maximum two images of the same motive – the others get dumped.
During the third run I mark the promising images using color labels:
- Red label: means I really like the picture (if someone says “show me your pictures”, I will show these)
- Yellow label: for stock images (for example … pictures of flowers)
- Green label: for nice portraits
- Blue label: for panoramas
I guess everyone will come up with its own categories. The important part is that these color labels will stay on the images indefinitely, as Lightroom can search and sort by color label. Tasks like “show me your favorite pictures from 2016” can be accomplished – just filter folder 2016 for all red labeled images. This is also very powerful in combination with Lightrooms “Smart Collection” feature.
During the development I will mainly focus on the color labeled images. Not color labeled images get cropped, aligned and/or occasionally auto tuned – but only if needed. The less work, the less time the better.
Post development and clean-up
For color labeled images (promising pictures) I will keep the original RAW file, as well as an 100% JPEG or 16-bit TIFF version of the edited image. All other, non marked images, will be transferred to 100% JPEGs, while the original RAW files will be deleted once development is finished.
The folder structure looks something like this in the end:
photography/album/ / 2017 - 02 - St. Peter Ording/ DSC00472.JPG DSC00476.ARW DSC00476.JPG DSC00482.JPG DSC00483.JPG
In the example “DSC00476.ARW” was a promising pictures.
Okay, let’s start migrating
So whats the game plan? Well, easy – get the information stored in the Lightroom catalog and transfer it to Darktable. Unfortunately, Darktable is not able to directly read Lightroom’s catalog file, but Lightroom is able to additional create sidecar files, and these can be read by Darktable.
Make sure the setting “Automatically write changes into XMP” is enabled. This creates xmp sidecar files when metadata is saved in Lightroom.
Open Lightroom settings via:
“Edit” ➡️ “Catalog Settings”
Make sure the color labels are have their default values.
This is important!
“Metadata” ➡️ “Color Label Set” ➡️ “Edit”… (or simply choose “Lightroom default” if available)
The following setting will allow Darktable to check for metadata/sidecar file changes during each startup – instead of just once during the image import process.
Open the Darktable global preferences by clicking on the gear symbol in the “lighttable” view.
Image 3: Darktable settings gear in lighttable view
Go to the “Core Options” register and enable the option “look for updated xmp files on startup.”
Metadata transfer overview
Which metadata one wants to transfer, is heavily workflow related. There are also settings one should not (want to) transfer between Lightroom and Darktable (see ❎️ marks in transfer table). As Darktable and Lightroom differ in the used “RAW processing engine”, settings such as, exposure values would lead to different results in the two applications.
The following table show the results of different types of metadata written in Lightroom, followed by an Darktable import.
= successfully imported
❎️ = successfully not imported
= not imported
|Flagged as “picked”|
|Applied image crop||❎️||❎️||❎️||❎️||❎️||❎️|
|Applied B&W preset||❎️||❎️||❎️||❎️||❎️||❎️|
|Added Author, Title, Caption, Copyright and sublocation||*||*||*||*||*||*|
*sublocation is not supported by Darktable
The migration from Lightroom to Darktable step-by-step
- Darktable is not running.
- Make sure no Darktable xmp files exist (i.e. files named <Filename>.<extension>.xmp) for the images you want to import.
- The Darktable library file has no record of the images you want to import.
This can be done either by right clicking a folder and choosing remove in the “collect images” section in the “lighttable” view while it is sorted by folder. Or one deletes the complete Darktable library file (default location ~/.config/darktable/darktable.lib). In case the image used to be in the library make sure to double-check step 1 and 2.
- Identify the metadata attributes you want to transfer, and make sure every image you want to transfer has these metadata attributes set in Lightroom.
- Now save the metadata into the xmp sidecar files:
In Lightrooms “Library” view, select all images you want to transfer to Darktable and press “CTRL + S” (or use “Metadata” ➡️ “Save metadata to image” in the menu bar.). This step will create/update xmp sidecar files.
- When everything is done – close Lightroom
- *optional step
In case you want to use a different root album folder for Darktable, now is the time to copy the folder to its new location.
- Open Darktable and choose “folder” from the import section in the “lighttable” view (see image 4, red marked area).
- Select desired the folder and press the “open” button. In case the folder contains sub-folder with images, make sure to check the option “import directories recursively” under “import options”.
- Verify the imported images result. All images known to Darktable can be found in the “collect images” section in the “lighttable” view (see image 4, blue marked area). In order to feel a bit more familiar with the structure, select “folders” from the drop-down menu in the section (see image 4, green marked).
To be continued …
- Introduction to Darktable (for Lightroom users)
- My new workflow with Darktable and digiKam
- Synchronization between Darktable and digiKam
- Use the comment sections for further requests