The use case

This isn’t exactly far-fetched but nothing seems to do it out of the box: migrate Glance images from a backend to the other.

We’re in the process of moving our Glance default store to Swift, it simply scales infinitely better than the file backend. Now, changing the default store does just that: change the default store. It makes it so the newly created images will be uploaded to the new store - it doesn’t do anything for the existing images.

It’s fine if you plan on keeping the old storage backend around but if you have to keep the images and want to get rid of the old backend, Glance doesn’t help you magically migrate images to the new shiny store.

Let’s look at how we can migrate images from file to Swift while keeping their attributes and properties, things like:

• UUID (as users might be relying on this, whether we like it or not)
• Name
• Container format
• Disk format
• Visibility - public or private
• Other params like min_disk, min_ram
• Custom properties
• etc.

The attempts

Delete and re-create the images with the same properties

My good friend Emilien Macchi (follow him on twitter @EmilienMacchi) got my hopes up with what seemed a simple but effective script to do almost just that.

It was flawed on a very important point, though - it is not possible to delete an image and create a new one with the same UUID. The script also doesn’t keep properties but some bash-fu could have fixed that if the UUID wasn’t problematic.

Essentially, the delete is a “soft” delete. The image remains in the database but at the status “deleted”.

Trick Glance into uploading the image directly to Swift

Looking at the Glance database, you can see how it stores the images’ locations for a file based image:

mysql> select * from image_locations \G
*************************** 1. row ***************************
        id: 1
  image_id: d872b3f2-45c7-41f4-b5f3-9be82df71780
     value: file:///var/lib/glance/images/d872b3f2-45c7-41f4-b5f3-9be82df71780
created_at: 2015-07-16 19:15:19
updated_at: 2015-07-16 19:15:49
deleted_at: NULL
   deleted: 0
 meta_data: {}
    status: active

[...]

In contrast, the location of the same image stored in Swift would look like this instead:

# In order from left to right:
# Tenant: services, User: glance, Password: password
# Keystone auth host: keystone.tld
# container: glance, image_uuid: d872b3f2-45c7-41f4-b5f3-9be82df71780
swift+http://services%3Aglance:password@keystone.tld:5000/v2.0/glance/d872b3f2-45c7-41f4-b5f3-9be82df71780

The location of the image is pretty predictable since everything is according to the configuration of your Swift store.

I tried to download the image and upload it directly to the Glance Swift container using Swiftclient and then update the image location in the database from the file-based one to the Swift one.

It turns out it’s not easily possible: Glance chunks larger images by itself prior to uploading the image with it’s own way of doing things. In Swift, an image created by Glance will look like this:

swift list glance
  d872b3f2-45c7-41f4-b5f3-9be82df71780
  d872b3f2-45c7-41f4-b5f3-9be82df71780-00001
  d872b3f2-45c7-41f4-b5f3-9be82df71780-00002
  d872b3f2-45c7-41f4-b5f3-9be82df71780-00003
  ...

I found this kind of odd since Swiftclient has a built-in mechanism to segment a large file into smaller chunks. You’d get something like this instead:

# Segment image into 32MB chunks
swift upload glance d872b3f2-45c7-41f4-b5f3-9be82df71780 --segment-size 32000000
swift list
  glance
  glance_segments
swift list glance
  d872b3f2-45c7-41f4-b5f3-9be82df71780
swift list glance_segments
  d872b3f2-45c7-41f4-b5f3-9be82df71780/1437240501.525209/268435456/32000000/00000000
  d872b3f2-45c7-41f4-b5f3-9be82df71780/1437240501.525209/268435456/32000000/00000001
  d872b3f2-45c7-41f4-b5f3-9be82df71780/1437240501.525209/268435456/32000000/00000002
  d872b3f2-45c7-41f4-b5f3-9be82df71780/1437240501.525209/268435456/32000000/00000003
  ...

The image in the glance container is actually a manifest file which references the chunks. If you download the manifest file, Swift’ll in fact serve you the chunks as one big file.

Anyway, back to the drawing board as this didn’t work out either.

The solution

What I ended up doing is a fairly generic script that could potentially be used to migrate from any backend to any backend. It’s rough around the edges but suits my needs right now. It also does everything through the API and there’s no manual SQL work involved.

What the script does:

• Authenticates a session through Openstackclient. The usual environment variables are fine.
• Lists all current images
• Downloads each image and create a new one (in the new backend) with the same properties (except UUID)
• Deletes the original image at the orginal location (glance location-delete) – This effectively deletes the image at the original location
• Updates the location of the original image to the location of the new image
• Protects both images to prevent them from being deleted and orphaned
• Deletes the temporary copy of the downloaded image

The result is that all your original images are created as new images on the new backend and the original images’ location point to the new image. You end up with two images but one location (file).

The repository for the script is available here: https://github.com/dmsimard/migrate-glance-backend

I definitely want to improve the script further when I have time, contributions are also welcome !