OpenStack: VM Migration on OpenStack

Posted on Posted in OpenStack, Teknocratica 2 Comments Tagged with ,

Hello All! Today’s post is on some work I did for exporting VMs between OpenStacks. Here are two scripts for doing the export and import; one for migrating between hypervisors (same OpenStack) and one for migrating between OpenStack implementations. They stop short of launching the VM but that could be done with a bit more work to identify flavors, map networks, and so on. Enjoy!

For the impatient: Here are the scripts:

Migrate VM between Hypervisors

Moving a VM between two hypervisors is very simple; you simply stop the VM, issue the nova migrate command, and then authorize the movement. In fact, if you have proper shared storage between your hypervisors you can use live migration via the nova live-migration command. I don’t have proper high-speed shared storage, so I have to do things the slower way. Let’s see the steps:

  1. Shutoff the machine using nova stop [vm].
  2. Migrate the VM using nova migrate [vm].
  3. Wait for the VM state to change to VERIFY_RESIZE; you can use nova show [vm] to get this state.
  4. Confirm the “resize” (really, the migration!) using the nova resize-confirm [vm].

The script above automates this process.

Migrate VM between OpenStacks

Moving a VM between OpenStacks requires a bit more work. The attached script automates the entire process except for launching (booting) the migrated VM.

  1. You need to have your “keystone” script files that provide credentials to both OpenStacks. You supply these files to the script.
  2. Get the VM ID to move by issuing nova show [VM] and noting the ID.
  3. Invoke the script by passing the following elements:
    • src_keystone_rc – file with OS_USERNAME, OS_AUTH_URL, and so on to the source OpenStack
    • dst_keystone_rc – file with OS_USERNAME, OS_AUTH_URL, and so on to the destination OpenStack
    • VM_ID – UUID for the VM to migrate (get this using nova show on the source OpenStack)
    • img_type – Glance image type for the created snapshot (“file”, “rbd”, or “swift”)
    • step – Nifty feature that allows you to indicate the starting “step” number; this permits you to rerun an interrupted script without waiting for all the intermediate steps to run over. Just pass in a 0 (zero) to run everything from the beginning.
  4. The script begins by verifying the VM ID on the source OpenStack: 
    # get info
l_rc=0
source "$l_in_src_keystone_rc"
echo "$$: nova show \"$l_in_vm_id\" 2>/dev/null > $l_tmp" >> $l_log
nova show "$l_in_vm_id" 2>/dev/null > $l_tmp
l_rc=$?
if [ $l_rc -ne 0 ]; then
  do_exit 3 "Invalid VM ID '$l_in_vm_id'"
fi

# get initial state of VM
l_vm_id=$(cat $l_tmp | grep -e "^| id" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
l_vm_name=$(cat $l_tmp | grep -e "^| name" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
l_vm_tenant_id=$(cat $l_tmp | grep -e "^| tenant_id" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
l_vm_state=$(cat $l_tmp | grep -e "^| OS-EXT-STS:vm_state" | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
l_vm_status=`cat "$l_tmp" | grep status | awk '{print $4}'`
l_vm_tenant=$(keystone tenant-get $l_vm_tenant_id | grep -e '  name  ' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
echo "$$: l_vm_id='$l_vm_id'" >> $l_log
echo "$$: l_vm_name='$l_vm_name'" >> $l_log
echo "$$: l_vm_tenant_id='$l_vm_tenant_id'" >> $l_log
echo "$$: l_vm_state='$l_vm_state'" >> $l_log
echo "$$: l_vm_status='$l_vm_status'" >> $l_log
echo "$$: l_vm_tenant='$l_vm_tenant'" >> $l_log
  5. Next, the script verifies the VM is stopped, and checks to see if a snapshot already exists. If you pass in a step of 1 (one), then if a snapshot already exists the script uses it. Otherwise it creates a new snapshot and gets the snapshot image information (type, container format): 
    # delete any existing snapshot
l_snap_name="$l_vm_name-snap"
l_snap_id=""
l_continue=1
while [ $l_continue -eq 1 ]; do
  # check for an existing snapshot ID
  echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_snap_name\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
  l_snap_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_snap_name" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
  if [[ "$l_snap_id" != "" ]]; then
    if [ "$l_in_step" -ge 1 ]; then
      echo "$$: $(date +"%Y%d%m-%H%M"): Step 0: Reuse old snapshot '$l_snap_id'" | tee -a $l_log
      l_continue=0
    else
      echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 0: Remove old snapshot '$l_snap_id'..."
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-delete $l_snap_id 2>&1" >> $l_log
      l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-delete $l_snap_id 2>&1)
      l_rc=$?
      echo "$$: l_msg='$l_msg'" >> $l_log
      if [ $l_rc -eq 0 ]; then echo "OK"; else do_exit 5 "Snapshot cleanup: $l_msg"; fi
      l_snap_id=""
    fi
  else
    l_continue=0
  fi
done

# create snapshot unless already exists
l_snap_container_format=""
l_snap_disk_format=""
if [ "$l_snap_id" = "" ]; then
  echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 0: Create new snapshot..."
  echo "$$: nova --os-tenant-name=\"$l_vm_tenant\" image-create --poll $l_vm_id \"$l_snap_name\" 2>&1" >> $l_log
  l_msg=$(nova --os-tenant-name="$l_vm_tenant" image-create --poll $l_vm_id "$l_snap_name" 2>&1)
  l_rc=$?
  echo "$$: l_msg='$l_msg'" >> $l_log
  if [ $l_rc -ne 0 ]; then do_exit 5 "Snapshot creation: $l_msg"; fi

  # get the snapshot ID
  echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_snap_name\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
  l_snap_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_snap_name" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
  l_rc=$?
  echo "$$: l_snap_id='$l_snap_id'" >> $l_log
  if [ $l_rc -eq 0 ]; then echo "OK ($l_snap_id)"; else do_exit 5 "Snapshot identification"; fi
fi

# get snapshot information
echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 1: Get snapshot info..."
echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-show $l_snap_id | grep -e 'container_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
l_snap_container_format=$(glance --os-tenant-name="$l_vm_tenant" image-show $l_snap_id | grep -e 'container_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-show $l_snap_id | grep -e 'disk_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
l_snap_disk_format=$(glance --os-tenant-name="$l_vm_tenant" image-show $l_snap_id | grep -e 'disk_format' | cut -d'|' -f 3 | sed -e 's#^ \+##; s#[ \t]*$##')
echo "container_format=$l_snap_container_format, disk_format=$l_snap_disk_format" | tee -a $l_log
  6. The script next pulls down the created snapshot image; make sure you have sufficient space on your local machine! 
    # pull down if not exists
if [ ! -f "$l_local_fname" ]; then
  echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 1: Pull snapshot '$l_snap_id' to '$l_local_fname'..."
  echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-download --file \"$l_local_fname\" $l_snap_id 2>&1" >> $l_log
  l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-download --file "$l_local_fname" $l_snap_id 2>&1)
  l_rc=$?
  echo "$$: l_msg='$l_msg'" >> $l_log
  if [ $l_rc -eq 0 ]; then echo "OK"; else do_exit 5 "Snapshot pull: $l_msg"; fi
fi
  7. The script sources in the destination keystone RC file, and uploads the snapshot: 
    # check to see if the image is already uploaded to destination
l_dst_image_name="$l_local_fname"
l_dst_image_id=""
l_continue=1
while [ $l_continue -eq 1 ]; do
  # check for an existing snapshot ID
  echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_dst_image_name\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
  l_dst_image_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_dst_image_name" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
  if [[ "$l_dst_image_id" != "" ]]; then
    if [ "$l_in_step" -ge 3 ]; then
      echo "$$: $(date +"%Y%d%m-%H%M"): Step 2: Reuse old destination image '$l_dst_image_id'" | tee -a $l_log
      l_continue=0
    else
      echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 2: Remove old destination image '$l_dst_image_id'..."
      echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-delete $l_dst_image_id 2>&1" >> $l_log
      l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-delete $l_dst_image_id 2>&1)
      l_rc=$?
      if [ $l_rc -eq 0 ]; then echo "OK"; else do_exit 5 "Dst image cleanup: $l_msg"; fi
      echo "$$: l_msg='$l_msg'" >> $l_log
      l_dst_image_id=""
    fi
  else
    echo "$$: l_dst_image_id='$l_dst_image_id'" >> $l_log
    l_continue=0
  fi
done

# upload the image to destination if necessary
if [ "$l_dst_image_id" = "" ]; then
  echo -n "$$: $(date +"%Y%d%m-%H%M"): Step 2: Upload snapshot to destination..."
  echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-create --store=$l_in_img_type --container-format=$l_snap_container_format --disk-format=$l_snap_disk_format --is-public=false --name=\"$l_dst_image_name\" < \"$l_local_fname\" 2>&1" >> $l_log
  l_msg=$(glance --os-tenant-name="$l_vm_tenant" image-create --store=$l_in_img_type --container-format=$l_snap_container_format --disk-format=$l_snap_disk_format --is-public=false --name="$l_dst_image_name" < "$l_local_fname" 2>&1)
  l_rc=$?
  echo "$$: l_msg='$l_msg'" >> $l_log
  if [ $l_rc -ne 0 ]; then do_exit 5 "Snapshot upload: $l_msg"; fi

  # get the uploaded image ID
  echo "$$: glance --os-tenant-name=\"$l_vm_tenant\" image-list | grep -e \"$l_local_fname\" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##'" >> $l_log
  l_dst_image_id=$(glance --os-tenant-name="$l_vm_tenant" image-list | grep -e "$l_local_fname" | head -n 1 | cut -d'|' -f 2 | sed -e 's#^ \+##; s#[ \t]*$##')
  l_rc=$?
  if [ $l_rc -eq 0 ]; then echo "OK ($l_dst_image_id)"; else do_exit 5 "Snapshot upload identification"; fi
  echo "$$: l_dst_image_id='$l_dst_image_id'" >> $l_log
fi

That is pretty much it. The nice thing about the script is that you can use it to perform mass migrations. Just be sure to delete any intermediate files so you don’t fill up your hard disk.

Enjoy!

About

Team-oriented systems mentor with deep knowledge of numerous software methodologies, technologies, languages, and operating systems. Excited about turning emerging technology into working production-ready systems. Focused on moving software teams to a higher level of world-class application development. Specialties:Software analysis and development...Product management through the entire lifecycle...Discrete product integration specialist!

2 Comments on “OpenStack: VM Migration on OpenStack”

  1. Hi,

    I am trying to migrate VM’s between OpenStacks using your scipt. But I am not clear what kind of keystone scipt files we will need to use your script. If you can kindly me some sample keystone script so that I can procede further.

    Thanks

    Reply

    • Here you go:

      
#!/bin/sh
export OS_USERNAME=[username]
export OS_PASSWORD=[passwword]
export OS_TENANT_NAME=[tenant]
export OS_AUTH_URL=http://[URL]:35357/v2.0
export PS1='[\u@\h \W(lvosksclu100-rc-admin)]$ '
      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

*