Openstack liberty 创建实例快照源码分析1
Posted lizhongwen1987
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Openstack liberty 创建实例快照源码分析1相关的知识，希望对你有一定的参考价值。
Openstack liberty中也支持对云主机执行快照，快照是备份系统中一种常用的数据保护技术，在生产系统故障后，能通过快照将系统快速恢复到快照时间点；那Openstack中的云主机快照是否也支持这种故障恢复呢？请看下文：
Openstack支持对处于运行或者停止状态的云主机执行快照，另外Openstack既可以从镜像启动云主机，也可以从启动磁盘启动云主机，根据条件组合，可以执行下面4中快照：
- 镜像启动云主机的离线快照
- 镜像启动云主机的在线快照
- 磁盘启动云主机的离线快照
- 磁盘启动云主机的在线快照
Openstack是否支持上述4种快照组合方式呢？ 下文将为您解答。
预计文章篇幅会很长，准备用两篇博文来分析具体的快照过程，本文是第一篇:分析从镜像启动云主机的离线/在线快照过程，下面来看具体内容：
镜像启动云主机的离线快照
通过Openstack Dashboard或者nova命令可以发起快照，如下：
//快照命令格式：nova image-create {server} {name}
//下面的命令对id=814a8ad8-9217-4c45-91c7-c2be2016e5da的云主机执行快
//照，快照名称为snapshot1
# nova image-create 814a8ad8-9217-4c45-91c7-c2be2016e5da snapshot1nova-api部分
根据nova-api启动时设置的路由，我们可以很容易的知道快照函数入口是：
nova/api/openstack/compute/servers.py/ServersController._action_create_image， 下面一起来看看代码：
def _action_create_image(self, req, id, body):
"""Snapshot a server instance.
输入参数如下：
req = Request对象，包含本次请求的上下文
id = 814a8ad8-9217-4c45-91c7-c2be2016e5da
body = {u‘createImage‘: {u‘name‘: u‘snapshot1‘, u‘metadata‘: {}}}
"""
#得到请求上下文，并执行权限验证
context = req.environ[‘nova.context‘]
authorize(context, action=‘create_image‘)
#从body中分解出快照名及属性
#image_name = ‘snapshot1‘
#metadata = {}
entity = body["createImage"]
image_name = common.normalize_name(entity["name"])
metadata = entity.get(‘metadata‘, {})
#从nova数据库中获取实例信息，包括：metadata,system_metadata
#security_groups,info_cache,flavor及pci_devices等属性信息
#返回InstanceV2对象
instance = self._get_server(context, req, id)
#数据库中获取该实例关联的所有块设备，返回BlockDeviceMappingList对
#象
bdms = objects.BlockDeviceMappingList.get_by_instance_uuid(
context, instance.uuid)
"""下文省略try{ }except异常处理代码"""
#这里判断系统磁盘类型是否是volume,如果是，说明是从磁盘启动的实例
#这种类型的快照，在下一篇博文中分析，敬请期待
if self.compute_api.is_volume_backed_instance(context, instance,
bdms):
authorize(context, action="create_image:allow_volume_backed")
image = self.compute_api.snapshot_volume_backed(
context,
instance,
image_name,
extra_properties=
metadata)
#镜像启动的实例，执行快照走这个分支，调用：nova/compute/api.py/API
#执行快照，下文具体分析
else:
image = self.compute_api.snapshot(context,
instance,
image_name,
extra_properties=metadata)
# build location of newly-created image entity
image_id = str(image[‘id‘])
#根据glance.conf配置，生成镜像url，我的例子中是：
#http://$glance_host:$glance_port/images/‘ffb841fd-d5f8-
#4146-bb29-b12eb5bbf6b2‘
image_ref = glance.generate_image_url(image_id)
resp = webob.Response(status_int=202)
resp.headers[‘Location‘] = image_ref
return resp
--------------------------------------------------------------
#接上文： nova/compute/api.py/API.snapshot
def snapshot(self, context, instance, name, extra_properties=None):
"""context 请求上下文
instance InstanceV2实例对象
name 快照名 ‘snapshot1’
extra_properties 快照属性 {}
"""
"""在glance数据库（images表）中添加一条类型为‘snapshot‘的条目，每
个properties属性作为一条记录添加到image_properties表；
{
‘status‘: u‘queued‘,
‘name‘: u‘snapshot1‘,
‘deleted‘: False,
‘container_format‘: u‘bare‘,
‘created_at‘: datetime.datetime(2016, 6, 22, 7, 26, 29,
tzinfo=<iso8601.Utc>),
‘disk_format‘: u‘raw‘,
‘updated_at‘: datetime.datetime(2016, 6, 22, 7, 26, 29,
tzinfo=<iso8601.Utc>),
‘id‘: u‘ffb841fd-d5f8-4146-bb29-b12eb5bbf6b2‘,
‘owner‘: u‘25520b29dce346d38bc4b055c5ffbfcb‘,
‘min_ram‘: 0,
‘checksum‘: None,
‘min_disk‘: 20,
‘is_public‘: False,
‘deleted_at‘: None,
‘properties‘: {
u‘image_type‘: u‘snapshot‘,
u‘instance_uuid‘: u‘814a8ad8-9217-4c45-91c7-c2be2016e5da‘,
u‘user_id‘: u‘b652f9bd65844f739684a20ed77e9a0f‘,
u‘base_image_ref‘: u‘e0cc468f-6501-4a85-9b19-70e782861387‘
},
‘size‘: 0
}
"""
image_meta = self._create_image(context, instance, name,
‘snapshot‘,
extra_properties=extra_properties)
#更新实例的任务状态：镜像快照等待中
instance.task_state = task_states.IMAGE_SNAPSHOT_PENDING
instance.save(expected_task_state=[None])
#通过rpc.cast异常rpc调用投递‘snapshot_instance‘消息到消息队列，
#之后`nova-compute`会处理该请求，下文具体分析
self.compute_rpcapi.snapshot_instance(context, instance,
image_meta[‘id‘])
return image_metanova-compute部分
收到nova-api发来的’snapshot_instance’快照请求后，nova-compute调用
nova/compute/manager.py/ComputeManager.snapshot_instance方法处理该请求，如下：
def snapshot_instance(self, context, image_id, instance):
"""该方法实现很简单：设置实例任务状态后，直接将请求转交给
_snapshot_instance方法处理
输入参数：
context 请求上下文
image_id = ffb841fd-d5f8-4146-bb29-b12eb5bbf6b2,nova-api传过
来的快照id
instance InstanceV2实例对象
"""
"""省略异常处理代码
如果实例为deleted或者deleting状态，则抛异常
"""
#更新实例任务状态：快照中
instance.task_state = task_states.IMAGE_SNAPSHOT
instance.save(
expected_task_state=task_states.IMAGE_SNAPSHOT_PENDING)
#下文具体分析
self._snapshot_instance(context, image_id, instance,
task_states.IMAGE_SNAPSHOT)
-------------------------------------------------------------
#接上文：
def _snapshot_instance(self, context, image_id, instance,
expected_task_state):
#返回一个设置admin=True的上下文副本
context = context.elevated()
"""省略异常处理代码
如果在执行快照过程中实例被删除deleting/deleted,或者快照被删除则抛异
常
"""
#获取实例电源状态，并更新
instance.power_state = self._get_power_state(context, instance)
instance.save()
#在日志中显示‘instance snapshotting‘日志
LOG.info(_LI(‘instance snapshotting‘), context=context,
instance=instance)
#如果实例处于非运行状态，则记录告警日志
if instance.power_state != power_state.RUNNING:
state = instance.power_state
running = power_state.RUNNING
LOG.warning(_LW(‘trying to snapshot a non-running‘
‘instance: (state: %(state)s expected: %(running)s)‘),
{‘state‘: state, ‘running‘: running},
instance=instance)
#通过notifier发送‘snapshot.start‘通知消息
self._notify_about_instance_usage(
context, instance, "snapshot.start")
#实例状态更新辅助函数
def update_task_state(task_state,
expected_state=expected_task_state):
instance.task_state = task_state
instance.save(expected_task_state=expected_state)
#调用LibvirtDriver.snapshot执行快照，下文具体分析
self.driver.snapshot(context, instance, image_id,
update_task_state)
#更新实例任务状态：None
instance.task_state = None
instance.save(expected_task_state=
task_states.IMAGE_UPLOADING)
#通过notifier发送‘snapshot.end‘通知消息
self._notify_about_instance_usage(context, instance,
"snapshot.end")
--------------------------------------------------------------
#接上文：nova/virt/libvirt/driver.py/LibvirtDriver.snapshot
def snapshot(self, context, instance, image_id, update_task_state):
"""省略异常处理代码
如果实例已经删除（不存在），则抛异常
"""
#通过libvirt获取instance表示的实例的libvirt.virDomain对象
guest = self._host.get_guest(instance)
virt_dom = guest._domain
#从instance的system_metadata属性提取系统盘镜像信息，返回
#ImageMeta对象
image_meta = objects.ImageMeta.from_instance(instance)
#从glance数据库提取快照信息，该信息在nova-api中记录数据库
#具体的信息情况上文nova-api一节
snapshot = self._image_api.get(context, image_id)
#从实例的xml配置文件中解析出系统磁盘信息
#disk_path = ‘rbd:vms/814a8ad8-9217-4c45-91c7-c2be2016e5da_disk‘
#source_format = raw
disk_path, source_format = libvirt_utils.find_disk(virt_dom)
#从磁盘路径中解析出后端存储类型，这里是rbd
source_type = libvirt_utils.get_disk_type_from_path(disk_path)
#修正后端存储类型及快照磁盘类型
#如果未能从磁盘路径中解析出后端存储类型，就用磁盘格式类型作为后端类型
#使用‘snapshot_image_format ‘或者后端存储类型作为快照磁盘类型，
#如果快照类型为lvm或者rbd，就修改为raw格式
if source_type is None:
source_type = source_format
image_format = CONF.libvirt.snapshot_image_format or source_type
if image_format == ‘lvm‘ or image_format == ‘rbd‘:
image_format = ‘raw‘
"""根据系统盘镜像属性，快照属性及快照磁盘格式生成快照属性字典，用来
上传快照文件时更新glance数据库条目，属性字典信息如下：
{
‘status‘: ‘active‘,
‘name‘: u‘snapshot1‘,
‘container_format‘: u‘bare‘,
‘disk_format‘: ‘raw‘,
‘is_public‘: False,
‘properties‘: {
‘kernel_id‘: u‘‘,
‘image_location‘: ‘snapshot‘,
‘image_state‘: ‘available‘,
‘ramdisk_id‘: u‘‘,
‘owner_id‘: u‘25520b29dce346d38bc4b055c5ffbfcb‘
}
}
"""
metadata = self._create_snapshot_metadata(image_meta,
instance,
image_format,
snapshot[‘name‘])
#本地的临时快照文件名
snapshot_name = uuid.uuid4().hex
#获取实例电源状态，用来判断是执行在线快照还是离线快照
state = guest.get_power_state(self._host)
"""判断是执行在线快照还是离线快照，在线快照需要同时满足下面的条件：
1. QEMU >= 1.3 && libvirt >= 1.0.0
2. nova后端存储非lvm或者rbd
3. 未开启外部存储加密功能 ephemeral_storage_encryption = False
4. 未关闭在线快照disable_libvirt_livesnapshot = False
在线快照的处理逻辑，在下文的‘镜像启动云主机的在线快照‘一节分析
"""
if (self._host.has_min_version(MIN_LIBVIRT_LIVESNAPSHOT_VERSION,
MIN_QEMU_LIVESNAPSHOT_VERSION,
host.HV_DRIVER_QEMU)
and source_type not in (‘lvm‘, ‘rbd‘)
and not CONF.ephemeral_storage_encryption.enabled
and not CONF.workarounds.disable_libvirt_livesnapshot):
live_snapshot = True
"""Abort is an idempotent operation, so make sure any
block jobs which may have failed are ended. This
operation also confirms the running instance, as
opposed to the system as a whole, has a new enough
version of the hypervisor (bug 1193146).
"""
try:
virt_dom.blockJobAbort(disk_path, 0)
except libvirt.libvirtError as ex:
error_code = ex.get_error_code()
if error_code == libvirt.VIR_ERR_CONFIG_UNSUPPORTED:
live_snapshot = False
else:
pass
#我采用ceph rbd作为glance及nova的后端存储，所以执行的是离线快照
else:
live_snapshot = False
#关机状态，执行离线快照
if state == power_state.SHUTDOWN:
live_snapshot = False
#如果采用的是非lxc虚拟化程序，执行离线快照并且实例处于运行或者暂停
#状态，在执行快照前需要卸载pci设备及sriov端口
if CONF.libvirt.virt_type != ‘lxc‘ and not live_snapshot:
if state == power_state.RUNNING or state ==
power_state.PAUSED:
self._detach_pci_devices(guest,
pci_manager.get_instance_pci_devs(instance))
self._detach_sriov_ports(context, instance, guest)
#使实例进入saved的状态，实例被挂起为执行快照做准备
guest.save_memory_state()
#根据后端存储创建驱动实例，我这里是Rbd
snapshot_backend = self.image_backend.snapshot(instance,
disk_path,
image_type=source_type)
#显示log
if live_snapshot:
LOG.info(_LI("Beginning live snapshot process"),
instance=instance)
else:
LOG.info(_LI("Beginning cold snapshot process"),
instance=instance)
#调用传进来的辅助函数，更新实例任务状态：等待镜像上传
update_task_state(task_state=task_states.IMAGE_PENDING_UPLOAD)
#根据nova.conf配置生成本地快照目录
#我的配置是：‘/opt/stack/data/nova/instances/snapshots‘
snapshot_directory = CONF.libvirt.snapshots_directory
fileutils.ensure_tree(snapshot_directory)
#生成一个临时目录
with utils.tempdir(dir=snapshot_directory) as tmpdir:
try:
#拼接生成本地临时快照文件
out_path = os.path.join(tmpdir, snapshot_name)
#在线快照的处理逻辑，在下文的‘镜像启动云主机的在线快照‘一节分
#析
if live_snapshot:
# NOTE(xqueralt): libvirt needs o+x in the temp
#directory
os.chmod(tmpdir, 0o701)
self._live_snapshot(context, instance, guest,
disk_path,
out_path,
source_format,
image_format,
image_meta)
#调用后端存储驱动执行快照，Rbd.snapshot_extract，内部实现
#调用‘qemu-img convert‘拷贝系统磁盘到out_path文件中，
#命令如下：
"""qemu-img convert -O raw rbd:vms/814a8ad8-9217-
4c45-91c7-c2be2016e5da_disk:id=cinder:
conf=/etc/ceph/ceph.conf‘
/opt/stack/data/nova/instances/snapshots/
tmptR6hog/e44639af86434069b38f835847083697
-f raw
"""
else:
snapshot_backend.snapshot_extract(out_path,
image_format)
finally:
guest = None
"""NOTE(dkang): because previous managedSave is not
called for LXC, _create_domain must not be called.
"""
"""上文卸载了pci设备及sriov端口，快照完成后需要重新挂载上
去"""
if CONF.libvirt.virt_type != ‘lxc‘ and not
live_snapshot:
if state == power_state.RUNNING:
guest = self._create_domain(domain=virt_dom)
elif state == power_state.PAUSED:
guest = self._create_domain(
domain=virt_dom, pause=True)
if guest is not None:
self._attach_pci_devices(guest,
pci_manager.get_instance_pci_devs(instance))
self._attach_sriov_ports(context, instance,
guest)
LOG.info(_LI("Snapshot extracted, beginning image"
"upload"), instance=instance)
#调用传进来的辅助方法，更新实例任务状态：上传镜像中
update_task_state(task_state=
task_states.IMAGE_UPLOADING,
expected_state=task_states.IMAGE_PENDING_UPLOAD)
#通过glance api上传快照到glance后端存储中存储，阅读
#过‘glance上传镜像源码分析’文章的读者应该知道，这一步相当于
#上传镜像过程中的最后一步：upload
with libvirt_utils.file_open(out_path) as image_file:
self._image_api.update(context,
image_id,
metadata,
image_file)
#打印日志
LOG.info(_LI("Snapshot image upload complete"),
instance=instance)到这里，从镜像启动的云主机的离线快照就分析完了，总结如下：
- 快照时，需要先在本地生成临时快照，再上传到glance，效率比较低
- 快照过程中，云主机包括如下任何状态转换：（None）镜像快照等待中 -> 快照中 -> 等待镜像上传 -> 上传镜像中 -> None
- 如果nova以lvm或者ceph rbd做后端存储，则任何情况下都不支持在线快照
- openstack中的实例快照以镜像形式存储在glance中，不同于通常理解的快照用于数据恢复
镜像启动云主机的在线快照
看完上文‘镜像启动云主机的离线快照’分析，相信您已经发现在代码实现上，‘镜像启动云主机的在线快照’差异都体现在上文分析的
nova/virt/libvirt/driver.py/LibvirtDriver.snapshot方法中，一起再来看看该方法的实现：
def snapshot(self, context, instance, image_id, update_task_state):
"""省略了与离线快照相关的代码及异常处理"""
......
"""判断是执行在线快照还是离线快照，在线快照需要同时满足下面的条件：
1. QEMU >= 1.3 && libvirt >= 1.0.0
2. nova后端存储非lvm或者rbd
3. 未开启外部存储加密功能 ephemeral_storage_encryption = False
4. 未关闭在线快照disable_libvirt_livesnapshot = False
如果满足上述所有的添加，则执行在线快照
"""
if (self._host.has_min_version(MIN_LIBVIRT_LIVESNAPSHOT_VERSION,
MIN_QEMU_LIVESNAPSHOT_VERSION,
host.HV_DRIVER_QEMU)
and source_type not in (‘lvm‘, ‘rbd‘)
and not CONF.ephemeral_storage_encryption.enabled
and not CONF.workarounds.disable_libvirt_livesnapshot):
live_snapshot = True
"""Abort is an idempotent operation, so make sure any
block jobs which may have failed are ended. This
operation also confirms the running instance, as
opposed to the system as a whole, has a new enough
version of the hypervisor (bug 1193146).
"""
try:
"""执行快照前，需要结束系统盘上的所有任务（libvirt）"""
virt_dom.blockJobAbort(disk_path, 0)
except libvirt.libvirtError as ex:
error_code = ex.get_error_code()
if error_code == libvirt.VIR_ERR_CONFIG_UNSUPPORTED:
live_snapshot = False
else:
pass
#离线快照
else:
live_snapshot = False
......
#生成一个临时目录
with utils.tempdir(dir=snapshot_directory) as tmpdir:
try:
#拼接生成本地临时快照文件
out_path = os.path.join(tmpdir, snapshot_name)
#如果满足前述的所有条件，就执行在线快照，下文具体分析
if live_snapshot:
# NOTE(xqueralt): libvirt needs o+x in the temp
#directory
os.chmod(tmpdir, 0o701)
"""disk_path, source_format分别表示系统盘的路径及
格式类型
out_path,image_format分别表示临时快照文件路径及
格式类型
"""
self._live_snapshot(context, instance, guest,
disk_path,
out_path,
source_format,
image_format,
image_meta)
#调用后端存储驱动执行快照，Rbd.snapshot_extract，内部实现
#调用‘qemu-img convert‘拷贝系统磁盘到out_path文件中，
#命令如下：
"""qemu-img convert -O raw rbd:vms/814a8ad8-9217-
4c45-91c7-c2be2016e5da_disk:id=cinder:
conf=/etc/ceph/ceph.conf‘
/opt/stack/data/nova/instances/snapshots/
tmptR6hog/e44639af86434069b38f835847083697
-f raw
"""
else:
snapshot_backend.snapshot_extract(out_path,
image_format)
finally:
guest = None
"""NOTE(dkang): because previous managedSave is not
called for LXC, _create_domain must not be called.
"""
"""上文卸载了pci设备及sriov端口，快照完成后需要重新挂载上
去"""
if CONF.libvirt.virt_type != ‘lxc‘ and not
live_snapshot:
if state == power_state.RUNNING:
guest = self._create_domain(domain=virt_dom)
elif state == power_state.PAUSED:
guest = self._create_domain(
domain=virt_dom, pause=True)
if guest is not None:
self._attach_pci_devices(guest,
pci_manager.get_instance_pci_devs(instance))
self._attach_sriov_ports(context, instance,
guest)
LOG.info(_LI("Snapshot extracted, beginning image"
"upload"), instance=instance)
......
---------------------------------------------------------------
#接上文：_live_snapshot
def _live_snapshot(self, context, instance, guest, disk_path,
out_path,
source_format,
image_format,
image_meta):
#创建一个BlockDevice对象
dev = guest.get_block_device(disk_path)
#返回实例的xml配置
xml = guest.get_xml_desc(dump_inactive=True, dump_sensitive=True)
#快照前，结束系统磁盘上的所有作业
try:
dev.abort_job()
except Exception:
pass
#通过qemu-img info 获取系统磁盘的virtual size
src_disk_size = libvirt_utils.get_disk_size(disk_path,
format=source_format)
#通过qemu-img info获取系统磁盘的backing_file
src_back_path = libvirt_utils.get_disk_backing_file(disk_path,
format=source_format,
basename=False)
"""通过qemu-img create创建一个cow文件，可能的命令如下：
qemu-img create -f qcow2 -o backing_file=$src_back_path
cluster_size=$cluster_size size=$src_disk_size disk_delta
"""
disk_delta = out_path + ‘.delta‘
libvirt_utils.create_cow_image(src_back_path, disk_delta,
src_disk_size)
#根据镜像属性确定是否需要静默文件系统，如果需要，则调用libvirt
#接口静默文件系统
require_quiesce = image_meta.properties.get(
‘os_require_quiesce‘, False)
if require_quiesce:
self.quiesce(context, instance, image_meta)
try:
""" NOTE (rmk): blockRebase cannot be executed on
persistent domains, so we need to temporarily undefine
it. If any part of this block fails, the domain is
re-defined regardless.
"""
#有持久配置的实例不支持rebase，所以需要先undefine掉配置文件
if guest.has_persistent_configuration():
guest.delete_configuration()
"""NOTE (rmk): Establish a temporary mirror of our root
disk and issue an abort once we have a complete copy.
"""
#disk_delta是在上文创建的qcow2格式的cow文件
dev.rebase(disk_delta, copy=True, reuse_ext=True,
shallow=True)
while dev.wait_for_job():
time.sleep(0.5)
dev.abort_job()
libvirt_utils.chown(disk_delta, os.getuid())
finally:
#重新define实例
self._host.write_instance_config(xml)
#如果上面静默了文件系统，这里要重新解冻文件系统
if require_quiesce:
self.unquiesce(context, instance, image_meta)
""" Convert the delta (CoW) image with a backing file to a
flat image with no backing file.
"""
"""通过qemu-img convert将cow格式的磁盘文件拷贝到out_path,可能命令
如下: qemu_img convert -f qcow2 -O $image_format $disk_delta
$out_path
"""
libvirt_utils.extract_snapshot(disk_delta, ‘qcow2‘,
out_path, image_format) 在线快照源码就分析到这里，与离线快照最大的区别就是：在线快照不会挂起实例；相同点是：都需要先在本地生成临时快照文件，再上传到glance。
Openstack Mitaka已经发布，发现它的release notes, 更新了以ceph rbd做后端存储的快照实现方式，后续打算专门写一篇博文介绍这次更新，敬请期待。
从启动盘启动云主机的离线（在线）快照，在下一篇博文’Openstack liberty 创建实例快照源码分析2’种分析，敬请期待
以上是关于Openstack liberty 创建实例快照源码分析1的主要内容，如果未能解决你的问题，请参考以下文章