cloudos怎么查看nova-api

Posted 2023-05-10

参考技术A 虚拟机下发异常，CAS平台看CPU 和内存资源都还充足，怎么进入对应计算节点查看nova-compute日志，通过cat /var/log/nova/nova-compute.log没找到日志，网络那里当时是能获取的，提示异常之后IP地址也没了。同步状态不行，IP地址也没有冲突，各位大佬指导一下，谢谢。

Liberty nova-api HTTP请求执行流程

本博客欢迎转载，但请注明出处 http://blog.csdn.net/ringoshen/article/details/51387038

由于能力与时间有限，文章内容难免错漏，望大家多加指正，相互进步！

0. 前言

这次看了一下nova list命令的执行过程，整个过程可以分为几步：HTTP请求、URLMap分发、过滤、APIRouter到具体执行函数，接下来使用Postman组个包并发送http请求作为开始对各个模块进行跟踪和注解。

1. HTTP请求

OpenStack组件都是通过RESTful API向外提供服务，也就是说可以通过http的方式操作OpenStack。而操作的大致步骤分为两步：身份认证、发送任务，我们可以看一下http命令的实际操作情况。
首先是身份认证，这一步由keystone完成，返回token。

之后我们可以拿这个获取到的token去进行具体操作。

2. URLMap分发

还是先看一下代码，之前在Liberty nova-api启动流程分析的最后我们看到osapi_compute app负责监听8774端口，其实这个服务就是一个URLMap的callable对象，现在接收到8774端口的http请求时，调用URLMap的call()。

# nova/api/openstack/urlmap.py

class URLMap(paste.urlmap.URLMap):
    def __call__(self, environ, start_response):
        # host = '172.29.152.111:8774'
        host = environ.get('HTTP_HOST', environ.get('SERVER_NAME')).lower()
        ... ...  ... ...
        ... ...  ... ...
        # path_info = '/v2.1/35a94cc2b7bd4f088019dbc61f6dce37'
        # mime_type = None, app_url = '/v2.1'
        # app = <function wrap at 0x7ff89c03e410>
        mime_type, app, app_url = self._path_strategy(host, port, path_info)
        if (app_url and app_url + '/' == path_info) or path_info == '/':
            supported_content_types.append('application/atom+xml')
        ... ...  ... ...
        ... ...  ... ...
        if app:
            # environ['nova.best_content_type'] = 'application/json'
            environ['nova.best_content_type'] = mime_type
            return app(environ, start_response)
        ... ...  ... ...
        ... ...  ... ...

# nova/api/openstack/urlmap.py

class URLMap(paste.urlmap.URLMap):
    def _path_strategy(self, host, port, path_info):
        ... ...  ... ...
        ... ...  ... ...
        # parts = ['', 'v2.1', '35a94cc2b7bd4f088019dbc61f6dce37']
        parts = path_info.split('/')
        if len(parts) > 1:
            # possible_app = <oslo_middleware.cors.CORS object at 0x7ff89da5ff50>
            # possible_app_url = '/v2.1'
            # 这边开始进行版本匹配，获取相应的app
            possible_app, possible_app_url = self._match(host, port, path_info)

            if possible_app and possible_app_url:
                # app_url = '/v2.1'
                app_url = possible_app_url
                # app = <function wrap at 0x7ff89c03e410>
                app = self._munge_path(possible_app, path_info, app_url)
        # mime_type = None
        # app = <function wrap at 0x7ff89c03e410>
        # app_url = '/v2.1'
        return mime_type, app, app_url

之后的_match()方法真正实现根据版本调用app。

# nova/api/openstack/urlmap.py

class URLMap(paste.urlmap.URLMap):
    # 下面涉及到初始化的时候设置的applications参数，这边列出初始化的结果
    # self.applications = 
    #     [((None, '/v2.1'), <oslo_middleware.cors.CORS object at 0x7ff89da5ff50>), 
    #     ((None, '/v2'), <oslo_middleware.cors.CORS object at 0x7ff89e15a310>), 
    #     ((None, ''), <nova.api.openstack.FaultWrapper object at 0x7ff89d9e5950>)]

    def _match(self, host, port, path_info):
        """Find longest match for a given URL path."""
        # eg. v21版本对应的是openstack_compute_api_v21 app，根据pipeline的定义，
        #     最后一个filter是cors，所以可以看见app的类型如下：
        #     domain = None
        #     app_url = '/v2.1'
        #     app = <oslo_middleware.cors.CORS object at 0x7ff89da5ff50>
        for (domain, app_url), app in self.applications:
            if domain and domain != host and domain != host + ':' + port:
                continue
            # 这里根据之前初始化设置的urlmap进行匹配
            if (path_info == app_url
                    or path_info.startswith(app_url + '/')):
                return app, app_url
        return None, None

3. 过滤

之前获取的app是经过filter包装的，接下来按顺序看一下各个filter。

[composite:openstack_compute_api_v21]
use = call:nova.api.auth:pipeline_factory_v21
keystone = cors compute_req_id faultwrap sizelimit authtoken keystonecontext osapi_compute_app_v21

3.1 cors

CORS（Cross-Origin Resource Sharing，跨域资源共享），比如在编写javascript应用时如果需要直接使用openstack api，有可能会遇到同源策略的问题，那这边就需要扩展一下跨域资源共享，从代码的角度而言就是在api返回的response的header中加入Access-Control-Request-XXX等信息，这边简单列一下响应代码。

[filter:cors]
paste.filter_factory = oslo_middleware.cors:filter_factory
oslo_config_project = nova

# oslo_middleware/base.py

class ConfigurableMiddleware(object):
    @webob.dec.wsgify
    def __call__(self, req):
        response = self.process_request(req)
        # response = None
        if response:
            return response
        # 获取app响应（调用下一个filter）
        response = req.get_response(self.application)
        (args, varargs, varkw, defaults) = getargspec(self.process_response)
        # 对返回的response进行处理
        if 'request' in args:
            return self.process_response(response, request=req)
        return self.process_response(response)

# oslo_middleware/cors.py

class CORS(base.ConfigurableMiddleware):
    def process_response(self, response, request=None):
        if 'Access-Control-Allow-Origin' in response.headers:
            return response
        if request.method == 'OPTIONS':
            return self._apply_cors_preflight_headers(request=request,
                                                      response=response)
        # 设置CORS头信息
        self._apply_cors_request_headers(request=request, response=response)

        # Finally, return the response.
        return response

# oslo_middleware/cors.py

class CORS(base.ConfigurableMiddleware):
    def _apply_cors_preflight_headers(self, request, response):
        ... ...  ... ...
        ... ...  ... ...
        response.headers['Vary'] = 'Origin'
        response.headers['Access-Control-Allow-Origin'] = origin
        if cors_config['allow_credentials']:
            response.headers['Access-Control-Allow-Credentials'] = 'true'
        if 'max_age' in cors_config and cors_config['max_age']:
            response.headers['Access-Control-Max-Age'] = str(cors_config['max_age'])
        response.headers['Access-Control-Allow-Methods'] = request_method
        if request_headers:
            response.headers['Access-Control-Allow-Headers'] = ','.join(request_headers)
        return response

3.2 compute_req_id

此模块就是给接收到的request进行编号，这样可以方便对request任务进行跟踪。

[filter:compute_req_id]
paste.filter_factory = nova.api.compute_req_id:ComputeReqIdMiddleware.factory

# nova/api/compute_req_id.py

class ComputeReqIdMiddleware(base.Middleware):

    @webob.dec.wsgify
    def __call__(self, req):
        # 创建一个uuid作为request id
        req_id = context.generate_request_id()
        # 在request的环境变量中加入req_id
        req.environ[ENV_REQUEST_ID] = req_id
        response = req.get_response(self.application)
        # 如果'x-compute-request-id'不在response的header中，那就加上req_id
        if HTTP_RESP_HEADER_REQUEST_ID not in response.headers:
            response.headers.add(HTTP_RESP_HEADER_REQUEST_ID, req_id)
        return response

# oslo_context/context.py

def generate_request_id():
    # 其实就是直接生成uuid
    return 'req-%s' % uuid.uuid4()

3.3 faultwrap

此模块只对错误进行包装处理。

[filter:faultwrap]
paste.filter_factory = nova.api.openstack:FaultWrapper.factory

# nova/api/openstack/__init__.py

class FaultWrapper(base_wsgi.Middleware):
    @webob.dec.wsgify(RequestClass=wsgi.Request)
    def __call__(self, req):
        try:
            return req.get_response(self.application)
        except Exception as ex:
            return self._error(ex, req)

3.4 sizelimit

此模块对request的body大小进行了限制，超出则抛出异常或者截取相应大小。

[filter:sizelimit]
paste.filter_factory = oslo_middleware:RequestBodySizeLimiter.factory

# oslo_middleware/sizelimit.py

class RequestBodySizeLimiter(base.ConfigurableMiddleware):
    @webob.dec.wsgify
    def __call__(self, req):
        # 最大request body大小
        max_size = self._conf_get('max_request_body_size')
        # 超出默认大小的body则异常
        if (req.content_length is not None and
                req.content_length > max_size):
            msg = _("Request is too large.")
            raise webob.exc.HTTPRequestEntityTooLarge(explanation=msg)
        # 没有content_length字段则截取max_size大小的body
        if req.content_length is None and req.is_body_readable:
            limiter = LimitingReader(req.body_file, max_size)
            req.body_file = limiter
        return self.application

3.5 authtoken

此模块主要是对token进行验证，并把token中的数据加到request的header中。

[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory

# keystonemiddleware/auth_token/__init__.py

def filter_factory(global_conf, **local_conf):
    """Returns a WSGI filter app for use with paste.deploy."""
    conf = global_conf.copy()
    conf.update(local_conf)

    def auth_filter(app):
        return AuthProtocol(app, conf)
    return auth_filter

# keystonemiddleware/auth_token/__init__.py

class BaseAuthProtocol(object):
    @webob.dec.wsgify(RequestClass=_request._AuthTokenRequest)
    def __call__(self, req):
        """Handle incoming request."""
        response = self.process_request(req)
        if response:
            return response
        response = req.get_response(self._app)
        return self.process_response(response)

接下来看一下process_request方法怎过滤request。

# keystonemiddleware/auth_token/__init__.py

class AuthProtocol(BaseAuthProtocol):
    def process_request(self, request):
        # 删除指定的header，防止被人仿冒身份进行认证
        request.remove_auth_headers()
        self._token_cache.initialize(request.environ)
        # 在基类的定义中完成token的验证
        resp = super(AuthProtocol, self).process_request(request)
        ... ...  ... ...
        ... ...  ... ...
        # 之后就是设置request header的相关参数
        if request.user_token_valid:
            user_auth_ref = request.token_auth._user_auth_ref
            request.set_user_headers(user_auth_ref)
            if self._include_service_catalog:
                request.set_service_catalog_headers(user_auth_ref)
        if request.service_token and request.service_token_valid:
            request.set_service_headers(request.token_auth._serv_auth_ref)
        if self.log.isEnabledFor(logging.DEBUG):
            self.log.debug('Received request from %s',
                           request.token_auth._log_format)

在基类的process_request中完成了token的验证，具体过程涉及keystone

# keystonemiddleware/auth_token/__init__.py

class BaseAuthProtocol(object):
    def process_request(self, request):
        user_auth_ref = None
        serv_auth_ref = None
        # 获取request的token
        if request.user_token:
            self.log.debug('Authenticating user token')
            try:
                # 获取token的数据进行验证并转化成AccessInfo形式
                # 这个动作稍复杂，主要是根据token是否在cache中进行不同的处理
                data, user_auth_ref = self._do_fetch_token(request.user_token)

                # 验证token是否快要过期等
                self._validate_token(user_auth_ref)
                self._confirm_token_bind(user_auth_ref, request)
            except ksm_exceptions.InvalidToken:
                self.log.info(_LI('Invalid user token'))
                request.user_token_valid = False
            else:
                request.user_token_valid = True
                request.token_info = data

        ... ...  ... ...
        ... ...  ... ...

        request.token_auth = _user_plugin.UserAuthPlugin(user_auth_ref, serv_auth_ref)

# keystonemiddleware/auth_token/__init__.py

class BaseAuthProtocol(object):
    def _do_fetch_token(self, token):
        # 获取token并做验证
        data = self.fetch_token(token)
        try:
            # 返回token数据以及转换成AccessInfo形式的token
            return data, access.create(body=data, auth_token=token)
        except Exception:
            self.log.warning(_LW('Invalid token contents.'), exc_info=True)
            raise ksm_exceptions.InvalidToken(_('Token authorization failed'))

# keystonemiddleware/auth_token/__init__.py

class AuthProtocol(BaseAuthProtocol):
    def fetch_token(self, token):
        data = None
        token_hashes = None
        try:
            # 为token生成hash，hash应该是token的标识符
            token_hashes = self._token_hashes(token)
            # 根据hash在cache中获取token
            cached = self._cache_get_hashes(token_hashes)

            if cached:
                # 如果token在cache中，则获取cache中的token
                data = cached
                # 进一步验证token是否被撤销
                if self._check_revocations_for_cached:
                    self._revocations.check(token_hashes)
            else:
                # 如果token不在cache中，则根据不同类型（pkiz，pki，etc）的token进行相应的处理和验证
                data = self._validate_offline(token, token_hashes)
                if not data:
                    data = self._identity_server.verify_token(token)
                # 将token存入cache
                self._token_cache.store(token_hashes[0], data)
        ... ...  ... ...
        ... ...  ... ...
        return data

3.6 keystonecontext

此模块负责把request header的内容读取出来赋值给context。

[filter:keystonecontext]
paste.filter_factory = nova.api.auth:NovaKeystoneContext.factory

# nova/api/auth.py

class NovaKeystoneContext(wsgi.Middleware):
    @webob.dec.wsgify(RequestClass=wsgi.Request)
    def __call__(self, req):
        ... ...  ... ...
        ... ...  ... ...
        # 获取request的header中的值并用以初始化RequestContext
        ctx = context.RequestContext(user_id,
                                     project_id,
                                     user_name=user_name,
                                     project_name=project_name,
                                     roles=roles,
                                     auth_token=auth_token,
                                     remote_address=remote_address,
                                     service_catalog=service_catalog,
                                     request_id=req_id,
                                     user_auth_plugin=user_auth_plugin)
        # 把上下文对象ctx加到request的环境变量
        req.environ['nova.context'] = ctx
        return self.application

4. APIRouter(V21)到具体执行函数

APIRouterV21继承自nova.wsgi.Router，主要完成对资源的加载以及http请求路由到具体方法。关于routes.mapper可以参考routes的官方文档 。

[app:osapi_compute_app_v21]
paste.app_factory = nova.api.openstack.compute:APIRouterV21.factory

这边首先看一下APIRouterV21的初始化部分，这一部分主要涉及资源的注册以及路由信息的创建。

# nova/api/openstack/compute/__init__.py

class APIRouterV21(nova.api.openstack.APIRouterV21):
    def __init__(self, init_only=None):
        # _loaded_extension_info = {}
        self._loaded_extension_info = extension_info.LoadedExtensionInfo()
        super(APIRouterV21, self).__init__(init_only)

# nova/api/openstack/__init__.py

class APIRouterV21(base_wsgi.Router):
    @staticmethod
    def api_extension_namespace():
        return 'nova.api.v21.extensions'

    def __init__(self, init_only=None, v3mode=False):
        # 验证扩展资源是否在黑名单中
        def _check_load_extension(ext):
            if (self.init_only is None or ext.obj.alias in
                self.init_only) and isinstance(ext.obj, extensions.V21APIExtensionBase):

                if (not CONF.osapi_v21.extensions_whitelist or
                        ext.obj.alias in CONF.osapi_v21.extensions_whitelist):

                    blacklist = CONF.osapi_v21.extensions_blacklist
                    if ext.obj.alias not in blacklist:
                        return self._register_extension(ext)
            return False

        ... ...  ... ...
        ... ...  ... ...
        # 使用stevedore模块载入setup.cfg中'nova.api.v21.extensions'下的所有资源，
        # 使用check_func对资源进行验证
        self.api_extension_manager = stevedore.enabled.EnabledExtensionManager(
            namespace=self.api_extension_namespace(),
            check_func=_check_load_extension,
            invoke_on_load=True,
            invoke_kwds={"extension_info": self.loaded_extension_info})
        if v3mode:
            mapper = PlainMapper()
        else:
            mapper = ProjectMapper()
        self.resources = {}
        if list(self.api_extension_manager):
            # 调用每个资源的get_resources()方法进行注册，同时使用
            # mapper.resource()建立对应关系
            self._register_resources_check_inherits(mapper)
            # 调用资源的get_controller_extensions()方法进行资源扩展
            self.api_extension_manager.map(self._register_controllers)

        ... ...  ... ...
        ... ...  ... ...

        super(APIRouterV21, self).__init__(mapper)

# nova/api/openstack/__init__.py

class APIRouterV21(base_wsgi.Router):
    def _register_resources_check_inherits(self, mapper):
        ... ...  ... ...
        ... ...  ... ...
        self._register_resources_list(ext_no_inherits, mapper)
        self._register_resources_list(ext_has_inherits, mapper)

# nova/api/openstack/__init__.py

class APIRouterV21(base_wsgi.Router):
    def _register_resources_list(self, ext_list, mapper):
        for ext in ext_list:
            self._register_resources(ext, mapper)

接下来是真正实现资源注册和mapper初始化的部分。

# nova/api/openstack/__init__.py

class APIRouterV21(base_wsgi.Router):
    def _register_resources(self, ext, mapper):
        handler = ext.obj
        LOG.debug("Running _register_resources on %s", ext.obj)
        # 使用各个资源的get_resources()方法进行创建资源，并且使用mapper创建路由信息
        for resource in handler.get_resources():
            LOG.debug('Extended resource: %s', resource.collection)

            inherits = None
            if resource.inherits:
                inherits = self.resources.get(resource.inherits)
                if not resource.controller:
                    resource.controller = inherits.controller
            # 创建一个ResourceV21对象作为之后mapper的controller
            wsgi_resource = wsgi.ResourceV21(resource.controller,
                                             inherits=inherits)
            self.resources[resource.collection] = wsgi_resource

            # 组建mapper.resource的kargs参数。
            # 这边以keypairs为例：
            # kargs = {'member': {}, 
            #          'controller': <nova.api.openstack.wsgi.ResourceV21 object at 0x7fe6842b8990>, 
            #          'collection': {}}
            # member和collection都为空说明除了标准映射之外不需要额外映射。
            kargs = dict(
                controller=wsgi_resource,
                collection=resource.collection_actions,
                member=resource.member_actions)

            if resource.parent:
                kargs['parent_resource'] = resource.parent
            if resource.member_name:
                member_name = resource.member_name
            else:
                # 以keypairs为例：
                # member_name = resource.collection = 'os-keypairs'
                member_name = resource.collection
            # 创建mapper映射规则
            mapper.resource(member_name, resource.collection,
                            **kargs)

            if resource.custom_routes_fn:
                    resource.custom_routes_fn(mapper, wsgi_resource)

至此，APIRouterV21已经初始化的过程已经大致了解了，之后开始继续之前的请求调用流程。
之前我们看到mapper中的controller是

# nova/api/openstack/wsgi.py

# 直接继承了Resource类
class ResourceV21(Resource):
    support_api_request_version = True

# nova/api/openstack/wsgi.py

class Resource(wsgi.Application):
    @webob.dec.wsgify(RequestClass=Request)
    def __call__(self, request):

        ... ...  ... ...
        ... ...  ... ...

        # 获取request中的'wsgiorg.routing_args'值，
        # 以nova list为例：action_args = {'action': u'detail', 'project_id': u'c884d1e936794f2a8e251342e1200c5d'}
        action_args = self.get_action_args(request.environ)
        # 获取方法名称
        # action_args = {'project_id': u'c884d1e936794f2a8e251342e1200c5d'}
        # action = 'detail'
        action = action_args.pop('action', None)

        try:
            # 获取报文格式和内容
            # content_type = 'text/plain'
            # body = ''
            content_type, body = self.get_body(request)

            # 获取返回的报文类型，在初始化时进行过设定
            # accept = 'application/json'
            accept = request.best_match_content_type()
        except exception.InvalidContentType:
            msg = _("Unsupported Content-Type")
            return Fault(webob.exc.HTTPBadRequest(explanation=msg))

        # 调用具体方法处理请求
        return self._process_stack(request, action, action_args,
                               content_type, body, accept)

接下来看一下调用处理请求的步骤。

# nova/api/openstack/wsgi.py

class Resource(wsgi.Application):
    def _process_stack(self, request, action, action_args,
                       content_type, body, accept):
        try:
            # 获取处理的方法meth
            # meth = <bound method ServersController.detail of <nova.api.openstack.compute.servers.ServersController object at 0x7f382bdcce90>>
            meth, extensions = self.get_method(request, action,
                                               content_type, body)
        ... ...  ... ...
        ... ...  ... ...
        try:
            contents = {}
            if self._should_have_body(request):
                if request.content_length == 0:
                    contents = {'body': None}
                    # 反序列化客户端request的body
                    contents = self.deserialize(body)
        except exception.MalformedRequestBody:
            msg = _("Malformed request body")
            return Fault(webob.exc.HTTPBadRequest(explanation=msg))

        # 更新请求的参数到action_args
        # contents = {}
        # action_args = {'project_id': u'c884d1e936794f2a8e251342e1200c5d'}
        action_args.update(contents)

        # 获取project_id(tenant_id)
        # project_id = u'c884d1e936794f2a8e251342e1200c5d'
        project_id = action_args.pop("project_id", None)
        context = request.environ.get('nova.context')
        ... ...  ... ...
        ... ...  ... ...
        # 执行前向扩展方法，扩展前序处理
        response, post = self.pre_process_extensions(extensions,
                                                     request, action_args)
        if not response:
            try:
                with ResourceExceptionHandler():
                    # ##### 执行controller中的请求处理方法 #####
                    # action_result = {'servers': []}
                    action_result = self.dispatch(meth, request, action_args)
            except Fault as ex:
                response = ex
            ... ...  ... ...
            ... ...  ... ...
            if resp_obj:
                # Do a preserialize to set up the response object
                if hasattr(meth, 'wsgi_code'):
                    resp_obj._default_code = meth.wsgi_code
                # 执行后向扩展
                response = self.post_process_extensions(post, resp_obj,
                                                        request, action_args)

            if resp_obj and not response:
                response = resp_obj.serialize(request, accept)
        ... ...  ... ...
        ... ...  ... ...
        # response = 200 OK
        #            Content-Type: application/json
        #            Content-Length: 15
        #            X-OpenStack-Nova-API-Version: 2.1
        #            Vary: X-OpenStack-Nova-API-Version
        #
        #            {"servers": []}
        return response

这里可以看一下dispatch方法的定义，可以看见就是直接调用处理方法。

# nova/api/openstack/wsgi.py

class Resource(wsgi.Application):
    def dispatch(self, method, request, action_args):
        try:
            # 直接调用method
            return method(req=request, **action_args)
        except exception.VersionNotFoundForAPIMethod:
            return Fault(webob.exc.HTTPNotFound())

最后我们看一下Controller中方法的定义。

# nova/api/openstack/compute/servers.py

class ServersController(wsgi.Controller):
    @extensions.expected_errors((400, 403))
    def detail(self, req):
        context = req.environ['nova.context']
        authorize(context, action="detail")
        try:
            # 获取servers列表
            servers = self._get_servers(req, is_detail=True)
        except exception.Invalid as err:
            raise exc.HTTPBadRequest(explanation=err.format_message())
        return servers

到此为止，一个简单的nova list 的http请求执行流程就结束了，这边我的环境中没有开启虚拟机，所以这边servers为空。