rest framework之解析器

Posted 2022-10-18 shenjianping

一、内置解析器

REST 框架包括一些内置的Parser类，允许你接受各种媒体类型的请求。还支持定义自己的自定义解析器，解析器用于解析客户端提交的数据。

• 内置解析器的使用

1、全局设置

可以使用 DEFAULT_PARSER_CLASSES 设置默认的全局解析器。例如，以下设置将只允许带有 JSON 内容的请求，而不是默认的 JSON 或表单数据。

REST_FRAMEWORK = 
    ‘DEFAULT_PARSER_CLASSES‘: (
        ‘rest_framework.parsers.JSONParser‘,
    )

这样在访问请求的每一个视图中，都被要求是JSON数据。

2、局部设置

在某一个视图下单独进行配置：

from rest_framework.parsers import JSONParser

class BookView(GenericViewSet):
    """
        该视图只接受JSON数据的post请求
    """
    parser_classes = [JSONParser,]

    def create(self,request):
        pass

• 内置解析器API种类

1、JSONParser

　　解析 JSON 请求内容。

　　.media_type： application/json

class JSONParser(BaseParser):
    """
    Parses JSON-serialized data.
    """
    media_type = ‘application/json‘
    renderer_class = renderers.JSONRenderer
    strict = api_settings.STRICT_JSON

    def parse(self, stream, media_type=None, parser_context=None):
        """
        Parses the incoming bytestream as JSON and returns the resulting data.
        """
        parser_context = parser_context or 
        encoding = parser_context.get(‘encoding‘, settings.DEFAULT_CHARSET)

        try:
            decoded_stream = codecs.getreader(encoding)(stream)
            parse_constant = json.strict_constant if self.strict else None
            return json.load(decoded_stream, parse_constant=parse_constant)
        except ValueError as exc:
            raise ParseError(‘JSON parse error - %s‘ % six.text_type(exc))

JSONParser

2、FormParser

　　解析 html 表单内容。request.data 是一个 QueryDict 字典，包含所有表单参数，通常需要同时使用 FormParser 和 MultiPartParser，以完全支持 HTML 表单数据。

　　.media_type： application/x-www-form-urlencoded

class FormParser(BaseParser):
    """
    Parser for form data.
    """
    media_type = ‘application/x-www-form-urlencoded‘

    def parse(self, stream, media_type=None, parser_context=None):
        """
        Parses the incoming bytestream as a URL encoded form,
        and returns the resulting QueryDict.
        """
        parser_context = parser_context or 
        encoding = parser_context.get(‘encoding‘, settings.DEFAULT_CHARSET)
        data = QueryDict(stream.read(), encoding=encoding)
        return data

FormParser

3、MultiPartParser

　　解析文件上传的 multipart HTML 表单内容。 request.data 是一个 QueryDict（其中包含表单参数和文件），通常需要同时使用 FormParser 和 MultiPartParser，以完全支持 HTML 表单数据。

　　.media_type： application/form-data

class MultiPartParser(BaseParser):
    """
    Parser for multipart form data, which may include file data.
    """
    media_type = ‘multipart/form-data‘

    def parse(self, stream, media_type=None, parser_context=None):
        """
        Parses the incoming bytestream as a multipart encoded form,
        and returns a DataAndFiles object.

        `.data` will be a `QueryDict` containing all the form parameters.
        `.files` will be a `QueryDict` containing all the form files.
        """
        parser_context = parser_context or 
        request = parser_context[‘request‘]
        encoding = parser_context.get(‘encoding‘, settings.DEFAULT_CHARSET)
        meta = request.META.copy()
        meta[‘CONTENT_TYPE‘] = media_type
        upload_handlers = request.upload_handlers

        try:
            parser = DjangoMultiPartParser(meta, stream, upload_handlers, encoding)
            data, files = parser.parse()
            return DataAndFiles(data, files)
        except MultiPartParserError as exc:
            raise ParseError(‘Multipart form parse error - %s‘ % six.text_type(exc))

MultiPartParser

4、FileUploadParser

　　解析文件上传内容。 request.data 是一个 QueryDict （只包含一个存有文件的 ‘file‘ key）。

　　如果与 FileUploadParser 一起使用的视图是用 filename URL 关键字参数调用的，那么该参数将用作文件名。

　　如果在没有 filename URL 关键字参数的情况下调用，则客户端必须在 Content-Disposition HTTP header 中设置文件名。例如 Content-Disposition: attachment; filename=upload.jpg。

　　.media_type： */*

注意：

• FileUploadParser 用于本地客户端，可以将文件作为原始数据请求上传。对于基于 Web 的上传，或者对于具有分段上传支持的本地客户端，您应该使用 MultiPartParser 解析器。
• 由于此解析器的 media_type 与任何 content type 都匹配，因此 FileUploadParser 通常应该是在 API 视图上设置的唯一解析器。
• FileUploadParser 遵循 Django 的标准 FILE_UPLOAD_HANDLERS 设置和 request.upload_handlers 属性。

基本用法实例：

# views.py
class FileUploadView(views.APIView):
    parser_classes = (FileUploadParser,)

    def put(self, request, filename, format=None):
        file_obj = request.data[‘file‘]
        # ...
        # do some stuff with uploaded file
        # ...
        return Response(status=204)

# urls.py
urlpatterns = [
    # ...
    url(r‘^upload/(?P<filename>[^/]+)$‘, FileUploadView.as_view())
]

基本用法实例

class FileUploadParser(BaseParser):
    """
    Parser for file upload data.
    """
    media_type = ‘*/*‘
    errors = 
        ‘unhandled‘: ‘FileUpload parse error - none of upload handlers can handle the stream‘,
        ‘no_filename‘: ‘Missing filename. Request should include a Content-Disposition header with a filename parameter.‘,
    

    def parse(self, stream, media_type=None, parser_context=None):
        """
        Treats the incoming bytestream as a raw file upload and returns
        a `DataAndFiles` object.

        `.data` will be None (we expect request body to be a file content).
        `.files` will be a `QueryDict` containing one ‘file‘ element.
        """
        parser_context = parser_context or 
        request = parser_context[‘request‘]
        encoding = parser_context.get(‘encoding‘, settings.DEFAULT_CHARSET)
        meta = request.META
        upload_handlers = request.upload_handlers
        filename = self.get_filename(stream, media_type, parser_context)

        if not filename:
            raise ParseError(self.errors[‘no_filename‘])

        # Note that this code is extracted from Django‘s handling of
        # file uploads in MultiPartParser.
        content_type = meta.get(‘HTTP_CONTENT_TYPE‘,
                                meta.get(‘CONTENT_TYPE‘, ‘‘))
        try:
            content_length = int(meta.get(‘HTTP_CONTENT_LENGTH‘,
                                          meta.get(‘CONTENT_LENGTH‘, 0)))
        except (ValueError, TypeError):
            content_length = None

        # See if the handler will want to take care of the parsing.
        for handler in upload_handlers:
            result = handler.handle_raw_input(stream,
                                              meta,
                                              content_length,
                                              None,
                                              encoding)
            if result is not None:
                return DataAndFiles(, ‘file‘: result[1])

        # This is the standard case.
        possible_sizes = [x.chunk_size for x in upload_handlers if x.chunk_size]
        chunk_size = min([2 ** 31 - 4] + possible_sizes)
        chunks = ChunkIter(stream, chunk_size)
        counters = [0] * len(upload_handlers)

        for index, handler in enumerate(upload_handlers):
            try:
                handler.new_file(None, filename, content_type,
                                 content_length, encoding)
            except StopFutureHandlers:
                upload_handlers = upload_handlers[:index + 1]
                break

        for chunk in chunks:
            for index, handler in enumerate(upload_handlers):
                chunk_length = len(chunk)
                chunk = handler.receive_data_chunk(chunk, counters[index])
                counters[index] += chunk_length
                if chunk is None:
                    break

        for index, handler in enumerate(upload_handlers):
            file_obj = handler.file_complete(counters[index])
            if file_obj is not None:
                return DataAndFiles(, ‘file‘: file_obj)

        raise ParseError(self.errors[‘unhandled‘])

    def get_filename(self, stream, media_type, parser_context):
        """
        Detects the uploaded file name. First searches a ‘filename‘ url kwarg.
        Then tries to parse Content-Disposition header.
        """
        try:
            return parser_context[‘kwargs‘][‘filename‘]
        except KeyError:
            pass

        try:
            meta = parser_context[‘request‘].META
            disposition = parse_header(meta[‘HTTP_CONTENT_DISPOSITION‘].encode(‘utf-8‘))
            filename_parm = disposition[1]
            if ‘filename*‘ in filename_parm:
                return self.get_encoded_filename(filename_parm)
            return force_text(filename_parm[‘filename‘])
        except (AttributeError, KeyError, ValueError):
            pass

    def get_encoded_filename(self, filename_parm):
        """
        Handle encoded filenames per RFC6266. See also:
        https://tools.ietf.org/html/rfc2231#section-4
        """
        encoded_filename = force_text(filename_parm[‘filename*‘])
        try:
            charset, lang, filename = encoded_filename.split(‘\\‘‘, 2)
            filename = urlparse.unquote(filename)
        except (ValueError, LookupError):
            filename = force_text(filename_parm[‘filename‘])
        return filename

FileUploadParser

二、源码

首先，请求进来后还是会走到APIView的dispatch方法，这在之前的认证、权限中已经详细说明：

1、dispatch

 def dispatch(self, request, *args, **kwargs):
        """
        `.dispatch()` is pretty much the same as Django‘s regular dispatch,
        but with extra hooks for startup, finalize, and exception handling.
        """
        self.args = args
        self.kwargs = kwargs
        #rest-framework重构request对象
        request = self.initialize_request(request, *args, **kwargs)
        self.request = request
        self.headers = self.default_response_headers  # deprecate?
        try:
            self.initial(request, *args, **kwargs)

            # Get the appropriate handler method
            #这里和CBV一样进行方法的分发
            if request.method.lower() in self.http_method_names:
                handler = getattr(self, request.method.lower(),
                                  self.http_method_not_allowed)
            else:
                handler = self.http_method_not_allowed

            response = handler(request, *args, **kwargs)

        except Exception as exc:
            response = self.handle_exception(exc)

        self.response = self.finalize_response(request, response, *args, **kwargs)
        return self.response

2、initialize_request

在dispatch方法中对request进行了重构，在initialize_request方法中：

    def initialize_request(self, request, *args, **kwargs):
        """
        Returns the initial request object.
        """
        parser_context = self.get_parser_context(request)

        return Request(
            request, #传入原request
            parsers=self.get_parsers(),#解析器
            authenticators=self.get_authenticators(), #认证
            negotiator=self.get_content_negotiator(),
            parser_context=parser_context
        )

显然在重构的request对象中封装了parsers，而它是通过get_parsers方法获取到的一个个parser对象的列表：

    def get_parsers(self):
        """
        Instantiates and returns the list of parsers that this view can use.
        """
        return [parser() for parser in self.parser_classes]

由此可知，request对象中已经封装了视图中提供的解析器对象列表。

3、request.data

从客户端发过来的数据，使用request.data进行接收，内部对其进行解析，查看rest framework的request对象（位于 rest_framework.request.Request）中的data：

class Request(object):
...
        @property
    def data(self):
        if not _hasattr(self, ‘_full_data‘):
            self._load_data_and_files()
        return self._full_data
...

4、_load_data_and_files

class Request(object):
...

    def _load_data_and_files(self):
        """
        Parses the request content into `self.data`.
        """
        if not _hasattr(self, ‘_data‘):

           #解析request.data
            self._data, self._files = self._parse()
            if self._files:
                self._full_data = self._data.copy()
                self._full_data.update(self._files)
            else:
                self._full_data = self._data

            # if a form media type, copy data & files refs to the underlying
            # http request so that closable objects are handled appropriately.
            if is_form_media_type(self.content_type):
                self._request._post = self.POST
                self._request._files = self.FILES

...    

5、_parse

    def _parse(self):
        """
        Parse the request content, returning a two-tuple of (data, files)

        May raise an `UnsupportedMediaType`, or `ParseError` exception.
        """
        media_type = self.content_type
        try:
            stream = self.stream
        except RawPostDataException:
            if not hasattr(self._request, ‘_post‘):
                raise
            # If request.POST has been accessed in middleware, and a method=‘POST‘
            # request was made with ‘multipart/form-data‘, then the request stream
            # will already have been exhausted.
            if self._supports_form_parsing():
                return (self._request.POST, self._request.FILES)
            stream = None

        if stream is None or media_type is None:
            if media_type and is_form_media_type(media_type):
                empty_data = QueryDict(‘‘, encoding=self._request._encoding)
            else:
                empty_data = 
            empty_files = MultiValueDict()
            return (empty_data, empty_files)

        parser = self.negotiator.select_parser(self, self.parsers)

        if not parser:
            raise exceptions.UnsupportedMediaType(media_type)

        try:
            parsed = parser.parse(stream, media_type, self.parser_context)
        except Exception:
            # If we get an exception during parsing, fill in empty data and
            # re-raise.  Ensures we don‘t simply repeat the error when
            # attempting to render the browsable renderer response, or when
            # logging the request or similar.
            self._data = QueryDict(‘‘, encoding=self._request._encoding)
            self._files = MultiValueDict()
            self._full_data = self._data
            raise

        # Parser classes may return the raw data, or a
        # DataAndFiles object.  Unpack the result as required.
        try:
            return (parsed.data, parsed.files)
        except AttributeError:
            empty_files = MultiValueDict()
            return (parsed, empty_files)

_parse

获取配置解析器的media_type，然后通过与请求头中的content_type进行比较选择合适的解析器

    parser = self.negotiator.select_parser(self, self.parsers)

6、select_parser

class DefaultContentNegotiation(BaseContentNegotiation):
    settings = api_settings

    def select_parser(self, request, parsers):
        """
        Given a list of parsers and a media type, return the appropriate
        parser to handle the incoming request.
        """
        for parser in parsers:
            #通过比较请求头中的content_type，匹配合适的解析器
            if media_type_matches(parser.media_type, request.content_type):
                return parser
        return None

7、以JSONParser为例

假设选择的就是视图配置的JSONParser解析器，那么会调用解析器的parser方法进行解析，parserd参数接收并且返回parserd.data，最后self._data = parserd.data

class JSONParser(BaseParser):
    """
    Parses JSON-serialized data.
    """
    media_type = ‘application/json‘
    renderer_class = renderers.JSONRenderer
    strict = api_settings.STRICT_JSON

    def parse(self, stream, media_type=None, parser_context=None):
        """
        Parses the incoming bytestream as JSON and returns the resulting data.
        """
        parser_context = parser_context or 
        encoding = parser_context.get(‘encoding‘, settings.DEFAULT_CHARSET)

        try:
            decoded_stream = codecs.getreader(encoding)(stream)
            parse_constant = json.strict_constant if self.strict else None
            return json.load(decoded_stream, parse_constant=parse_constant)
        except ValueError as exc:
            raise ParseError(‘JSON parse error - %s‘ % six.text_type(exc))

总结：

• APIView中的dispatch将视图中配置的解析器对象封装到request对象中
• request.data触发解析
• 获取请求中的content_type，并且与解析器中的media_type进行匹配，选择合适的解析器
• 使用解析器中的parser方法进行解析
• 将解析结果赋值给request._data

参考文档：https://q1mi.github.io/Django-REST-framework-documentation/api-guide/parsers_zh/#_1