Python文件处理里encoding和encode有事区别，bytes类型是啥意思？
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Python文件处理里encoding和encode有事区别，bytes类型是啥意思？相关的知识，希望对你有一定的参考价值。
python问题我来回答你。
首先你要知道的是，字符串在Python内部的表示是unicode(统一码、万国码)编码，很多编程语言都是这么设计的，各个国家通用编码，因此，在做编码转换时，通常需要以unicode作为中间编码，即先将其他编码的字符串解码（decode）成unicode，再从unicode编码（encode）成另一种编码。
decode的作用是将其他编码的字符串转换成unicode编码，如str1.decode('gb2312')，表示将gb2312编码的字符串str1转换成unicode编码。
encode的作用是将unicode编码转换成其他编码的字符串，如str2.encode('gb2312')，表示将unicode编码的字符串str2转换成gb2312编码。
因此，转码的时候一定要先搞明白，字符串str是什么编码，然后decode成unicode，然后再encode成其他编码。
bytes类型是 Python 3.x版本新增的数据类型，在 Python 2.x 中是不存在的。字符串是以字符为单位进行处理的，bytes类型是以字节为单位处理的。
bytes 只负责以字节序列的形式（二进制形式）来存储数据，至于这些数据到底表示什么内容（字符串、数字、图片、音频等），完全由程序的解析方式决定。
说白了，bytes 只是简单地记录内存中的原始数据，至于如何使用这些数据，bytes 并不在意，你想怎么使用就怎么使用，bytes 并不约束你的行为。
bytes 类型的数据非常适合在互联网上传输，可以用于网络通信编程；bytes 也可以用来存储图片、音频、视频等二进制格式的文件。
举个例子：
b = b'' # 创建一个空的bytes
b = byte() # 创建一个空的bytes
b = b'hello' # 直接指定这个hello是bytes类型
b = bytes('string',encoding='编码类型') #利用内置bytes方法，将字符串转换为指定编码的bytes
b = str.encode('编码类型') # 利用字符串的encode方法编码成bytes，默认为utf-8类型
bytes.decode('编码类型')：将bytes对象解码成字符串，默认使用utf-8进行解码。
参考技术A 建议你用c语言来做个试验，用写二进制bytes的方式写文件，要求记事本打开后能看到中文，做完试验你就对编码理解了Python文件处理
1. 文件的操作
1.1 打开文件
格式：
def open(file, mode=\'r\', buffering=None, encoding=None, errors=None, newline=None, closefd=True)
源码：
1 def open(file, mode=\'r\', buffering=None, encoding=None, errors=None, newline=None, closefd=True): # known special case of open
2 """
3 Open file and return a stream. Raise IOError upon failure.
4
5 file is either a text or byte string giving the name (and the path
6 if the file isn\'t in the current working directory) of the file to
7 be opened or an integer file descriptor of the file to be
8 wrapped. (If a file descriptor is given, it is closed when the
9 returned I/O object is closed, unless closefd is set to False.)
10
11 mode is an optional string that specifies the mode in which the file
12 is opened. It defaults to \'r\' which means open for reading in text
13 mode. Other common values are \'w\' for writing (truncating the file if
14 it already exists), \'x\' for creating and writing to a new file, and
15 \'a\' for appending (which on some Unix systems, means that all writes
16 append to the end of the file regardless of the current seek position).
17 In text mode, if encoding is not specified the encoding used is platform
18 dependent: locale.getpreferredencoding(False) is called to get the
19 current locale encoding. (For reading and writing raw bytes use binary
20 mode and leave encoding unspecified.) The available modes are:
21
22 ========= ===============================================================
23 Character Meaning
24 --------- ---------------------------------------------------------------
25 \'r\' open for reading (default)
26 \'w\' open for writing, truncating the file first
27 \'x\' create a new file and open it for writing
28 \'a\' open for writing, appending to the end of the file if it exists
29 \'b\' binary mode
30 \'t\' text mode (default)
31 \'+\' open a disk file for updating (reading and writing)
32 \'U\' universal newline mode (deprecated)
33 ========= ===============================================================
34
35 The default mode is \'rt\' (open for reading text). For binary random
36 access, the mode \'w+b\' opens and truncates the file to 0 bytes, while
37 \'r+b\' opens the file without truncation. The \'x\' mode implies \'w\' and
38 raises an `FileExistsError` if the file already exists.
39
40 Python distinguishes between files opened in binary and text modes,
41 even when the underlying operating system doesn\'t. Files opened in
42 binary mode (appending \'b\' to the mode argument) return contents as
43 bytes objects without any decoding. In text mode (the default, or when
44 \'t\' is appended to the mode argument), the contents of the file are
45 returned as strings, the bytes having been first decoded using a
46 platform-dependent encoding or using the specified encoding if given.
47
48 \'U\' mode is deprecated and will raise an exception in future versions
49 of Python. It has no effect in Python 3. Use newline to control
50 universal newlines mode.
51
52 buffering is an optional integer used to set the buffering policy.
53 Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
54 line buffering (only usable in text mode), and an integer > 1 to indicate
55 the size of a fixed-size chunk buffer. When no buffering argument is
56 given, the default buffering policy works as follows:
57
58 * Binary files are buffered in fixed-size chunks; the size of the buffer
59 is chosen using a heuristic trying to determine the underlying device\'s
60 "block size" and falling back on `io.DEFAULT_BUFFER_SIZE`.
61 On many systems, the buffer will typically be 4096 or 8192 bytes long.
62
63 * "Interactive" text files (files for which isatty() returns True)
64 use line buffering. Other text files use the policy described above
65 for binary files.
66
67 encoding is the name of the encoding used to decode or encode the
68 file. This should only be used in text mode. The default encoding is
69 platform dependent, but any encoding supported by Python can be
70 passed. See the codecs module for the list of supported encodings.
71
72 errors is an optional string that specifies how encoding errors are to
73 be handled---this argument should not be used in binary mode. Pass
74 \'strict\' to raise a ValueError exception if there is an encoding error
75 (the default of None has the same effect), or pass \'ignore\' to ignore
76 errors. (Note that ignoring encoding errors can lead to data loss.)
77 See the documentation for codecs.register or run \'help(codecs.Codec)\'
78 for a list of the permitted encoding error strings.
79
80 newline controls how universal newlines works (it only applies to text
81 mode). It can be None, \'\', \'\\n\', \'\\r\', and \'\\r\\n\'. It works as
82 follows:
83
84 * On input, if newline is None, universal newlines mode is
85 enabled. Lines in the input can end in \'\\n\', \'\\r\', or \'\\r\\n\', and
86 these are translated into \'\\n\' before being returned to the
87 caller. If it is \'\', universal newline mode is enabled, but line
88 endings are returned to the caller untranslated. If it has any of
89 the other legal values, input lines are only terminated by the given
90 string, and the line ending is returned to the caller untranslated.
91
92 * On output, if newline is None, any \'\\n\' characters written are
93 translated to the system default line separator, os.linesep. If
94 newline is \'\' or \'\\n\', no translation takes place. If newline is any
95 of the other legal values, any \'\\n\' characters written are translated
96 to the given string.
97
98 If closefd is False, the underlying file descriptor will be kept open
99 when the file is closed. This does not work when a file name is given
100 and must be True in that case.
101
102 A custom opener can be used by passing a callable as *opener*. The
103 underlying file descriptor for the file object is then obtained by
104 calling *opener* with (*file*, *flags*). *opener* must return an open
105 file descriptor (passing os.open as *opener* results in functionality
106 similar to passing None).
107
108 open() returns a file object whose type depends on the mode, and
109 through which the standard file operations such as reading and writing
110 are performed. When open() is used to open a file in a text mode (\'w\',
111 \'r\', \'wt\', \'rt\', etc.), it returns a TextIOWrapper. When used to open
112 a file in a binary mode, the returned class varies: in read binary
113 mode, it returns a BufferedReader; in write binary and append binary
114 modes, it returns a BufferedWriter, and in read/write mode, it returns
115 a BufferedRandom.
116
117 It is also possible to use a string or bytearray as a file for both
118 reading and writing. For strings StringIO can be used like a file
119 opened in a text mode, and for bytes a BytesIO can be used like a file
120 opened in a binary mode.
121 """
122 pass
- file:被打开的文件名称。
- mode：文件打开模式，默认模式为r。
- buffering：设置缓存模式。0表示不缓存，1表示缓存，如果大于1则表示缓存区的大小，以字节为单位。
- encoding:字符编码。建议都带上参数 encoding=\'UTF-8\'
afile=open("filetest.txt","w",encoding=\'UTF-8\')
afile.write("这里是要写到文件的内容")
afile.close()
afile=open("filetest.txt","r",encoding=\'UTF-8\')
filstr=afile.read()#读文件到变量里面，返回字符串
afile.close()
Print(filstr)
|
文件打开模式 |
|
|
‘r’ |
以只读方式打开文件，默认。 |
|
‘w’ |
以写入方式打开文件。先删除原文件内容，再重新写入新内容。如果文件不存在，则创建。 |
|
‘x’ |
以写入方式创建新文件并打开文件。 |
|
‘a’ |
以写入的方式打开文件，在文件末尾追加新内容。如果文件不存在，则创建。 |
|
‘b’ |
以二进制模式打开文件，与r、w、a、+结合使用。对于图片、视频等必须用b模式打开。 |
|
‘t’ |
文本模式，默认。与r、w、a结合使用。 |
|
‘+’ |
打开文件，追加读写。 |
|
‘U’ |
universal newline mode (deprecated弃用)支持所有的换行符号。即"U"表示在读取时，可以将 \\r \\n \\r\\n自动转换成 \\n （与 r 或r+ 模式同使用） |
语法：
#!/usr/bin/env python
# -*- coding:utf-8 -*-
f = open(\'demo\')
print(f)
f.close()
# <_io.TextIOWrapper name=\'demo\' mode=\'r\' encoding=\'UTF-8\'>
#python3默认以可读(r)、\'utf-8\'编码方式打开文件
###===r模式打开文件===
f = open(\'demo.txt\',mode=\'r\',encoding=\'utf-8\') #文件句柄
data = f.read()
print(data)
f.close()
###===w模式打开文件===
#a.该文件存在
f = open(\'demo.txt\',mode=\'w\',encoding=\'utf-8\')
# data = f.read() #io.UnsupportedOperation: not readable,不可读
f.close()
#发现使用w打开原有文件，尽管什么都不做，该文件内容已被清空(删除)
###b.该文件不存在
f = open(\'demo1.txt\',mode=\'w\',encoding=\'utf-8\')
f.close()
#不存在则创建该文件
###===a模式打开文件===
f = open(\'demo2.txt\',mode=\'a\',encoding=\'utf-8\')
# data = f.read() #io.UnsupportedOperation: not readable
f.write(\'这是一首词牌名!\')
f.close()
#该文件不存在则创建新文件并写入内容，存在则在后面追加内容。
###===b模式打开文件===
# f = open(\'demo2.txt\',mode=\'b\',encoding=\'utf-8\')
# #ValueError: binary mode doesn\'t take an encoding argument
# f = open(\'demo2.txt\',mode=\'b\')
#alueError: Must have exactly one of create/read/write/append mode and at most one plus
f = open(\'demo3.txt\',mode=\'br\')
data = f.read()
print(data)
f.close()
###===t模式打开文件===
# f = open(\'demo3.txt\',\'t\')
#ValueError: Must have exactly one of create/read/write/append mode and at most one plus
f = open(\'demo3.txt\',\'tr\')
data = f.read()
print(data)
f.close()
###===x模式打开文件===
# 如果文件存在则报错：FileExistsError: [Errno 17] File exists: \'demo4.txt\'
f = open(\'demo41.txt\',mode=\'x\',encoding=\'utf-8\')
f.write(\'人生苦短，我用python\')
f.close()
###===+模式打开文件===
# f = open(\'demo4.txt\',\'+\')
#ValueError: Must have exactly one of create/read/write/append mode and at most one plus
#+需要和a/r/w/x结合使用,并且添加追加功能，即写文件不删除原内容
f = open(\'demo4.txt\',\'r+\')
data = f.read()
print(data)
f.write("人生苦短，我用python") #不删除原有内容
f.close()
###===U模式打开文件===
f = open(\'demo5.txt\',\'U\')
# f.write(\'hello\') #io.UnsupportedOperation: not writable不可写
print(f.readlines()) #[\'adcdefg\\n\', \' dsfsdfs\\n\', \'hijklmn\']
f.close()
with...as...打开文件：
通过with打开文件，每次执行完毕之后会自动关闭该文件。
#打开一个文件
with open(\'demo\',\'r\',encoding=\'utf-8\') as f:
print(f.read())
#连续打开两个文件
with open(\'demo1.txt\',\'r\',encoding=\'utf-8\') as f1,open(\'demo2.txt\',\'r\',encoding=\'utf-8\') as f2:
print(f1.read(),f2.read())
1.2 文件读取
文件读取有三种方式，read()、readline()、readlines()。
1 def read(self, size=-1): # known case of _io.FileIO.read
2 """
3 Read at most size bytes, returned as bytes.
4
5 Only makes one system call, so less data may be returned than requested.
6 In non-blocking mode, returns None if no data is available.
7 Return an empty bytes object at EOF.
8 """
9 return ""
10
11 def readline(self, limit=-1):
12 """Read and return one line from the stream.
13
14 :type limit: numbers.Integral
15 :rtype: unicode
16 """
17 pass
18
19 def readlines(self, hint=-1):
20 """Read and return a list of lines from the stream.
21
22 :type hint: numbers.Integral
23 :rtype: list[unicode]
24 """
25 return []
read():
从文件中一次性读取所有内容，该方法耗内存。
#Read at most size bytes, returned as bytes.
f = open(\'demo\',mode=\'r\',encoding=\'utf-8\')
data = f.read()
print(data)
f.close()
f = open(\'demo.txt\',mode=\'r\',encoding=\'utf-8\')
data = f.read(5) #读取文件前5个字节
print(data)
f.close()
readline():
每次读取文件中的一行，需要使用用真表达式循环读取文件。当文件指针移动到末尾时，依然使用readline()读取文件将出现错误。因此需要添加1个判断语句，判断指针是否移动到文件尾部，并终止循环。
f = open(\'demo\',mode=\'r\',encoding=\'utf-8\')
while True:
line = f.readline()
if line:
print(line.strip())
else:
print(line) #空
print(bool(line)) #False
break
可以加参数，参数为整数，表示每行读几个字节，知道行末尾。
f = open(\'demo\',mode=\'r\',encoding=\'utf-8\')
while True:
line = f.readline(8)
if line:
print(line.strip())
else:
break
\'\'\'
###################demo文件内容##############
伫倚危楼风细细。
望极春愁，黯黯生天际。
草色烟光残照里。无言谁会凭阑意。
拟把疏狂图一醉。
对酒当歌，强乐还无味。
衣带渐宽终不悔。为伊消得人憔悴。
#########################################
运行结果：
伫倚危楼风细细。
望极春愁，黯黯生
天际。
草色烟光残照里。
无言谁会凭阑意。
拟把疏狂图一醉。
对酒当歌，强乐还
无味。
衣带渐宽终不悔。
为伊消得人憔悴。
\'\'\'
readlines():
表示多行读取，需要通过循环访问readlines()返回列表中的元素，该方法耗内存。
f = open(\'demo\',\'r\')
lines = f.readlines()
print(lines)
print(type(lines)) #返回的是一个列表<class \'list\'>
for line in lines:
print(line.strip())
f.close()
练习：读取前5行内容
1 #方法一：使用readlines一次性读取
2 f = open(\'demo\',mode=\'r\',encoding=\'utf-8\')
3 data = f.readlines()
4 for index,line in enumerate(data):
5 if index==5:
6 break
7 print(line)
8 f.close()
9
10 #方法二：使用readline逐行读取（推荐该方法，省内存）
11 f = open(\'demo\',mode=\'r\',encoding=\'utf-8\')
12 for line in range(5):
13 print(f.readline())
1.3 文件的写入
文件的写入可以使用write()、writelines()方法写入。write()把字符串写入文件，writelines()可以把列表中存储的内容写入文件。
使用wiritelines()写文件的速度更快，如果需要写入文件的字符串较多，可以使用writelines。如果只需写入少量字符串则可以使用write()
源码：
def write(self, s):
"""Write the unicode string s to the stream and return the number of
characters written.
:type b: unicode
:rtype: int
"""
return 0
def writelines(self, lines):
"""Write a list of lines to the stream.
:type lines: collections.Iterable[unicode]
:rtype: None
"""
pass
write():
f = open(\'demo\',\'a+\')
print(f.write(\'\\n这是一首词牌名\'))
f.close()
writelines():
l = [\'a\',\'b\',\'c\',\'1\',\'2\',\'3\']
f = open(\'demo\',\'a+\')
f.writelines(l) #写入内容：abcabc123
f.close()
1.4 其他操作
def close(self, *args, **kwargs): # real signature unknown
"""
Flush and close the IO object.
This method has no effect if the file is already closed.
"""
pass
#刷新并关闭文件
f = open(\'a.txt\',\'a+\',encoding=\'utf-8\')
f.write(\'hello\')
f.close()
def fileno(self, *args, **kwargs): # real signature unknown
"""
Returns underlying file descriptor if one exists.
OSError is raised if the IO object does not use a file descriptor.
"""
pass
#文件描述符
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
print(f.fileno())
f.close()
def flush(self, *args, **kwargs): # real signature unknown
"""
Flush write buffers, if applicable.
This is not implemented for read-only and non-blocking streams.
"""
pass
# 刷新文件内部缓冲区
f = open(\'a.txt\',\'a+\',encoding=\'utf-8\')
print(f.fileno())
f.write("JJJJJJJJJJJJJJJJJJ")
f.flush()
input = input("wait for input:")
f.close()
def isatty(self, *args, **kwargs): # real signature unknown
"""
Return whether this is an \'interactive\' stream.
Return False if it can\'t be determined.
"""
pass
#判断文件是不是tty设备
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
print(f.isatty())
f.close()
def readable(self, *args, **kwargs): # real signature unknown
"""
Return whether object was opened for reading.
If False, read() will raise OSError.
"""
pass
#判断文件是否可读
f = open(\'a.txt\',\'w\',encoding=\'utf-8\')
print(f.readable()) #False，因为使用w模式不可读
f.close()
def seek(self, *args, **kwargs): # real signature unknown
"""
Change stream position.
Change the stream position to the given byte offset. The offset is
interpreted relative to the position indicated by whence. Values
for whence are:
* 0 -- start of stream (the default); offset should be zero or positive
* 1 -- current stream position; offset may be negative
* 2 -- end of stream; offset is usually negative
Return the new absolute position.
"""
pass
#把文件的指针移动到一个新的位置。
#格式：seek(offset[,whence]),offset表示相对于whence的位置。whence用于设置相对位置的起点，如果whence省略，offset表示相对文件开头的位置。
#0表示从文件开头开始计算。
#1表示从当前位置开始计算。
#2表示从文件末尾开始计算。
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
print(f.read(2))
print(f.tell())
f.seek(0)
print(f.read(3))
f.close()
def seekable(self, *args, **kwargs): # real signature unknown
"""
Return whether object supports random access.
If False, seek(), tell() and truncate() will raise OSError.
This method may need to do a test seek().
"""
pass
#判断指针是否操作
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
print(f.seekable())
f.close()
def tell(self, *args, **kwargs): # real signature unknown
""" Return current stream position. """
pass
#返回指针位置
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
f.read(2)
print(f.tell()) #2
f.close()
def truncate(self, *args, **kwargs): # real signature unknown
"""
Truncate file to size bytes.
File pointer is left unchanged. Size defaults to the current IO
position as reported by tell(). Returns the new size.
"""
pass
#截断数据
f = open(\'a.txt\',\'r+\',encoding=\'utf-8\')
f.truncate() #不加参数文件清空
f.close()
#加参数，从文件开头截取
f = open(\'a.txt\',\'r+\',encoding=\'utf-8\')
f.truncate(4) #截取文件前面4个字节
f.close()
def writable(self, *args, **kwargs): # real signature unknown
"""
Return whether object was opened for writing.
If False, write() will raise OSError.
"""
pass
#判断文件是否可写
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
print(f.writable()) #False，因为是用r模式打开
f.close()
1.5 文件的删除
文件的删除需要使用os模块和os.path模块。os模块提供了对系统环境、文件、目录等操作系统级的接口函数。文件的删除需要调用remove()函数实现。要删除文件之前需要先判断文件是否存在，若存在则删除，不存在则不进行操作，如果文件被其他占用则不能删除，报错如下：PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。: \'a.txt\'。
import os
if os.path.exists(\'test.txt\'):
os.remove(\'test.txt\')
1.6 文件的复制
文件复制方法一：读、写方式复制
#边读边写
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
f_new = open(\'a_new.txt\',\'w\',encoding=\'utf-8\')
for line in f:
f_new.write(line)
f.close()
f_new.close()
#或者先全部读取，然后再写
f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
f_new = open(\'a_new.txt\',\'w\',encoding=\'utf-8\')
src = f.read()
f_new.write(src)
f.close()
f_new.close()
文件复制方法二：使用shutil模块
import shutil
shutil.copyfile(\'a.txt\',\'a1.txt\')
1.7 文件重命名
使用os.rename()模块进行文件重命名。
文件重命名：
import os
li = os.listdir(\'.\')
print(li) #返回一个文件列表
if \'a1.txt\' in li: #判断文件是否存在
os.rename(\'a1.txt\',\'A1.txt\')
文件修改后缀：
import os
#将a.doc修改为a.html
f_list = os.listdir(\'.\')
print(f_list) #f_list是一个文件列表，如[\'a.doc\', \'b.txt\', \'c.txt\']
for f_name in f_list: #循环该列吧
p = f_name.find(\'.\') #查找每个元素(字符串).的位置，如a.doc的位置是1。
print(p)
print(f_name) #a.doc
if f_name[p+1:] == \'doc\': #截取.后面的所有字符，如果是doc
new_name = f_name[:p+1]+\'html\' #则截取.和.前面的字符串加上html，即新字符串为a.html
os.rename(f_name,new_name) #将新名字重命名给旧文件上面
上面的f_name.find(\'.\')也可以使用os.path.splitext(\'.\')来返回一个元组实现，如下。
import os
f_list = os.listdir(\'.\')
for f_name in f_list:
li = os.path.splitext(f_name) #以.截取，如(\'a\', \'.doc\')
print(li)
if li[1] == \'.doc\':
new_name = li[0]+\'.html\'
os.rename(f_name,new_name)
1.8 文件的替换
文件的替换原理：将原文件读取，替换相应内容，并将新内容写入新文件中。
1 \'\'\'
2 a.txt内容如下：
3 abcdefg
4 hijklmn
5 xyz
6 xyz
7 \'\'\'
8
9 f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
10 f_new = open(\'aa.txt\',\'w\',encoding=\'utf-8\')
11
12 for line in f:
13 if \'xyz\'==line:
14 line = \'XYZ\'
15 f_new.write(line)
16
17 f.close()
18 f_new.close()
1 \'\'\'
2 a.txt内容如下：
3 abcdefg
4 hijklmn
5 xyz
6 xyz
7 \'\'\'
8
9 f = open(\'a.txt\',\'r\',encoding=\'utf-8\')
10 f_new = open(\'aa.txt\',\'w\',encoding=\'utf-8\')
11
12 for line in f.readlines():
13 f_new.write(line.replace(\'xyz\',\'XYZ\'))
14
15 f.close()
16 f_new.close()
以上是关于Python文件处理里encoding和encode有事区别，bytes类型是啥意思？的主要内容，如果未能解决你的问题，请参考以下文章
[python]UnicodeEncodeError: 'gbk' codec can't encode character 解决方法