[Python]pyhon去除txt文件重复行 python 2020.2.10

Posted 2021-03-10 雾霾王者

代码如下：

 1 import shutil
 2 
 3 readPath=‘E:/word4.txt‘ #要处理的文件
 4 writePath=‘E:/word5.txt‘ #要写入的文件
 5 lines_seen=set()
 6 outfiile=open(writePath,‘a+‘,encoding=‘utf-8‘)
 7 f=open(readPath,‘r‘,encoding=‘utf-8‘)
 8 for line in f:
 9     if line not in lines_seen:
10         outfiile.write(line)
11         lines_seen.add(line)