利用Python将txt文件批量去除重复行内容
Posted 皓月盈江
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了利用Python将txt文件批量去除重复行内容相关的知识,希望对你有一定的参考价值。
main.py
# -*- coding: utf-8 -*-
if __name__ == '__main__':
with open(r'cfg.txt', 'r', encoding='utf-8') as f:
listNet = f.readlines()
print(f"去重前数量:len(listNet)")
# print(listNet)
# 去除有’ \\n‘, ’ ‘,’\\n’
listNet = [x.strip() for x in listNet]
# print(listNet)
# 去除空字符串''
listNet = [x.strip() for x in listNet if x.strip() != '']
# listNet = list(filter(None, listNet))
print(f"去除空行后数量:len(listNet)")
# print(listNet)
# 去重
listNet = list(set(listNet))
print(f"去重后数量:len(listNet)")
# print(listNet)
with open(r'cfg_new.txt', 'w', encoding='utf-8') as f2:
for i in range(len(listNet)):
listNet[i] = listNet[i] + '\\n'
f2.writelines(listNet)
print(f"数据处理后,已成功写入文件!")
以上是关于利用Python将txt文件批量去除重复行内容的主要内容,如果未能解决你的问题,请参考以下文章