求助sed命令匹配并替换无效的换行符

Posted 2023-04-19

目前有这样一个文本信息：
2019-03-13 11:00:33.403777 CST|gpadmin|template1|p21310|th687359968|::1|27676|2019-03-13 11:00:33 CST|0||cmd7|seg-1||||sx1|LOG: |00000|statement: SELECT typname, typlen FROM pg_type WHERE oid=25||||||SELECT typname, typlen FROM pg_type WHERE oid=25|0||postgres.c|1616|
2019-03-13 11:00:33.407718 CST|gpadmin|template1|p21310|th687359968|::1|27676|2019-03-13 11:00:33 CST|0||cmd9|seg-1||||sx1|LOG: |00000|statement:
SELECT dbid, content, role, preferred_role, mode, status,
hostname, address, port, datadir
FROM pg_catalog.gp_segment_configuration
ORDER BY content, preferred_role DESC||||||
SELECT dbid, content, role, preferred_role, mode, status,
hostname, address, port, datadir
FROM pg_catalog.gp_segment_configuration
ORDER BY content, preferred_role DESC
|0||postgres.c|1616|
2019-03-13 11:00:33.409371 CST|gpadmin|template1|p21310|th687359968|::1|27676|2019-03-13 11:00:33 CST|0||cmd11|seg-1||||sx1|LOG: |00000|statement: SELECT typname, typlen FROM pg_type WHERE oid=21||||||SELECT typname, typlen FROM pg_type WHERE oid=21|0||postgres.c|1616|
2019-03-13 11:00:34.082752 CST|||p21298|th687359968||||0|||seg-1|||||LOG: |00000|database system is shut down|||||||0||xlog.c|8245|
2019-03-13 23:59:51.858271 CST|||p21455|th877815776||||0|||seg-1|||||LOG: |00000|3rd party error log:
line=||||||||SysLoggerMain|syslogger.c|705|
2019-03-13 23:59:51.858300 CST|||p21455|th877815776||||0|||seg-1|||||LOG: |00000|3rd party error log:
164||||||||SysLoggerMain|syslogger.c|705|
2019-03-13 23:59:51.858345 CST|||p21455|th877815776||||0|||seg-1|||||LOG: |00000|3rd party error log:||||||||SysLoggerMain|syslogger.c|705|
2019-03-13 23:59:51.858479 CST|gpmon|gpperfmon|p18776|th877815776|[local]||2019-03-13 23:59:51 CST|0|con8378|cmd1|seg-1||dx16920||sx1|LOG: |00000|Planner produced plan :0||||||insert into segment_history select * from _segment_tail;|0||orca.c|58|

这是一段日志文件，目前因为其中的非法换行问题，导致无法解析；现在想通过sed把无效的换行去掉，条件是每行以“2019-03-13 23:59:51.858300 CST”这种日期格式开头的认为是有效记录的开始，不是这个格式开头的，认为是之前的行没结束而换行导致的。

通过sed处理后，希望可以得到内容，每行都是以“2019-03-13 23:59:51.858300 CST”日期格式开头的完整行，不再有无效换行；
谢谢各位大侠了。

参考技术A sed ':a; N; s/\n//; ta' filename | sed 's/\([0-9]\4\-[0-9]\2\-[0-9]\2\ [0-9]\2\:[0-9]\2\:[0-9]\2\\.[0-9]\6\ CST\)/\n\1/g' > new_file
如果直接修改文件的话:
sed -i ':a; N; s/\n//; ta' filename && sed -i 's/\([0-9]\4\-[0-9]\2\-[0-9]\2\ [0-9]\2\:[0-9]\2\:[0-9]\2\\.[0-9]\6\ CST\)/\n\1/g' filename

sed命令:删除匹配行和替换

删除以a开头的行

sed -i ‘/^a.*/d‘ tmp.txt

-i 表示操作在源文件上生效.否则操作内存中数据,并不写入文件中.
在分号内的/d表示删除匹配的行

 

替换匹配行:

sed -i ‘s/^a.*/haha/g‘ tmp.txt

分号内的s/表示替换
/g表示全局替换