将 RTSP 信号与 FFMPEG 同步
Posted
技术标签:
【中文标题】将 RTSP 信号与 FFMPEG 同步【英文标题】:Synchronizing RTSP signals with FFMPEG 【发布时间】:2018-04-26 08:32:10 【问题描述】:我已经实现将两个 RTSP 信号与 FFMPEG 合并为一个,但在结果流中,第一个输入相对于第二个输入有 2 到 5 秒的延迟。与第二个相比,延迟总是第一个输入。
两个 RTSP 信号来自相同的相机型号、相同的配置、相同的房间......
但是,如果我将相同的 RTSP 信号(两者之一)作为输入 1 和输入 2,则会发生同样的事情。尽管是相同的信号,但第一个输入与第二个输入相比有所延迟。
我怎样才能让它们同步?
这是我执行的命令:
ffmpeg -rtsp_transport tcp -thread_queue_size 512 -rtbufsize 50M -r 15
-i rtsp://XXXX -rtsp_transport tcp -thread_queue_size 512 -rtbufsize 50M -r 15
-c:a aac -i rtsp://YYYY -filter_complex "[0:v]pad=iw*2:ih,setpts=PTS-STARTPTS[bg];
[1:v]setpts=PTS-STARTPTS[fg]; [bg][fg]overlay=w[out]" -map "[out]" -f hls
-hls_time 2 -hls_list_size 5 -use_localtime 1 -use_localtime_mkdir 1
-hls_segment_filename 'LIVE/file-%s.ts' -map a -ar 16000 -ac 1 -ab 64000 -c:a aac
-y output.m3u8
这里有流程信息:
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
[rtsp @ 0x7fe284000e00] Missing PPS in sprop-parameter-sets, ignoring
Input #0, rtsp, from 'rtsp://XXXX':
Metadata:
title : Media Presentation
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1280x720, 15 fps, 15 tbr, 90k tbn, 30 tbc
Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
[rtsp @ 0x7fe28484de00] Missing PPS in sprop-parameter-sets, ignoring
Input #1, rtsp, from 'rtsp://XXXX':
Metadata:
title : Media Presentation
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #1:0: Video: h264 (Main), yuv420p(progressive), 1280x720, 15 fps, 15 tbr, 90k tbn, 30 tbc
Stream #1:1: Audio: aac (LC), 16000 Hz, mono, fltp
Stream mapping:
Stream #0:0 (h264) -> pad (graph 0)
Stream #1:0 (h264) -> setpts (graph 0)
overlay (graph 0) -> Stream #0:0 (libx264)
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0x7fe286802000] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fe286802000] profile High, level 4.0
[libx264 @ 0x7fe286802000] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=1 weightp=2 keyint=250 keyint_min=15 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[hls @ 0x7fe286800000] Opening 'LIVE/file-1524728763.ts' for writing
Output #0, hls, to 'output.m3u8':
Metadata:
title : Media Presentation
encoder : Lavf57.83.100
Stream #0:0: Video: h264 (libx264), yuv420p, 2560x720, q=-1--1, 15 fps, 90k tbn, 15 tbc (default)
Metadata:
encoder : Lavc57.107.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp, 64 kb/s
Metadata:
encoder : Lavc57.107.100 aac
[hls @ 0x7fe286800000] Opening 'LIVE/file-1524728782.ts' for writinged=1.09x
[hls @ 0x7fe286800000] Opening 'output.m3u8.tmp' for writing
[hls @ 0x7fe286800000] Opening 'output.m3u8.tmp' for writing=N/A speed=1.07x
frame= 396 fps= 15 q=-1.0 Lsize=N/A time=00:00:27.00 bitrate=N/A speed=1.05x
video:2946kB audio:147kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
【问题讨论】:
【参考方案1】:来自 ffmpeg 文档 https://ffmpeg.org/ffmpeg-filters.html#Examples-139: 尝试使用
设置视频点setpts='(RTCTIME - RTCSTART) / (TB * 1000000)'
(一些文档说这已被弃用,但我使用的是 11 月 27 日的版本并且它可以工作。)
将视频流同步到第一个流,即
-map "[out]",0:v
并在音频上设置 pts
-af 'asetpts=N/SR/TB'
YMMV,但它适用于我的廉价中国 IP 摄像头
【讨论】:
以上是关于将 RTSP 信号与 FFMPEG 同步的主要内容,如果未能解决你的问题,请参考以下文章
将 wav 和 mp4 与 ffmpeg 合并时,音视频同步漂移缓慢