了解GStreamer
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了了解GStreamer相关的知识，希望对你有一定的参考价值。
参考技术A GStreamer是一个基于glib实现的跨平台多媒体框架。以下是架构图：GStreamer实现了多媒体处理的框架，主要是一个管道（pipline）包含多个元件（element）组成。管道继承自容器（bin），容器可以方便管理元件，用于串联元件，处理修改元件的状态，生命周期等。管道会提供一个消息总线（bus），用于元件的状态同步。而元件会包含一个或多个垫片（pad），用于数据的源（src）和槽（sink），每个数据源鱼一个数据槽相连接。这样多个元件在管道中就形成了复杂的处理流程了。
首先初始化GStreamer库，库前缀为 gst_ ，glib前缀为 g_ 。
然后创建管道
创建对应的元件，采用了工厂模式
连接元件
整个管道就创建好了，然后我们就让管道开始工作
最后，一直等待管道完成工作
完成我们要的功能后，释放整个管道，防止资源泄露。整个管道可以有容器统一处理，所以不需要每个元件都去释放。同时glib实现的GObject是用引用计数实现的资源回收，而且GstElement继承自GInitiallyUnowned，所以元件默认是没有人引用的。容器可以完美的释放元件。
以上就是一个基本的GStreamer项目。接下来我们看看GStreamer有哪些常用元件。
基本流程摘抄自 这里
常用元件摘抄自 这里 .
GStreamer系列之ubuntu18上安装使用
文章目录
前言
网上教程看的我一脸懵逼，可能因为有新旧版本的问题，总之，经过一番了解，现在整理一版自己能看懂的gstreamer的资料。我的使用的是ubuntu18 阿里云镜像直接apt安装的，版本为gstreamer1.14
正文
参考文档
照目前看来，gstreamer是由一个个插件，中间使用管道连接起来使用的。用好gstreamer的关键就是了解它各个插件的用法，当然最好还是详细看下gstreamer的底层机制。参考https://www.cnblogs.com/xleng/p/10948838.html
ubuntu18上安装
安装前置
sudo apt update
sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio
【问题1】Err:2 http://cn.archive.ubuntu.com/ubuntu bionic/universe amd64 libfaad2 amd64 2.8.8-1
【解决方案】连接失败，把apt更换国内源，比如ali
$ sudo su - # 切换到 root 目录
# cd /etc/apt/
# mv sources.list sources.list.bak
# vi /etc/apt/sources.list # 将下面的源复制到文件中, 保存退出
# apt update # 更新本地 apt 源
deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
查看安装情况
dpkg -l | grep gstreamer
验证
参考官方文档
$ gst-inspect-1.0 fakesrc
$ gst-launch-1.0 -v fakesrc silent=false num-buffers=3 ! fakesink silent=false
$ gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
ps: If autovideosink doesn’t work, try an element that’s specific for your operating system and windowing system, such as ximagesink or glimagesink or (on windows) d3dvideosink.
gstreamer的使用
1 查看当前插件
$ gst-inspect-1.0
coretracers: latency (GstTracerFactory)
coretracers: log (GstTracerFactory)
coretracers: rusage (GstTracerFactory)
coretracers: stats (GstTracerFactory)
coretracers: leaks (GstTracerFactory)
videorate: videorate: Video rate adjuster
videotestsrc: videotestsrc: Video test source
adder: adder: Adder
tcp: socketsrc: socket source
tcp: tcpclientsink: TCP client sink
tcp: tcpclientsrc: TCP client source
tcp: tcpserversink: TCP server sink
tcp: tcpserversrc: TCP server source
tcp: multifdsink: Multi filedescriptor sink
tcp: multisocketsink: Multi socket sink
audiomixer: audiomixer: AudioMixer
audiomixer: liveadder: AudioMixer
audiomixer: audiointerleave: AudioInterleave
audiotestsrc: audiotestsrc: Audio test source
videoscale: videoscale: Video scaler
ximagesink: ximagesink: Video sink
gio: giosink: GIO sink
gio: giosrc: GIO source
gio: giostreamsink: GIO stream sink
gio: giostreamsrc: GIO stream source
volume: volume: Volume
audioresample: audioresample: Audio resampler
videoconvert: videoconvert: Colorspace converter
typefindfunctions: video/x-ms-asf: asf, wm, wma, wmv
typefindfunctions: audio/x-musepack: mpc, mpp, mp+
typefindfunctions: audio/x-au: au, snd
typefindfunctions: video/x-msvideo: avi
typefindfunctions: audio/qcelp: qcp
typefindfunctions: video/x-cdxa: dat
typefindfunctions: video/x-vcd: dat
typefindfunctions: audio/x-imelody: imy, ime, imelody
typefindfunctions: audio/midi: mid, midi
typefindfunctions: audio/riff-midi: mid, midi
typefindfunctions: audio/mobile-xmf: mxmf
typefindfunctions: video/x-fli: flc, fli
typefindfunctions: application/x-id3v2: mp3, mp2, mp1, mpga, ogg, flac, tta
typefindfunctions: application/x-id3v1: mp3, mp2, mp1, mpga, ogg, flac, tta
typefindfunctions: application/x-apetag: mp3, ape, mpc, wv
typefindfunctions: audio/x-ttafile: tta
typefindfunctions: audio/x-mod: 669, amf, ams, dbm, digi, dmf, dsm, gdm, far, imf, it, j2b, mdl, med, mod, mt2, mtm, okt, psm, ptm, sam, s3m, stm, stx, ult, umx, xm
typefindfunctions: audio/mpeg: mp3, mp2, mp1, mpga
typefindfunctions: audio/x-ac3: ac3, eac3
typefindfunctions: audio/x-dts: dts
typefindfunctions: audio/x-gsm: gsm
typefindfunctions: video/mpeg-sys: mpe, mpeg, mpg
typefindfunctions: video/mpegts: ts, mts
typefindfunctions: application/ogg: ogg, oga, ogv, ogm, ogx, spx, anx, axa, axv
typefindfunctions: video/mpeg-elementary: mpv, mpeg, mpg
typefindfunctions: video/mpeg4: m4v
typefindfunctions: video/x-h263: h263, 263
typefindfunctions: video/x-h264: h264, x264, 264
typefindfunctions: video/x-h265: h265, x265, 265
typefindfunctions: video/x-nuv: nuv
typefindfunctions: audio/x-m4a: m4a
typefindfunctions: application/x-3gp: 3gp
typefindfunctions: video/quicktime: mov, mp4
typefindfunctions: image/x-quicktime: qif, qtif, qti
typefindfunctions: image/jp2: jp2
typefindfunctions: image/x-jpc: jpc, j2k
typefindfunctions: video/mj2: mj2
typefindfunctions: text/html: htm, html
typefindfunctions: application/vnd.rn-realmedia: ra, ram, rm, rmvb
typefindfunctions: application/x-pn-realaudio: ra, ram, rm, rmvb
typefindfunctions: application/x-shockwave-flash: swf, swfl
typefindfunctions: application/dash+xml: mpd, MPD
typefindfunctions: application/vnd.ms-sstr+xml: no extensions
typefindfunctions: video/x-flv: flv
typefindfunctions: text/plain: txt
typefindfunctions: text/utf-16: txt
typefindfunctions: text/utf-32: txt
typefindfunctions: text/uri-list: ram
typefindfunctions: application/itc: itc
typefindfunctions: application/x-hls: m3u8
typefindfunctions: application/sdp: sdp
typefindfunctions: application/smil: smil
typefindfunctions: application/ttml+xml: ttml+xml
typefindfunctions: application/xml: xml
typefindfunctions: audio/x-wav: wav
typefindfunctions: audio/x-aiff: aiff, aif, aifc
typefindfunctions: audio/x-svx: iff, svx
typefindfunctions: audio/x-paris: paf
typefindfunctions: audio/x-nist: nist
typefindfunctions: audio/x-voc: voc
typefindfunctions: audio/x-sds: sds
typefindfunctions: audio/x-ircam: sf
typefindfunctions: audio/x-w64: w64
typefindfunctions: audio/x-rf64: rf64
typefindfunctions: audio/x-shorten: shn
typefindfunctions: application/x-ape: ape
typefindfunctions: image/jpeg: jpg, jpe, jpeg
typefindfunctions: image/gif: gif
typefindfunctions: image/png: png
typefindfunctions: image/bmp: bmp
typefindfunctions: image/tiff: tif, tiff
typefindfunctions: image/webp: webp
typefindfunctions: image/x-exr: exr
typefindfunctions: image/x-portable-pixmap: pnm, ppm, pgm, pbm
typefindfunctions: video/x-matroska: mkv, mka, mk3d, webm
typefindfunctions: application/mxf: mxf
typefindfunctions: video/x-mve: mve
typefindfunctions: video/x-dv: dv, dif
typefindfunctions: audio/x-amr-nb-sh: amr
typefindfunctions: audio/x-amr-wb-sh: amr
typefindfunctions: audio/iLBC-sh: ilbc
typefindfunctions: audio/x-sbc: sbc
typefindfunctions: audio/x-sid: sid
typefindfunctions: image/x-xcf: xcf
typefindfunctions: video/x-mng: mng
typefindfunctions: image/x-jng: jng
typefindfunctions: image/x-xpixmap: xpm
typefindfunctions: image/x-sun-raster: ras
typefindfunctions: application/x-bzip: bz2
typefindfunctions: application/x-gzip: gz
typefindfunctions: application/zip: zip
typefindfunctions: application/x-compress: Z
typefindfunctions: subtitle/x-kate: no extensions
typefindfunctions: application/x-subtitle-vtt: vtt
typefindfunctions: audio/x-flac: flac
typefindfunctions: audio/x-vorbis: no extensions
typefindfunctions: video/x-theora: no extensions
typefindfunctions: application/x-ogm-video: no extensions
typefindfunctions: application/x-ogm-audio: no extensions
typefindfunctions: application/x-ogm-text: no extensions
typefindfunctions: audio/x-speex: no extensions
typefindfunctions: audio/x-celt: no extensions
typefindfunctions: application/x-ogg-skeleton: no extensions
typefindfunctions: text/x-cmml: no extensions
typefindfunctions: application/x-executable: no extensions
typefindfunctions: audio/aac: aac, adts, adif, loas
typefindfunctions: audio/x-spc: spc
typefindfunctions: audio/x-wavpack: wv, wvp
typefindfunctions: audio/x-wavpack-correction: wvc
typefindfunctions: audio/x-caf: caf
typefindfunctions: application/postscript: ps
typefindfunctions: image/svg+xml: svg
typefindfunctions: application/x-rar: rar
typefindfunctions: application/x-tar: tar
typefindfunctions: application/x-ar: a
typefindfunctions: application/x-ms-dos-executable: dll, exe, ocx, sys, scr, msstyles, cpl
typefindfunctions: video/x-dirac: no extensions
typefindfunctions: multipart/x-mixed-replace: no extensions
typefindfunctions: application/x-mmsh: no extensions
typefindfunctions: video/vivo: viv
typefindfunctions: audio/x-nsf: nsf
typefindfunctions: audio/x-gym: gym
typefindfunctions: audio/x-ay: ay
typefindfunctions: audio/x-gbs: gbs
typefindfunctions: audio/x-vgm: vgm
typefindfunctions: audio/x-sap: sap
typefindfunctions: video/x-ivf: ivf
typefindfunctions: audio/x-kss: kss
typefindfunctions: application/pdf: pdf
typefindfunctions: application/msword: doc
typefindfunctions: image/vnd.adobe.photoshop: psd
typefindfunctions: image/vnd.wap.wbmp: no extensions
typefindfunctions: application/x-yuv4mpeg: no extensions
typefindfunctions: image/x-icon: no extensions
typefindfunctions: image/x-degas: no extensions
typefindfunctions: application/octet-stream: no extensions
typefindfunctions: application/x-ssa: ssa, ass
typefindfunctions: video/x-pva: pva
typefindfunctions: audio/x-xi: xi
typefindfunctions: audio/audible: aa, aax
audioconvert: audioconvert: Audio converter
subparse: subparse_typefind: srt, sub, mpsub, mdvd, smi, txt, dks, vtt
subparse: subparse: Subtitle parser
subparse: ssaparse: SSA Subtitle Parser
playback: playbin: Player Bin 2
playback: playbin3: Player Bin 3
playback: playsink: Player Sink
playback: subtitleoverlay: Subtitle Overlay
playback: streamsynchronizer: Stream Synchronizer
playback: decodebin: Decoder Bin
playback: decodebin3: Decoder Bin 3
playback: uridecodebin: URI Decoder
playback: uridecodebin3: URI Decoder
playback: urisourcebin: URI reader
playback: parsebin: Parse Bin
pbtypes: GstVideoMultiviewFlagsSet (GstDynamicTypeFactory)
rawparse: unalignedaudioparse: unalignedaudioparse
rawparse: unalignedvideoparse: unalignedvideoparse
rawparse: rawaudioparse: rawaudioparse
rawparse: rawvideoparse: rawvideoparse
encoding: encodebin: Encoder Bin
app: appsrc: AppSrc
app: appsink: AppSink
audiorate: audiorate: Audio rate adjuster
coreelements: streamiddemux: Streamid Demux
coreelements: valve: Valve element
coreelements: multiqueue: MultiQueue
coreelements: typefind: TypeFind
coreelements: tee: Tee pipe fitting
coreelements: filesink: File Sink
coreelements: queue2: Queue 2
coreelements: queue: Queue
coreelements: output-selector: Output selector
coreelements: input-selector: Input selector
coreelements: identity: Identity
coreelements: funnel: Funnel pipe fitting
coreelements: filesrc: File Source
coreelements: fdsink: Filedescriptor Sink
coreelements: fdsrc: Filedescriptor Source
coreelements: fakesink: Fake Sink
coreelements: fakesrc: Fake Source
coreelements: downloadbuffer: DownloadBuffer
coreelements: dataurisrc: data: URI source element
coreelements: concat: Concat
coreelements: capsfilter: CapsFilter
opengl: glfilterglass: OpenGL glass filter
opengl: gldeinterlace: OpenGL deinterlacing filter
opengl: gltestsrc: Video test source
opengl: glstereosplit: GLStereoSplit
opengl: glviewconvert: OpenGL Multiview/3D conversion filter
opengl: glfilterapp: OpenGL application filter
opengl: glshader: OpenGL fragment shader filter
opengl: glcolorscale: OpenGL color scale
opengl: gleffects_laplacian: Laplacian Convolution Demo Effect
opengl: gleffects_blur: Blur with 9x9 separable convolution Effect
opengl: gleffects_sobel: Sobel edge detection Effect
opengl: gleffects_glow: Glow Lighting Effect
opengl: gleffects_sin: All Grey but Red Effect
opengl: gleffects_xray: Glowing negative effect
opengl: gleffects_lumaxpro: Luma Cross Processing Effect
opengl: gleffects_xpro: Cross Processing Effect
opengl: gleffects_sepia: Sepia Toning Effect
opengl: gleffects_heat: Heat Signature Effect
opengl: gleffects_square: Square Effect
opengl: gleffects_bulge: Bulge Effect
opengl: gleffects_twirl: Twirl Effect
opengl: gleffects_fisheye: FishEye Effect
opengl: gleffects_tunnel: Light Tunnel Effect
opengl: gleffects_stretch: Stretch Effect
opengl: gleffects_squeeze: Squeeze Effect
opengl: gleffects_mirror: Mirror Effect
opengl: gleffects_identity: Do nothing Effect
opengl: gleffects: Gstreamer OpenGL Effects
opengl: glfiltercube: OpenGL cube filter
opengl: glsrcbin: GL Src Bin
opengl: glsinkbin: GL Sink Bin
opengl: glfilterbin: GL Filter Bin
opengl: glcolorbalance: Video balance
opengl: glcolorconvert: OpenGL color converter
opengl: gldownload: OpenGL downloader
opengl: glupload: OpenGL uploader
opengl: glimagesinkelement: OpenGL video sink
opengl: glimagesink: GL Sink Bin
staticelements: bin: Generic bin
staticelements: pipeline: Pipeline object
Total count: 25 plugins, 253 features
ps: 25个插件指的是 coretracers， videorate，videotestsrc， adder，tcp，audiomixer，audiotestsrc，videoscale，ximagesink，gio，volume，audioresample，videoconvert，typefindfunctions，audioconvert，subparse，playback，pbtypes，rawparse，encoding，app，audiorate，coreelements，opengl，staticelements
各插件用法参考官方文档 https://gstreamer.freedesktop.org/documentation/plugins_doc.html?gi-language=c
同时可以看到还有很多插件这里并未安装
当然也可以直接查看某个插件是否存在
# 查看nvidia硬件解码器是否存在
$ gst-inspect-1.0 nvdec
No such element or plugin 'nvdec'
2 了解如何使用
官方提供了比较丰富的tutorials，有耐心的可以一个个试试
这里是中文教程
这里介绍了tools的用法，包含了gst-launch-1.0, gst-inspect-1.0 and gst-discoverer-1.0 Basic tutorial 10: GStreamer tools
这里介绍了好用的元素，比如bin Basic tutorial 14: Handy elements
3 编译安装
由于apt获取安装的gstreamer少了许多想要的功能，这里采用编译安装
源代码 git仓库
以上是关于了解GStreamer的主要内容，如果未能解决你的问题，请参考以下文章