Android 图形架构 之四——图形缓冲区的申请和消费流程及核心类
Posted 薛瑄
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Android 图形架构 之四——图形缓冲区的申请和消费流程及核心类相关的知识，希望对你有一定的参考价值。
刚才有个朋友问我，博主发生什么事了，给我发了几张截图，我一看，哦，原来是有个大帅哔看了文章，说是，博主，我能白嫖你的文章，我说年轻人，点个赞再走，他说不点，我说点一个，他说不点，我说点一个，他说不点，我说我这文章对你有用，他不服气，说要先看看。我说可以，很快啊，看完后，就是一个复制，一个粘贴，一个网页关闭，我大意了啊，没有删除文章。按传统博客的有用为止，他说已经输了啊。 后来他说他是乱点的，这可不是乱点的啊，训练有素。我劝年轻人好好点赞，耗子尾汁，谢谢朋友们
前言
android应用的UI显示到Display的过程中，SurfaceFlinger扮演的角色只是“Flinger”，就是定于检查Layer更新，然后计算DirtyRegion，然后将结果推送给底层显示驱动进行显示。
应用层把UI内容 写入到GraphicBuffer，SurfaceFlinger读取数据后 ，合成显示。BufferQueue 是用来管理GraphicBuffer的
Android 图形架构 之一 ——概述
Android 图形架构 之二—— SurfaceFlinger 启动和连接
Android 图形架构 之三—— 创建Layer、Surface、SurfaceControl
Android 图形架构 之四——图形缓冲区的申请和消费流程及核心类
Android 图形架构 之五——深入分析addView所发生的的一切
Android 图形架构 之六——深入分析draw()是如何工作的
Android 图形架构 之七——Choreographer 源码分析
Android图形架构 之八——硬件VSync、VSync-app、Vsync-sf
一、生产者消费者模型
对GraphicBuffer的管理 使用的是生产者消费者模型，app 产生数据，通知SurfaceFlinger ，SurfaceFlinger读取数据。
app属于Java层，BufferQueue/SurfaceFlinger属于native层。也就是说BufferQueue也是隶属SurfaceFlinger，所有工作围绕SurfaceFlinger展开。
二、关于缓冲区的核心类
先看一下这几个类的关系
图二
1.1、BufferQueue
生产者、消费者 和 核心服务BufferQueueCore 是由BufferQueue来创建，BufferQueue的createBufferQueue函数，位于frameworks/native/libs/gui/BufferQueue.cpp：
代码一：
void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer,
sp<IGraphicBufferConsumer>* outConsumer,
const sp<IGraphicBufferAlloc>& allocator) //allocator == NULL
......
//创建一个BufferQueueCore，她是核心
sp<BufferQueueCore> core(new BufferQueueCore(allocator));
......
//用BufferQueueCore创建生产者
sp<IGraphicBufferProducer> producer(new BufferQueueProducer(core));
......
//用core创建消费者
sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core));
......
//向外面传入指针赋值
*outProducer = producer;
*outConsumer = consumer;
1.2、 BufferQueueCore
核心都是这个BufferQueueCore，创建生产者 消费者，也使用到了它的对象，他是管理图形缓冲区的中枢。这里举一个SurfaceTexture的例子，来看看他们之间的关系：
可以认为BufferQueueCore是一个服务中心，生产者、消费者都要通过它来管理buffer。下面来看一下，它的构造函数
代码二：
// BufferQueueCore.h
class BufferQueueCore : public virtual RefBase
friend class BufferQueueProducer;
friend class BufferQueueConsumer;
public:
...
typedef Vector<BufferItem> Fifo;
private:
...
String8 mConsumerName;
sp<IConsumerListener> mConsumerListener;
sp<IProducerListener> mConnectedProducerListener;
BufferQueueDefs::SlotsType mSlots;
// mQueue is a FIFO of queued buffers used in synchronous mode.
Fifo mQueue;
// mFreeSlots contains all of the slots which are FREE and do not currently
// have a buffer attached.
std::set<int> mFreeSlots;
// mFreeBuffers contains all of the slots which are FREE and currently have
// a buffer attached.
std::list<int> mFreeBuffers;
// mUnusedSlots contains all slots that are currently unused. They should be
// free and not have a buffer attached.
std::list<int> mUnusedSlots;
// mActiveBuffers contains all slots which have a non-FREE buffer attached.
std::set<int> mActiveBuffers;
...
-
mQueue
是一个新建先出队列，存储了一队 BufferItem 数据，即一组缓存区。 -
mSlots
一个数据类型 是BufferSlot 的数组（通常是64个），每个 BufferSlot对应一个缓存。- BufferSlot 的成员变量：sp< GraphicBuffer >mGraphicBuffer;记录这个slot所涉及的缓冲区
- BufferSlot 的成员变量：mBufferState;用于跟踪这个缓冲区的状态
-
mConsumerListener
通知消费者 ，进行数据的处理。 -
mConnectedProducerListener
当前生产消费模型中的，生产者回调接口。
其余变量的都有注释，就不翻译了
1.3、BufferState
在最新的Android 10 代码中，使用变量来表示缓冲区的各状态的数量
BufferSlot的定义位于frameworks/native/include/gui/BufferSlot.h中：它是一个结构体
代码三：
struct BufferState
// All slots are initially FREE (not dequeued, queued, acquired, or shared).
BufferState()
: mDequeueCount(0),
mQueueCount(0),
mAcquireCount(0),
mShared(false)
uint32_t mDequeueCount;
uint32_t mQueueCount;
uint32_t mAcquireCount;
bool mShared;
// A buffer can be in one of five states, represented as below:
//
// | mShared | mDequeueCount | mQueueCount | mAcquireCount |
// --------|---------|---------------|-------------|---------------|
// FREE | false | 0 | 0 | 0 |
// DEQUEUED| false | 1 | 0 | 0 |
// QUEUED | false | 0 | 1 | 0 |
// ACQUIRED| false | 0 | 0 | 1 |
// SHARED | true | any | any | any |
//
// FREE indicates that the buffer is available to be dequeued by the
// producer. The slot is "owned" by BufferQueue. It transitions to DEQUEUED
// when dequeueBuffer is called.
//
// DEQUEUED indicates that the buffer has been dequeued by the producer, but
// has not yet been queued or canceled. The producer may modify the
// buffer's contents as soon as the associated release fence is signaled.
// The slot is "owned" by the producer. It can transition to QUEUED (via
// queueBuffer or attachBuffer) or back to FREE (via cancelBuffer or
// detachBuffer).
//
// QUEUED indicates that the buffer has been filled by the producer and
// queued for use by the consumer. The buffer contents may continue to be
// modified for a finite time, so the contents must not be accessed until
// the associated fence is signaled. The slot is "owned" by BufferQueue. It
// can transition to ACQUIRED (via acquireBuffer) or to FREE (if another
// buffer is queued in asynchronous mode).
//
// ACQUIRED indicates that the buffer has been acquired by the consumer. As
// with QUEUED, the contents must not be accessed by the consumer until the
// acquire fence is signaled. The slot is "owned" by the consumer. It
// transitions to FREE when releaseBuffer (or detachBuffer) is called. A
// detached buffer can also enter the ACQUIRED state via attachBuffer.
//
// SHARED indicates that this buffer is being used in shared buffer
// mode. It can be in any combination of the other states at the same time,
// except for FREE (since that excludes being in any other state). It can
// also be dequeued, queued, or acquired multiple times.
;
官方文档已经说得很清楚了，这里再啰嗦一下：
- FREE：buffer当前可用，可以被生产者dequeued，此时owner是BufferQueueCore，当 dequeuebuffer调用时，状态可以转为dequeued。
- DEQUEUED：buffer已经被dequeued，还没被queue或canceld，此时owner是producer。
- QUEUED：buffer已经被生产者填充，并被queued，此时的owner是bufferQueueCore。
- ACQUIRED：buffer已经被消费者获得，此时的owner是consumer。
一般的buffer大致会经过FREE->DEQUEUED->QUEUED->ACQUIRED->FREE这个流程，涉及到的函数，下面会讲到，这些函数，通常在systrace 性能分析的时候，会经常看到，是有标志性的意义，你可能需要寻找这几个关键函数的，来判断是否丢帧、卡顿的原因
1.4、BufferSlot
代码四：
struct BufferSlot
BufferSlot()
: mGraphicBuffer(nullptr),
mEglDisplay(EGL_NO_DISPLAY),
mBufferState(),
mRequestBufferCalled(false),
mFrameNumber(0),
mEglFence(EGL_NO_SYNC_KHR),
mFence(Fence::NO_FENCE),
mAcquireCalled(false),
mNeedsReallocation(false)
// mGraphicBuffer points to the buffer allocated for this slot or is NULL
// if no buffer has been allocated.
sp<GraphicBuffer> mGraphicBuffer;
// mEglDisplay is the EGLDisplay used to create EGLSyncKHR objects.
EGLDisplay mEglDisplay;
// mBufferState is the current state of this buffer slot.
BufferState mBufferState;
;
- mGraphicBuffer代表一块图形缓冲区GraphicBuffer，用于应用绘制UI。
- mBufferState类型为BufferState
接下来，我们分别从生产者 消费者的角度来分析
三、生产者
IGraphicBufferProducer/IProducerListener 生产者
IGraphicBufferProducer 是生产者接口，实现了 IInterface 可以用于跨进程通信。
代码五：
// IGraphicBufferProducer.h
class IGraphicBufferProducer : public IInterface
public:
DECLARE_HYBRID_META_INTERFACE(GraphicBufferProducer,
HGraphicBufferProducer)
...
// 根据指定参数申请一块 Buffer ，索引值为 slot ，同步为 fence
// 从 BufferQueue 中出队一块缓存 GraphicBuffer
virtual status_t dequeueBuffer(int* slot, sp<Fence>* fence,
uint32_t w, uint32_t h,
PixelFormat format, uint64_t usage, uint64_t* outBufferAge,
FrameEventHistoryDelta* outTimestamps) = 0;
// 获取 slot 位置的 GraphicBuffer
virtual status_t requestBuffer(int slot, sp<GraphicBuffer>* buf) = 0;
// 客户端已经向 slot 位置的 Buffer 填充完数据
// IGraphicBufferProducer 得到 Buffer 的输入信息，
// slot 这块缓存 GraphicBuffer 进入队列 BufferQueue
virtual status_t queueBuffer(int slot, const QueueBufferInput& input,
QueueBufferOutput* output) = 0;
// 释放 slot 位置的 GraphicBuffer
virtual status_t detachBuffer(int slot) = 0;
virtual status_t detachNextBuffer(sp<GraphicBuffer>* outBuffer,
sp<Fence>* outFence) = 0;
// 根据指定的 buffer 获取 slot
virtual status_t attachBuffer(int* outSlot,
const sp<GraphicBuffer>& buffer) = 0;
// 释放 slot 位置的 buffer
virtual status_t cancelBuffer(int slot, const sp<Fence>& fence) = 0;
...
// 客户端根据 api 类型，连接 IGraphicBufferProducer ，
// 客户端得到缓存的相关信息 QueueBufferOutput
virtual status_t connect(const sp<IProducerListener>& listener,
int api, bool producerControlledByApp,
QueueBufferOutput* output) = 0;
// 断开连接
virtual status_t disconnect(int api,
DisconnectMode mode = DisconnectMode::Api) = 0;
// 获取消费者名称
virtual String8 getConsumerName() const = 0;
...
;
IProducerListener 是 IGraphicBufferProducer 对应的回调接口。
代码六：
// IProducerListener.h
class ProducerListener : public virtual RefBase
public:
ProducerListener()
virtual ~ProducerListener();
virtual void onBufferReleased() = 0; // Asynchronous
virtual bool needsReleaseNotify() = 0;
;
class IProducerListener : public ProducerListener, public IInterface
public:
DECLARE_META_INTERFACE(ProducerListener)
;
class BnProducerListener : public BnInterface<IProducerListener>
public:
virtual status_t onTransact(uint32_t code, const Parcel& data,
Parcel* reply, uint32_t flags = 0);
virtual bool needsReleaseNotify();
;
class DummyProducerListener : public BnProducerListener
public:
virtual ~DummyProducerListener();
virtual void onBufferReleased()
virtual bool needsReleaseNotify() return false;
;
BufferQueueProducer 生产者实现类
BufferQueueProducer 是 IGraphicBufferConsumer 的实现类，实现了生产者对应的功能。
代码七：
// BufferQueueProducer.h
class BufferQueueProducer : public BnGraphicBufferProducer,
private IBinder::DeathRecipient
...
private:
...
sp<BufferQueueCore> mCore;
// This references mCore->mSlots.
BufferQueueDefs::SlotsType& mSlots;
String8 mConsumerName;
...
sp<Fence> mLastQueueBufferFence;
Rect mLastQueuedCrop;
uint32_t mLastQueuedTransform;
BufferQueueProducer 中持有 BufferQueueCore 对象； mSlots 指向 mCore->mSlots ；同时保持了生产消费模型中，对应消费者的名称。
四、消费者
IGraphicBufferConsumer/IConsumerListener 消费者
IGraphicBufferConsumer 是消费者接口，实现了 IInterface 可以用于跨进程通信。
代码八：
// IGraphicBufferConsumer.h
class IGraphicBufferConsumer : public IInterface
public:
DECLARE_META_INTERFACE(GraphicBufferConsumer)
...
// 从 BufferQueue 中获取一块准备好了的缓存 GraphicBuffer
virtual status_t acquireBuffer(BufferItem* buffer,
nsecs_t presentWhen, uint64_t maxFrameNumber = 0) = 0;
// 释放 slot 位置的缓存
virtual status_t detachBuffer(int slot) = 0;
// 根据指定的 GraphicBuffer 获取 slot
virtual status_t attachBuffer(int* outSlot,
const sp<GraphicBuffer>& buffer) = 0;
// 移除指定 slot 位置的缓存
virtual status_t releaseBuffer(int buf, uint64_t frameNumber,
EGLDisplay display, EGLSyncKHR fence,
const sp<Fence>& releaseFence) = 0;
...
// 连接一个消费者进入 BufferQueue
virtual status_t consumerConnect(const sp<IConsumerListener>& consumer,
bool controlledByApp) = 0;
// 从 BufferQueue 断开连接
virtual status_t consumerDisconnect() = 0;
...
;
IConsumerListener 是 IGraphicBufferConsumer 对应的回调接口：
代码九：
// IConsumerListener.h
class ConsumerListener : public virtual RefBase
public:
ConsumerListener()
virtual ~ConsumerListener();
// onDisconnect is called when a producer disconnects from the BufferQueue.
virtual void onDisconnect() /* Asynchronous */
// onFrameAvailable is called from queueBuffer each time an additional frame becomes available
// for consumption. This means that frames that are queued while in asynchronous mode only
// trigger the callback if no previous frames are pending. Frames queued while in synchronous
// mode always trigger the callback. The item passed to the callback will contain all of the
// information about the queued frame except for its GraphicBuffer pointer, which will always be
// null (except if the consumer is SurfaceFlinger).
//
// This is called without any lock held and can be called concurrently by multiple threads.
virtual void onFrameAvailable(const BufferItem& item) = 0; /* Asynchronous */
// onFrameReplaced is called from queueBuffer if the frame being queued is replacing an existing
// slot in the queue. Any call to queueBuffer that doesn't call onFrameAvailable will call this
// callback instead. The item passed to the callback will contain all of the information about
// the queued frame except for its GraphicBuffer pointer, which will always be null.
//
// This is called without any lock held and can be called concurrently by multiple threads.
virtual void onFrameReplaced(const BufferItem& /* item */) /* Asynchronous */
// onBuffersReleased is called to notify the buffer consumer that the BufferQueue has released
// its references to one or more GraphicBuffers contained in its slots. The buffer consumer
// should then call BufferQueue::getReleasedBuffers to retrieve the list of buffers.
//
// This is called without any lock held and can be called concurrently by multiple threads.
virtual void onBuffersReleased() = 0; /* Asynchronous */
// onSidebandStreamChanged is called to notify the buffer consumer that the BufferQueue's
// sideband buffer stream has changed. This is called when a stream is first attached and when
// it is either detached or replaced by a different stream.
virtual void onSidebandStreamChanged() = 0; /* Asynchronous */
// Notifies the consumer of any new producer-side timestamps and returns the combined frame
// history that hasn't already been retrieved.
//
// WARNING: This method can only be called when the BufferQueue is in the consumer's process.
virtual void addAndGetFrameTimestamps(const NewFrameEventsEntry* /*newTimestamps*/,
FrameEventHistoryDelta* /*outDelta*/)
;
class IConsumerListener : public ConsumerListener, public IInterface
public:
DECLARE_META_INTERFACE(ConsumerListener)
;
class BnConsumerListener : public SafeBnInterface<IConsumerListener>
public:
BnConsumerListener() : SafeBnInterface<IConsumerListener>("BnConsumerListener")
status_t onTransact(uint32_t code, const Parcel& data, Parcel* reply,
uint32_t flags = 0) override;
;
BufferQueueConsumer 是 IGraphicBufferConsumer 的实现类，实现了消费者对应的功能。是一个回调接口，如果BufferQueue中有数据准备好了通知消费者取走数据。取走数据的时候，需要调用acquireBuffer函数，将缓冲区状态变成ACQUIRED，使用完之后调用releaseBuffer函数可以把缓冲区数据归还给BufferQueueCore，这样缓冲区就变成FREE。
代码十：
// BufferQueueConsumer.h
class BufferQueueConsumer : public BnGraphicBufferConsumer
...
private:
sp<BufferQueueCore> mCore;
// This references mCore->mSlots.
BufferQueueDefs::SlotsType& mSlots;
String8 mConsumerName;
BufferQueueConsumer 中持有 BufferQueueCore 对象； mSlots 指向 mCore->mSlots 。
五、图形缓冲区的生产和消费
这里依然使用Android 图形架构 之一 ——概述 中图片
5.1、申请图形缓冲区
申请图形缓冲区，是从队列中取出数据 ，所以函数名叫dequeueBuffer ，它的作用是在应用程序一端请求绘制图像时，向BufferQueue中申请一块可用的GraphicBuffer，有了这个buffer就可以绘制图像数据了。
bufferqueuecore中mSlots数组管理缓冲区，最大容量是64，这个mSlots一开始静态分配了64个bufferslot大小的空间，但是其中的数据缓冲区GraphicBuffer的分配是动态的
在UI绘制的draw阶段会调用 dequeueBuffer，详见文章Android 图形架构 之六——深入分析draw()是如何工作的
代码路径：frameworks/native/libs/gui/BufferQueueProducer.cpp
代码十一：
status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
uint32_t width, uint32_t height, PixelFormat format,
uint64_t usage, uint64_t* outBufferAge,
FrameEventHistoryDelta* outTimestamps)
ATRACE_CALL();
// Autolock scope
std::lock_guard<std::mutex> lock(mCore->mMutex);
mConsumerName = mCore->mConsumerName;
// 队列的状态为被抛弃的，就直接返回了
if (mCore->mIsAbandoned)
BQ_LOGE("dequeueBuffer: BufferQueue has been abandoned");
return NO_INIT;
//是否与BufferQueue建立连接
if (mCore->mConnectedApi == BufferQueueCore::NO_CONNECTED_API)
BQ_LOGE("dequeueBuffer: BufferQueue has no connected producer");
return NO_INIT;
// Autolock scope
BQ_LOGV("dequeueBuffer: w=%u h=%u format=%#x, usage=%#" PRIx64, width, height, format, usage);
//宽高不正常，则返回失败
if ((width && !height) || (!width && height))
BQ_LOGE("dequeueBuffer: invalid size: w=%u h=%u", width, height);
return BAD_VALUE;
//定义返回值
status_t returnFlags = NO_ERROR;
EGLDisplay eglDisplay = EGL_NO_DISPLAY;
EGLSyncKHR eglFence = EGL_NO_SYNC_KHR;
bool attachedByConsumer = false;
// Autolock scope
std::unique_lock<std::mutex> lock(mCore->mMutex);
// If we don't have a free buffer, but we are currently allocating, we wait until allocation
// is finished such that we don't allocate in parallel.
// mFreeBuffers 表示 状态为FREE，其已经绑定了GraphicBuffer的BufferSlot 的集合
// 这种集合为空，并且 正在 为GraphicBuffer分配内存空间
if (mCore->mFreeBuffers.empty() && mCore->mIsAllocating)
mDequeueWaitingForAllocation = true;
//等待分配完成
mCore->waitWhileAllocatingLocked(lock);
mDequeueWaitingForAllocation = false;
mDequeueWaitingForAllocationCondition.notify_all();
if (format == 0)
format = mCore->mDefaultBufferFormat;
// Enable the usage bits the consumer requested
usage |= mCore->mConsumerUsageBits;
const bool useDefaultSize = !width && !height;
if (useDefaultSize)
width = mCore->mDefaultWidth;
height = mCore->mDefaultHeight;
//寻找 BufferSlot
int found = BufferItem::INVALID_BUFFER_SLOT;
while (found == BufferItem::INVALID_BUFFER_SLOT)
//获取到可用的BufferSlot，它在mSlots的下标保存在found中
status_t status = waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue, lock, &found);
if (status != NO_ERROR)
return status;
// found 值没有发生变化，说明创建失败
if (found == BufferQueueCore::INVALID_BUFFER_SLOT)
BQ_LOGE("dequeueBuffer: no available buffer slots");
return -EBUSY;
//获取found 索引以上是关于Android 图形架构 之四——图形缓冲区的申请和消费流程及核心类的主要内容，如果未能解决你的问题，请参考以下文章
Android 图形架构 之七——Choreographer 源码分析
Android 图形架构 之七——Choreographer 源码分析
Android 图形架构 之二—— SurfaceFlinger 启动和连接