Camera1 源码解析系列—— Camera1 takePicture() 流程解析

Posted 2022-11-24 ByteSaid

前言

前面几篇文章已经把 Camera 控制流的部分梳理得比较清楚了。在 Camera 流程中，还有一个重要的部分，即数据流。
Camera API 1 中，数据流主要是通过函数回调的方式，依照从下往上的方向，逐层 return 到 Applications 中。
本篇将数据流与 Camera 的控制流结合起来，从 takePicture() 方法切入，追踪一个比较完整的 Camera 流程。

1 Open 流程

Camera Open 的流程，在之前的一篇文章中已经比较详细地描述了。在这里，再关注一下这个流程中 Libraries 和 HAL 层的部分。

1.1 CameraService.cpp

• 路径：frameworks/av/services/camera/libcameraservice/CameraService.cpp
• connect()：
  • 注意这里真正实现逻辑是在 connectHelper() 函数中。
• connectHelper()：
  • 这个函数实现比较长，截取其中的一段。
  • 首先，如果客户端实例已经存在于 MediaRecorder ，则直接将其取出返回。
  • 若不存在，则先获取 deviceVersion，然后再调用 makeClient() 函数创建一个客户端。

template<class CALLBACK, class CLIENT>
Status CameraService::connectHelper(const sp<CALLBACK>& cameraCb, const String8& cameraId,
        int api1CameraId, int halVersion, const String16& clientPackageName, int clientUid,
        int clientPid, apiLevel effectiveApiLevel, bool legacyMode, bool shimUpdateOnly,
        /*out*/sp<CLIENT>& device) 
    binder::Status ret = binder::Status::ok();
    ...
        sp<BasicClient> tmp = nullptr;
        if(!(ret = makeClient(this, cameraCb, clientPackageName,
                cameraId, api1CameraId, facing,
                clientPid, clientUid, getpid(), legacyMode,
                halVersion, deviceVersion, effectiveApiLevel,
                /*out*/&tmp)).isOk()) 
            return ret;
        
        client = static_cast<CLIENT*>(tmp.get());
    ...
    return ret;

• makeClient()：
  • 主要是根据 API 版本以及 HAL 版本来选择生成具体的 Client 实例。
  • 如果是HAL1 + Camera1，则创建的是CameraClient，如果是HAL3 + Camera1，则创建的是Camera2Client。

Status CameraService::makeClient(const sp<CameraService>& cameraService,
        const sp<IInterface>& cameraCb, const String16& packageName, const String8& cameraId,
        int api1CameraId, int facing, int clientPid, uid_t clientUid, int servicePid,
        bool legacyMode, int halVersion, int deviceVersion, apiLevel effectiveApiLevel,
        /*out*/sp<BasicClient>* client) 

    if (halVersion < 0 || halVersion == deviceVersion) 
        // Default path: HAL version is unspecified by caller, create CameraClient
        // based on device version reported by the HAL.
        switch(deviceVersion) 
          case CAMERA_DEVICE_API_VERSION_1_0:
            if (effectiveApiLevel == API_1)   // Camera1 API route
                sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
                *client = new CameraClient(cameraService, tmp, packageName,
                        api1CameraId, facing, clientPid, clientUid,
                        getpid(), legacyMode);
             else  // Camera2 API route
                ALOGW("Camera using old HAL version: %d", deviceVersion);
                return STATUS_ERROR_FMT(ERROR_DEPRECATED_HAL,
                        "Camera device \\"%s\\" HAL version %d does not support camera2 API",
                        cameraId.string(), deviceVersion);
            
            break;
          case CAMERA_DEVICE_API_VERSION_3_0:
          case CAMERA_DEVICE_API_VERSION_3_1:
          case CAMERA_DEVICE_API_VERSION_3_2:
          case CAMERA_DEVICE_API_VERSION_3_3:
          case CAMERA_DEVICE_API_VERSION_3_4:
            if (effectiveApiLevel == API_1)  // Camera1 API route
                sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
                *client = new Camera2Client(cameraService, tmp, packageName,
                        cameraId, api1CameraId,
                        facing, clientPid, clientUid,
                        servicePid, legacyMode);
             else  // Camera2 API route
                sp<hardware::camera2::ICameraDeviceCallbacks> tmp =
                        static_cast<hardware::camera2::ICameraDeviceCallbacks*>(cameraCb.get());
                *client = new CameraDeviceClient(cameraService, tmp, packageName, cameraId,
                        facing, clientPid, clientUid, servicePid);
            
            break;
          default:
            // Should not be reachable
            ALOGE("Unknown camera device HAL version: %d", deviceVersion);
            return STATUS_ERROR_FMT(ERROR_INVALID_OPERATION,
                    "Camera device \\"%s\\" has unknown HAL version %d",
                    cameraId.string(), deviceVersion);
        
     else 
        // A particular HAL version is requested by caller. Create CameraClient
        // based on the requested HAL version.
        if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 &&
            halVersion == CAMERA_DEVICE_API_VERSION_1_0) 
            // Only support higher HAL version device opened as HAL1.0 device.
            sp<ICameraClient> tmp = static_cast<ICameraClient*>(cameraCb.get());
            *client = new CameraClient(cameraService, tmp, packageName,
                    api1CameraId, facing, clientPid, clientUid,
                    servicePid, legacyMode);
         else 
            // Other combinations (e.g. HAL3.x open as HAL2.x) are not supported yet.
            ALOGE("Invalid camera HAL version %x: HAL %x device can only be"
                    " opened as HAL %x device", halVersion, deviceVersion,
                    CAMERA_DEVICE_API_VERSION_1_0);
            return STATUS_ERROR_FMT(ERROR_ILLEGAL_ARGUMENT,
                    "Camera device \\"%s\\" (HAL version %d) cannot be opened as HAL version %d",
                    cameraId.string(), deviceVersion, halVersion);
        
    
    return Status::ok();

1.2 CameraHardwareInterface.h

• 路径：frameworks/av/services/camera/libcameraservice/device1/CameraHardwareInterface.h
• setCallback()：
  • 设置 notify 回调，这用来通知数据已经更新。
  • 设置 data 回调以及 dataTimestamp 回调，对应的是函数指针 mDataCb 与 mDataCvTimestamp 。
  • 注意到，设置 mDevice->ops 对应回调函数时，传入的不是之前设置的函数指针，而是 __data_cb 这样的函数。在该文件中，实现了 __data_cb ，将回调函数做了一层封装。

/** Set the notification and data callbacks */
void setCallbacks(notify_callback notify_cb,
                  data_callback data_cb,
                  data_callback_timestamp data_cb_timestamp,
                  void* user)

    mNotifyCb = notify_cb;
    mDataCb = data_cb;
    mDataCbTimestamp = data_cb_timestamp;
    mCbUser = user;

    ALOGV("%s(%s)", __FUNCTION__, mName.string());

    if (mDevice->ops->set_callbacks) 
        mDevice->ops->set_callbacks(mDevice,
                               __notify_cb,
                               __data_cb,
                               __data_cb_timestamp,
                               __get_memory,
                               this);
    

• __data_cb()：
  • 对原 callback 函数简单封装，附加了一个防止数组越界判断。

static void __data_cb(int32_t msg_type,
                      const camera_memory_t *data, unsigned int index,
                      camera_frame_metadata_t *metadata,
                      void *user)

    ALOGV("%s", __FUNCTION__);
    CameraHardwareInterface *__this =
            static_cast<CameraHardwareInterface *>(user);
    sp<CameraHeapMemory> mem(static_cast<CameraHeapMemory *>(data->handle));
    if (index >= mem->mNumBufs) 
        ALOGE("%s: invalid buffer index %d, max allowed is %d", __FUNCTION__,
             index, mem->mNumBufs);
        return;
    
    __this->mDataCb(msg_type, mem->mBuffers[index], metadata, __this->mCbUser);

2 控制流

2.1 Framework

2.1.1 Camera.java

• 路径：frameworks/base/core/java/android/hardware/Camera.java
• takePicture()：
  • 设置快门回调。
  • 设置各种类型的图片数据回调。
  • 调用 JNI takePicture 方法。
  • 注意，传入的参数 msgType 是根据相应 CallBack 是否存在而确定的，每种 Callback 应该对应一个二进制中的数位（如 1，10，100 中 1 的位置），于是这里采用 |= 操作给它赋值。

public final void takePicture(ShutterCallback shutter, PictureCallback raw,
        PictureCallback postview, PictureCallback jpeg) 
    mShutterCallback = shutter;
    mRawImageCallback = raw;
    mPostviewCallback = postview;
    mJpegCallback = jpeg;

    // If callback is not set, do not send me callbacks.
    int msgType = 0;
    if (mShutterCallback != null) 
        msgType |= CAMERA_MSG_SHUTTER;
    
    if (mRawImageCallback != null) 
        msgType |= CAMERA_MSG_RAW_IMAGE;
    
    if (mPostviewCallback != null) 
        msgType |= CAMERA_MSG_POSTVIEW_FRAME;
    
    if (mJpegCallback != null) 
        msgType |= CAMERA_MSG_COMPRESSED_IMAGE;
    

    native_takePicture(msgType);
    mFaceDetectionRunning = false;

2.2 Android Runtime

2.2.1 android_hardware_Camera.cpp

• 路径：frameworks/base/core/jni/android_hardware_Camera.cpp
  • takePicture()：
  • 获取已经打开的 camera 实例，调用其 takePicture() 接口。
  • 注意，在这个函数中，对于 RAW_IMAGE 有一些附加操作：
    • 如果设置了 RAW 的 callback ，则要检查上下文中，是否能找到对应 Buffer。
    • 若无法找到 Buffer ，则将 CAMERA_MSG_RAW_IMAGE 的信息去掉，换成 CAMERA_MSG_RAW_IMAGE_NOTIFY。
    • 替换后，就只会获得 notification 的消息，而没有对应的图像数据。

static void android_hardware_Camera_takePicture(JNIEnv *env, jobject thiz, jint msgType)

    ALOGV("takePicture");
    JNICameraContext* context;
    sp<Camera> camera = get_native_camera(env, thiz, &context);
    if (camera == 0) return;

    /*
     * When CAMERA_MSG_RAW_IMAGE is requested, if the raw image callback
     * buffer is available, CAMERA_MSG_RAW_IMAGE is enabled to get the
     * notification _and_ the data; otherwise, CAMERA_MSG_RAW_IMAGE_NOTIFY
     * is enabled to receive the callback notification but no data.
     *
     * Note that CAMERA_MSG_RAW_IMAGE_NOTIFY is not exposed to the
     * Java application.
     */
    if (msgType & CAMERA_MSG_RAW_IMAGE) 
        ALOGV("Enable raw image callback buffer");
        if (!context->isRawImageCallbackBufferAvailable()) 
            ALOGV("Enable raw image notification, since no callback buffer exists");
            msgType &= ~CAMERA_MSG_RAW_IMAGE;
            msgType |= CAMERA_MSG_RAW_IMAGE_NOTIFY;
        
    

    if (camera->takePicture(msgType) != NO_ERROR) 
        jniThrowRuntimeException(env, "takePicture failed");
        return;
    

2.3 C/C++ Libraries

2.3.1 Camera.cpp

• 路径：frameworks/av/camera/Camera.cpp
• takePicture()：
  • 获取一个 ICamera，调用其 takePicture 接口。

// take a picture
status_t Camera::takePicture(int msgType)

    ALOGV("takePicture: 0x%x", msgType);
    sp <::android::hardware::ICamera> c = mCamera;
    if (c == 0) return NO_INIT;
    return c->takePicture(msgType);

2.3.2 ICamera.cpp

• 路径：frameworks/av/camera/ICamera.cpp
• takePicture()：
  • 利用 Binder 机制发送相应指令到服务端。
  • 如果是 HAL1 + Camera1，实际调用到的是 CameraClient::takePicture() 函数。
  • 如果是 HAL3 + Camera1，实际调用到的是 Camera2Client::takePicture() 函数。

// take a picture - returns an IMemory (ref-counted mmap)
status_t takePicture(int msgType)

    ALOGV("takePicture: 0x%x", msgType);
    Parcel data, reply;
    data.writeInterfaceToken(ICamera::getInterfaceDescriptor());
    data.writeInt32(msgType);
    remote()->transact(TAKE_PICTURE, data, &reply);
    status_t ret = reply.readInt32();
    return ret;

2.3.3 CameraClient.cpp

• 路径：frameworks/av/services/camera/libcameraservice/api1/CameraClient.cpp
• takePicture()：
  • 注意，CAMERA_MSG_RAW_IMAGE 指令与 CAMERA_MSG_RAW_IMAGE_NOTIFY 指令不能同时有效，需要进行对应的检查。
  • 对传入的指令过滤，只留下与 takePicture() 操作相关的。
  • 调用 CameraHardwareInterface 中的 takePicture() 接口。

// take a picture - image is returned in callback
status_t CameraClient::takePicture(int msgType) 
    LOG1("takePicture (pid %d): 0x%x", getCallingPid(), msgType);

    Mutex::Autolock lock(mLock);
    status_t result = checkPidAndHardware();
    if (result != NO_ERROR) return result;

    if ((msgType & CAMERA_MSG_RAW_IMAGE) &&
        (msgType & CAMERA_MSG_RAW_IMAGE_NOTIFY)) 
        ALOGE("CAMERA_MSG_RAW_IMAGE and CAMERA_MSG_RAW_IMAGE_NOTIFY"
                " cannot be both enabled");
        return BAD_VALUE;
    

    // We only accept picture related message types
    // and ignore other types of messages for takePicture().
    int picMsgType = msgType
                        & (CAMERA_MSG_SHUTTER |
                           CAMERA_MSG_POSTVIEW_FRAME |
                           CAMERA_MSG_RAW_IMAGE |
                           CAMERA_MSG_RAW_IMAGE_NOTIFY |
                           CAMERA_MSG_COMPRESSED_IMAGE);

    enableMsgType(picMsgType);

    return mHardware->takePicture();

2.3.4 Camera2Client.cpp

• 路径：frameworks/av/services/camera/libcameraservice/api1/Camera2Client.cpp
• takePicture()：
  • 调用了 mCaptureSequencer->startCapture()。

status_t Camera2Client::takePicture(int msgType) 
    ATRACE_CALL();
    Mutex::Autolock icl(mBinderSerializationLock);
    status_t res;
    if ( (res = checkPid(__FUNCTION__) ) != OK) return res;
    
        SharedParameters::Lock l(mParameters);
        switch (l.mParameters.state) 
            case Parameters::DISCONNECTED:
            case Parameters::STOPPED:
            case Parameters::WAITING_FOR_PREVIEW_WINDOW:
                ALOGE("%s: Camera %d: Cannot take picture without preview enabled",
                        __FUNCTION__, mCameraId);
                return INVALID_OPERATION;
            case Parameters::PREVIEW:
                // Good to go for takePicture
                res = commandStopFaceDetectionL(l.mParameters);
                if (res != OK) 
                    ALOGE("%s: Camera %d: Unable to stop face detection for still capture",
                            __FUNCTION__, mCameraId);
                    return res;
                
                l.mParameters.state = Parameters::STILL_CAPTURE;
                break;
            case Parameters::RECORD:
                // Good to go for video snapshot
                l.mParameters.state = Parameters::VIDEO_SNAPSHOT;
                break;
            case Parameters::STILL_CAPTURE:
            case Parameters::VIDEO_SNAPSHOT:
                ALOGE("%s: Camera %d: Already taking a picture",
                        __FUNCTION__, mCameraId);
                return INVALID_OPERATION;
        
        ALOGV("%s: Camera %d: Starting picture capture", __FUNCTION__, mCameraId);
        res = updateProcessorStream(mJpegProcessor, l.mParameters);
        if (res != OK) 
            ALOGE("%s: Camera %d: Can't set up still image stream: %s (%d)",
                    __FUNCTION__, mCameraId, strerror(-res), res);
            return res;
        
    
    // Need HAL to have correct settings before (possibly) triggering precapture
    syncWithDevice();
    res = mCaptureSequencer->startCapture(msgType);
    if (res != OK) 
        ALOGE("%s: Camera %d: Unable to start capture: %s (%d)",
                __FUNCTION__, mCameraId, strerror(-res), res);
    
    return res;

2.3.4.1 CaptureSequencer.cpp

• 路径：frameworks/av/services/camera/libcameraservice/api1/client2/CaptureSequencer.cpp
• startCapture()：
  • 这里 mStartCaptureSignal.signal() 会激活 mStartCaptureSignal 条件。

status_t CaptureSequencer::startCapture(int msgType) 
    ALOGV("%s", __FUNCTION__);
    ATRACE_CALL();
    Mutex::Autolock l(mInputMutex);
    if (mBusy) 
        ALOGE("%s: Already busy capturing!", __FUNCTION__);
        return INVALID_OPERATION;
    
    if (!mStartCapture) 
        mMsgType = msgType;
        mStartCapture = true;
        mStartCaptureSignal.signal();
    
    return OK;

CaptureSequencer 模块是 take picture 操作的重点，在 Camera2Client 中进行了创建，首先来看 CaptureSequencer 线程的 threadLoop 函数：

bool CaptureSequencer::threadLoop() 
 
    sp<Camera2Client> client = mClient.promote();
    if (client == 0) return false;
 
    CaptureState currentState;
    
        Mutex::Autolock l(mStateMutex);
        currentState = mCaptureState;
    
 
    currentState = (this->*kStateManagers[currentState])(client);
 
    Mutex::Autolock l(<
以上是关于Camera1 源码解析系列—— Camera1 takePicture() 流程解析的主要内容，如果未能解决你的问题，请参考以下文章