Camera的架構與Android系統的整體架構保持一致,如下圖所示,本文主要從以下四個方面對其進行說明。 Framework:Camera.java Android Runtime:android_hardware_Camera.cpp Library:Camera Client和Camera S... ...
Camera的架構與Android系統的整體架構保持一致,如下圖所示,本文主要從以下四個方面對其進行說明。
- Framework:Camera.java
- Android Runtime:android_hardware_Camera.cpp
- Library:Camera Client和Camera Service
- HAL:CameraHardwareInterface
一、Framework:Camera.java
Camera是應用層軟體直接使用的類,涵蓋了啟動、預覽、拍攝及關閉等操作攝像頭的全部介面。Camera.java在Android源碼中的路徑為:framework/base/core/java/android/hardware。為了說明整個Camera系統的架構,這裡暫不橫向分析Camera.java的功能,下麵從open()方法著手:
public static Camera open() { int numberOfCameras = getNumberOfCameras(); CameraInfo cameraInfo = new CameraInfo(); for (int i = 0; i < numberOfCameras; i++) { getCameraInfo(i, cameraInfo); if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) { return new Camera(i); } } return null; }
open()方法需要註意以下幾點:
- getNumberOfCameras為native方法,實現在android_hardware_Camera.cpp中;
- CameraInfo是Camera定義的靜態內部類,包含facing、orientation、canDisableShutterSound;
- getCameraInfo內部調用native方法_getCameraInfo獲取攝像頭信息;
- open()預設啟動的是後置攝像頭(CAMERA_FACING_BACK)。
/** used by Camera#open, Camera#open(int) */ Camera(int cameraId) { int err = cameraInitNormal(cameraId); if (checkInitErrors(err)) { switch(err) { case EACCESS: throw new RuntimeException("Fail to connect to camera service"); case ENODEV: throw new RuntimeException("Camera initialization failed"); default: // Should never hit this. throw new RuntimeException("Unknown camera error"); } } }
Camera構造器的核心實現在cameraInitNormal中,cameraInitNormal調用cameraInitVersion,並傳入參數cameraId和CAMERA_HAL_API_VERSION_NORMAL_CONNECT,後者代表HAL的版本。
private int cameraInitVersion(int cameraId, int halVersion) {
……
String packageName = ActivityThread.currentPackageName(); return native_setup(new WeakReference<Camera>(this), cameraId, halVersion, packageName); }
cameraInitNormal調用本地方法native_setup(),由此進入到android_hardware_Camera.cpp中,native_setup()的簽名如下:
private native final int native_setup(Object camera_this, int cameraId, int halVersion, String packageName);
二、Android Runtime:android_hardware_Camera.cpp
native_setup()被動態註冊到JNI,通過JNI調用android_hardware_Camera_native_setup()方法。
static JNINativeMethod camMethods[] = { …… { "native_setup", "(Ljava/lang/Object;ILjava/lang/String;)V", (void*)android_hardware_Camera_native_setup } …… };
JNI的重點是android_hardware_Camera_native_setup()方法的實現:
// connect to camera service static jint android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz, jobject weak_this, jint cameraId, jint halVersion, jstring clientPackageName) { // Convert jstring to String16 const char16_t *rawClientName = env->GetStringChars(clientPackageName, NULL); jsize rawClientNameLen = env->GetStringLength(clientPackageName); String16 clientName(rawClientName, rawClientNameLen); env->ReleaseStringChars(clientPackageName, rawClientName); sp<Camera> camera; if (halVersion == CAMERA_HAL_API_VERSION_NORMAL_CONNECT) { // Default path: hal version is don't care, do normal camera connect. camera = Camera::connect(cameraId, clientName, Camera::USE_CALLING_UID); } else { jint status = Camera::connectLegacy(cameraId, halVersion, clientName, Camera::USE_CALLING_UID, camera); if (status != NO_ERROR) { return status; } } if (camera == NULL) { return -EACCES; } // make sure camera hardware is alive if (camera->getStatus() != NO_ERROR) { return NO_INIT; } jclass clazz = env->GetObjectClass(thiz); if (clazz == NULL) { // This should never happen jniThrowRuntimeException(env, "Can't find android/hardware/Camera"); return INVALID_OPERATION; } // We use a weak reference so the Camera object can be garbage collected. // The reference is only used as a proxy for callbacks. sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera); context->incStrong((void*)android_hardware_Camera_native_setup); camera->setListener(context); // save context in opaque field env->SetLongField(thiz, fields.context, (jlong)context.get()); return NO_ERROR; }
android_hardware_Camera_native_setup()方法通過調用Camera::connect()方法請求連接CameraService服務。入參中:
- clientName是通過將clientPackageName從jstring轉換為String16格式得到;
- Camera::USE_CALLING_UID是定義在Camera.h中的枚舉類型,其值為ICameraService::USE_CALLING_UID(同樣為枚舉類型,值為-1)。
Camera::connect()位於Camera.cpp中,由此進入到Library層。
三、Library:Camera Client和Camera Service
如上述架構圖中所示,ICameraService.h、ICameraClient.h和ICamera.h三個類定義了Camera的介面和架構,ICameraService.cpp和Camera.cpp兩個文件用於Camera架構的實現,Camera的具體功能在下層調用硬體相關的介面來實現。Camera.h是Camera系統對上層的介面。
具體的,Camera類繼承模板類CameraBase,Camera::connect()調用了CameraBase.cpp中的connect()方法。
sp<Camera> Camera::connect(int cameraId, const String16& clientPackageName, int clientUid) { return CameraBaseT::connect(cameraId, clientPackageName, clientUid); }
CameraBase實際上又繼承了IBinder的DeathRecipient內部類,DeathRecipient虛擬繼承自RefBase。RefBase是Android中的引用計數基礎類,其中定義了incStrong、decStrong、incWeak和decWeak等涉及sp/wp的指針操作函數,當然這扯遠了。
template <typename TCam> struct CameraTraits { }; template <typename TCam, typename TCamTraits = CameraTraits<TCam> > class CameraBase : public IBinder::DeathRecipient { public: static sp<TCam> connect(int cameraId, const String16& clientPackageName, int clientUid); …… }
class DeathRecipient : public virtual RefBase { public: virtual void binderDied(const wp<IBinder>& who) = 0; };
回到Camera::connect()的實現上,其中,new TCam(cameraId)生成BnCameraClient對象,BnCameraClient定義在ICameraClient.h文件中,繼承自模板類BnInterface。getCameraService()方法返回CameraService的服務代理BpCameraService,BpCameraService同樣繼承自模板類BnInterface。然後通過Binder通信發送CONNECT命令,當BnCameraService收到CONNECT命令後調用CameraService的connect()成員函數來做相應的處理。
template <typename TCam, typename TCamTraits> sp<TCam> CameraBase<TCam, TCamTraits>::connect(int cameraId, const String16& clientPackageName, int clientUid) { ALOGV("%s: connect", __FUNCTION__); sp<TCam> c = new TCam(cameraId); // BnCameraClient sp<TCamCallbacks> cl = c; status_t status = NO_ERROR; const sp<ICameraService>& cs = getCameraService(); // BpCameraService if (cs != 0) { TCamConnectService fnConnectService = TCamTraits::fnConnectService; status = (cs.get()->*fnConnectService)(cl, cameraId, clientPackageName, clientUid, /*out*/ c->mCamera); } if (status == OK && c->mCamera != 0) { c->mCamera->asBinder()->linkToDeath(c); c->mStatus = NO_ERROR; } else { ALOGW("An error occurred while connecting to camera: %d", cameraId); c.clear(); } return c; }
class BnCameraClient: public BnInterface<ICameraClient> { public: virtual status_t onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags = 0); };
class BpCameraService: public BpInterface<ICameraService> { public: BpCameraService(const sp<IBinder>& impl) : BpInterface<ICameraService>(impl) { } …… }
註:connect()函數在BpCameraService和BnCameraService的父類ICameraService中聲明為純虛函數,在BpCameraService和CameraService中分別給出了實現,BpCameraService作為代理類,提供介面給客戶端,真正實現在BnCameraService的子類CameraService中。
在BpCameraService中,connect()函數實現如下:
// connect to camera service (android.hardware.Camera) virtual status_t connect(const sp<ICameraClient>& cameraClient, int cameraId, const String16 &clientPackageName, int clientUid, /*out*/ sp<ICamera>& device) { Parcel data, reply; data.writeInterfaceToken(ICameraService::getInterfaceDescriptor()); data.writeStrongBinder(cameraClient->asBinder()); data.writeInt32(cameraId); data.writeString16(clientPackageName); data.writeInt32(clientUid); remote()->transact(BnCameraService::CONNECT, data, &reply); // BpBinder的transact()函數向IPCThreadState實例發送消息,通知其有消息要發送給binder driver
if (readExceptionCode(reply)) return -EPROTO; status_t status = reply.readInt32(); if (reply.readInt32() != 0) { device = interface_cast<ICamera>(reply.readStrongBinder()); // client端讀出server返回的bind } return status; }
首先將傳遞過來的Camera對象cameraClient轉換成IBinder類型,將調用的參數寫到Parcel(可理解為Binder通信的管道)中,通過BpBinder的transact()函數發送消息,然後由BnCameraService去響應該連接,最後就是等待服務端返回,如果成功則生成一個BpCamera實例。
真正的服務端響應實現在BnCameraService的onTransact()函數中,其負責解包收到的Parcel並執行client端的請求的方法。
status_t BnCameraService::onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags) { switch(code) {
…… case CONNECT: { CHECK_INTERFACE(ICameraService, data, reply); sp<ICameraClient> cameraClient = interface_cast<ICameraClient>(data.readStrongBinder()); // 使用Camera的Binder對象生成Camera客戶代理BpCameraClient實例 int32_t cameraId = data.readInt32(); const String16 clientName = data.readString16(); int32_t clientUid = data.readInt32(); sp<ICamera> camera; status_t status = connect(cameraClient, cameraId, clientName, clientUid, /*out*/camera); // 將生成的BpCameraClient對象作為參數傳遞到CameraService的connect()函數中 reply->writeNoException(); reply->writeInt32(status); // 將BpCamera對象以IBinder的形式打包到Parcel中返回 if (camera != NULL) { reply->writeInt32(1); reply->writeStrongBinder(camera->asBinder()); } else { reply->writeInt32(0); } return NO_ERROR; } break; …… } }
主要的處理包括:
- 通過data中Camera的Binder對象生成Camera客戶代理BpCameraClient實例;
- 將生成的BpCameraClient對象作為參數傳遞到CameraService(/frameworks/av/services/camera /libcameraservice/CameraService.cpp)的connect()函數中,該函數會返回一個BpCamera實例;
- 將在上述實例對象以IBinder的形式打包到Parcel中返回。
最後,BpCamera實例是通過CameraService::connect()函數返回的。CameraService::connect()實現的核心是調用connectHelperLocked()函數根據HAL不同API的版本創建不同的client實例(早期版本中好像沒有connectHelperLocked()這個函數,但功能基本相似)。
status_t CameraService::connectHelperLocked( /*out*/ sp<Client>& client, /*in*/ const sp<ICameraClient>& cameraClient, int cameraId, const String16& clientPackageName, int clientUid, int callingPid, int halVersion, bool legacyMode) { int facing = -1; int deviceVersion = getDeviceVersion(cameraId, &facing); if (halVersion < 0 || halVersion == deviceVersion) { // Default path: HAL version is unspecified by caller, create CameraClient // based on device version reported by the HAL. switch(deviceVersion) { case CAMERA_DEVICE_API_VERSION_1_0: client = new CameraClient(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid(), legacyMode); break; case CAMERA_DEVICE_API_VERSION_2_0: case CAMERA_DEVICE_API_VERSION_2_1: case CAMERA_DEVICE_API_VERSION_3_0: case CAMERA_DEVICE_API_VERSION_3_1: case CAMERA_DEVICE_API_VERSION_3_2: client = new Camera2Client(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid(), legacyMode); break; case -1: ALOGE("Invalid camera id %d", cameraId); return BAD_VALUE; default: ALOGE("Unknown camera device HAL version: %d", deviceVersion); return INVALID_OPERATION; } } else { // A particular HAL version is requested by caller. Create CameraClient // based on the requested HAL version. if (deviceVersion > CAMERA_DEVICE_API_VERSION_1_0 && halVersion == CAMERA_DEVICE_API_VERSION_1_0) { // Only support higher HAL version device opened as HAL1.0 device. client = new CameraClient(this, cameraClient, clientPackageName, cameraId, facing, callingPid, clientUid, getpid(), legacyMode); } else { // Other combinations (e.g. HAL3.x open as HAL2.x) are not supported yet. ALOGE("Invalid camera HAL version %x: HAL %x device can only be" " opened as HAL %x device", halVersion, deviceVersion, CAMERA_DEVICE_API_VERSION_1_0); return INVALID_OPERATION; } } status_t status = connectFinishUnsafe(client, client->getRemote()); if (status != OK) { // this is probably not recoverable.. maybe the client can try again return status; } mClient[cameraId] = client; LOG1("CameraService::connect X (id %d, this pid is %d)", cameraId, getpid()); return OK; }
可見,在CAMERA_DEVICE_API_VERSION_2_0之前使用CameraClient進行實例化,之後則採用Camera2Client進行實例化。以CameraClient為例,其initialize()函數如下:
status_t CameraClient::initialize(camera_module_t *module) { int callingPid = getCallingPid(); status_t res; LOG1("CameraClient::initialize E (pid %d, id %d)", callingPid, mCameraId); // Verify ops permissions res = startCameraOps(); if (res != OK) { return res; } char camera_device_name[10]; snprintf(camera_device_name, sizeof(camera_device_name), "%d", mCameraId); mHardware = new CameraHardwareInterface(camera_device_name); res = mHardware->initialize(&module->common); if (res != OK) { ALOGE("%s: Camera %d: unable to initialize device: %s (%d)", __FUNCTION__, mCameraId, strerror(-res), res); mHardware.clear(); return res; } mHardware->setCallbacks(notifyCallback, dataCallback, dataCallbackTimestamp, (void *)(uintptr_t)mCameraId); // Enable zoom, error, focus, and metadata messages by default enableMsgType(CAMERA_MSG_ERROR | CAMERA_MSG_ZOOM | CAMERA_MSG_FOCUS | CAMERA_MSG_PREVIEW_METADATA | CAMERA_MSG_FOCUS_MOVE); LOG1("CameraClient::initialize X (pid %d, id %d)", callingPid, mCameraId); return OK; }
上述函數中,主要註意以下流程:
- 加粗的代碼CameraHardwareInterface新建了了一個Camera硬體介面,當然,camera_device_name為攝像頭設備名;
- mHardware->initialize(&module->common)調用底層硬體的初始化方法;
- mHardware->setCallbacks將CamerService處的回調函數註冊到HAL處。
CameraHardwareInterface定義了Camera的硬體抽象特征,由此進入到HAL。
四、HAL:CameraHardwareInterface
CameraHardwareInterface的作用在於鏈接Camera Server和V4L2,通過實現CameraHardwareInterface可以屏蔽不同的driver對Camera Server的影響。CameraHardwareInterface同樣虛擬繼承自RefBase。
class CameraHardwareInterface : public virtual RefBase { public: CameraHardwareInterface(const char *name) { mDevice = 0; mName = name; } …… }
CameraHardwareInterface中包含了控制通道和數據通道,控制通道用於處理預覽和視頻獲取的開始/停止、拍攝照片、自動對焦等功能,數據通道通過回調函數來獲得預覽、視頻錄製、自動對焦等數據。當需要支持新的硬體時就需要繼承於CameraHardwareInterface ,來實現對應的功能。CameraHardwareInterface提供的public方法如下:
在前一節中,initialize()函數調用了mHardware->initialize和mHardware->setCallbacks,下麵來看下CameraHardwareInterface.h對其的實現。
status_t initialize(hw_module_t *module) { ALOGI("Opening camera %s", mName.string()); camera_module_t *cameraModule = reinterpret_cast<camera_module_t *>(module); camera_info info; status_t res = cameraModule->get_camera_info(atoi(mName.string()), &info); if (res != OK) return res; int rc = OK; if (module->module_api_version >= CAMERA_MODULE_API_VERSION_2_3 && info.device_version > CAMERA_DEVICE_API_VERSION_1_0) { // Open higher version camera device as HAL1.0 device. rc = cameraModule->open_legacy(module, mName.string(), CAMERA_DEVICE_API_VERSION_1_0, (hw_device_t **)&mDevice); } else { rc = CameraService::filterOpenErrorCode(module->methods->open( module, mName.string(), (hw_device_t **)&mDevice)); } if (rc != OK) { ALOGE("Could not open camera %s: %d", mName.string(), rc); return rc; } initHalPreviewWindow(); return rc; }
在initialize()方法中,通過cameraModule->open_legacy打開攝像頭模組,initHalPreviewWindow()用於初始化Preview的相關流opspreview_stream_ops,初始化hal的預覽視窗。
void initHalPreviewWindow() { mHalPreviewWindow.nw.cancel_buffer = __cancel_buffer; mHalPreviewWindow.nw.lock_buffer = __lock_buffer; mHalPreviewWindow.nw.dequeue_buffer = __dequeue_buffer; mHalPreviewWindow.nw.enqueue_buffer = __enqueue_buffer; mHalPreviewWindow.nw.set_buffer_count = __set_buffer_count; mHalPreviewWindow.nw.set_buffers_geometry = __set_buffers_geometry; mHalPreviewWindow.nw.set_crop = __set_crop; mHalPreviewWindow.nw.set_timestamp = __set_timestamp; mHalPreviewWindow.nw.set_usage = __set_usage; mHalPreviewWindow.nw.set_swap_interval = __set_swap_interval; mHalPreviewWindow.nw.get_min_undequeued_buffer_count = __get_min_undequeued_buffer_count; }
/** Set the notification and data callbacks */ void setCallbacks(notify_callback notify_cb, data_callback data_cb, data_callback_timestamp data_cb_timestamp, void* user) { mNotifyCb = notify_cb; mDataCb = data_cb; mDataCbTimestamp = data_cb_timestamp; mCbUser = user; ALOGV("%s(%s)", __FUNCTION__, mName.string()); if (mDevice->ops->set_callbacks) { mDevice->ops->set_callbacks(mDevice, __notify_cb, __data_cb, __data_cb_timestamp, __get_memory, this); } }set_callbacks中,__notify_cb、__data_cb、__data_cb_timestamp和__get_memory分別消息回調,數據回調,時間戳回調,以及記憶體相關操作的回調。
以上通過簡略分析應用層調用Camera.open()之後在Framework、ART、Library以及HAL層的響應,來說明Android中Camera系統的整體架構,希望對讀者能有一定的幫助,後續將在理解Camera整體架構的基礎,探索更加高效的Preview方式,敬請期待!