Camerax ImageCapture 传递给 Chaquopy 太慢了
Posted
技术标签:
【中文标题】Camerax ImageCapture 传递给 Chaquopy 太慢了【英文标题】:Camerax ImageCapture passing to Chaquopy too slow 【发布时间】:2021-06-23 17:30:34 【问题描述】:我用 Chaquopy 制作了 Camerax 应用程序:
ImageCapture 用例捕获图像(jpg) 将其转换为位图 将位图转换为字符串 将字符串传递给 Python Python OpenCV 获取图像分辨率并将其返回给 android(如字符串) 在 Textview 上显示应用程序正在运行,但问题是:
-
应用程序太慢:15-20 秒(字符串转换?)
显示错误的分辨率(分辨率在 ImageCapture 配置中设置)
Android CameraAtivity 代码:
'''
public class CameraActivity extends AppCompatActivity
private PreviewView previewView;
private ImageCapture imageCapture;
private ListenableFuture<ProcessCameraProvider> cameraProviderFuture;
private TextView textView;
private Button button;
@Override
protected void onCreate(@Nullable Bundle savedInstanceState)
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_camera);
previewView = findViewById(R.id.previewView);
cameraProviderFuture = ProcessCameraProvider.getInstance(this);
textView = findViewById(R.id.length);
button = findViewById(R.id.button);
cameraProviderFuture.addListener(new Runnable()
@Override
public void run()
try
ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
bindImageAnalysis(cameraProvider);
catch (ExecutionException | InterruptedException e)
e.printStackTrace();
, ContextCompat.getMainExecutor(this));
private void bindImageAnalysis(@NonNull ProcessCameraProvider cameraProvider)
Preview preview = new Preview.Builder().build();
CameraSelector cameraSelector = new CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_BACK).build();
preview.setSurfaceProvider(previewView.createSurfaceProvider());
Executor cameraexecutor = ContextCompat.getMainExecutor(this);
ImageCapture imageCapture = new ImageCapture.Builder().setTargetResolution(new Size(1200, 1600)).build();
button.setOnClickListener(new View.OnClickListener()
@Override
public void onClick(View view)
imageCapture.takePicture(cameraexecutor, new ImageCapture.OnImageCapturedCallback()
@Override
public void onCaptureSuccess(@NonNull ImageProxy image)
ByteBuffer byteBuffer = image.getPlanes()[0].getBuffer();
byte[] bytes = new byte[byteBuffer.capacity()];
byteBuffer.get(bytes);
Bitmap bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, baos);
byte[] imageBytes = baos.toByteArray();
String imgString = Base64.encodeToString(imageBytes, Base64.DEFAULT);
PyObject obj = pythonn(imgString);
textView.setText(obj.toString());
@Override
public void onError(@NonNull ImageCaptureException error)
error.printStackTrace();
);
);
cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector, imageCapture,
preview);
private PyObject pythonn(String imgString)
if(!Python.isStarted())
Python.start(new AndroidPlatform(this ));
Python py = Python.getInstance();
final PyObject pyobj = py.getModule("shape");
PyObject obj = pyobj.callAttr("main",imgString);
return obj;
'''
Python 代码;
import cv2
import numpy as np
import base64
def main(imgString):
decoded_data = base64.b64decode(imgString)
np_data = np.fromstring(decoded_data,np.uint8)
img = cv2.imdecode(np_data,cv2.IMREAD_UNCHANGED)
if img.shape[0] > img.shape[1]:
img = cv2.transpose(img)
return str(str(img.shape[0]) + "_" + str(img.shape[1]))
所有代码都在 Github 上: https://github.com/kintipu/Camerax_ImageCapture_Chaquopy_OpenCV_ImageResolution/tree/master
2021 年 3 月 29 日更新: 我将尝试使用 bytearray,就像 mhsmith 建议的那样。 与此同时,我尝试了另一件事:
捕获图像并将其保存为 jpg
然后直接从python中读取图片 应用可以在模拟器上运行,但不能在真手机上运行 安卓代码: '''
公共类 CameraActivity 扩展 AppCompatActivity 私有 PreviewView 预览视图; 私有图像捕获图像捕获; 私有 ListenableFuture cameraProviderFuture; 私有文本视图文本视图; 私人按钮按钮; 私有上下文上下文;
@Override
protected void onCreate(@Nullable Bundle savedInstanceState)
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_camera);
previewView = findViewById(R.id.previewView);
cameraProviderFuture = ProcessCameraProvider.getInstance(this);
textView = findViewById(R.id.length);
button = findViewById(R.id.button);
cameraProviderFuture.addListener(new Runnable()
@Override
public void run()
try
ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
bindImageAnalysis(cameraProvider);
catch (ExecutionException | InterruptedException e)
e.printStackTrace();
, ContextCompat.getMainExecutor(this));
private void bindImageAnalysis(@NonNull ProcessCameraProvider cameraProvider)
Preview preview = new Preview.Builder().build();
CameraSelector cameraSelector = new CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_BACK).build();
preview.setSurfaceProvider(previewView.createSurfaceProvider());
Executor cameraexecutor = ContextCompat.getMainExecutor(this);
ImageCapture imageCapture = new ImageCapture.Builder().setTargetResolution(new Size(1200, 1600)).build();
button.setOnClickListener(new View.OnClickListener()
@Override
public void onClick(View v)
File file = new File(getBatchDirectoryName(), "photo"+ ".jpg");
ImageCapture.OutputFileOptions outputFileOptions = new ImageCapture.OutputFileOptions.Builder(file).build();
imageCapture.takePicture(outputFileOptions, cameraexecutor, new ImageCapture.OnImageSavedCallback ()
@Override
public void onImageSaved(@NonNull ImageCapture.OutputFileResults outputFileResults)
new Handler().post(new Runnable()
@Override
public void run()
Toast.makeText(CameraActivity.this, "Image Saved successfully" + file.getAbsolutePath(), Toast.LENGTH_LONG).show();
PyObject obj = pythonn();
textView.setText(obj.toString());
);
@Override
public void onError(@NonNull ImageCaptureException error)
error.printStackTrace();
);
);
cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector, imageCapture,
preview);
public String getBatchDirectoryName()
String app_folder_path = "";
app_folder_path = Environment.getExternalStorageDirectory().toString() + "/images";
File dir = new File(app_folder_path);
if (!dir.exists() && !dir.mkdirs())
return app_folder_path;
private PyObject pythonn()
if(!Python.isStarted())
Python.start(new AndroidPlatform(this ));
Python py = Python.getInstance();
final PyObject pyobj = py.getModule("shape");
PyObject obj = pyobj.callAttr("main");
return obj;
'''
Python 代码:
'''
import cv2
import numpy as np
from android.os import Environment
path = str(str(Environment.getExternalStorageDirectory()) + "/images" + "/photo.jpg")
def main():
img = cv2.imread(path)
return str(str(img.shape[0]) + "_" + str(img.shape[1]))
'''
2021 年 4 月 4 日更新:
通过一些修改,我让它变得更快:
将 bytearray 之类的图像传递给 python(而不是字符串和 base64 转换) - 根据 Malcom Smith 的建议 位图压缩设置为 JPEG(而不是 PNG) Python 在 Create 时启动安卓代码; '''
public class CameraActivity extends AppCompatActivity
private PreviewView previewView;
private ImageCapture imageCapture;
private ListenableFuture<ProcessCameraProvider> cameraProviderFuture;
private TextView textView;
private Button button;
@Override
protected void onCreate(@Nullable Bundle savedInstanceState)
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_camera);
previewView = findViewById(R.id.previewView);
cameraProviderFuture = ProcessCameraProvider.getInstance(this);
textView = findViewById(R.id.length);
button = findViewById(R.id.button);
if(!Python.isStarted())
Python.start(new AndroidPlatform(this ));
cameraProviderFuture.addListener(new Runnable()
@Override
public void run()
try
ProcessCameraProvider cameraProvider = cameraProviderFuture.get();
bindImageAnalysis(cameraProvider);
catch (ExecutionException | InterruptedException e)
e.printStackTrace();
, ContextCompat.getMainExecutor(this));
private void bindImageAnalysis(@NonNull ProcessCameraProvider cameraProvider)
Preview preview = new Preview.Builder().build();
CameraSelector cameraSelector = new CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_BACK).build();
preview.setSurfaceProvider(previewView.createSurfaceProvider());
Executor cameraexecutor = ContextCompat.getMainExecutor(this);
ImageCapture imageCapture = new ImageCapture.Builder().setTargetResolution(new Size(1200, 1600)).build();
button.setOnClickListener(new View.OnClickListener()
@Override
public void onClick(View view)
imageCapture.takePicture(cameraexecutor, new ImageCapture.OnImageCapturedCallback()
@Override
public void onCaptureSuccess(@NonNull ImageProxy image)
ByteBuffer byteBuffer = image.getPlanes()[0].getBuffer();
byte[] bytes = new byte[byteBuffer.capacity()];
byteBuffer.get(bytes);
Bitmap bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
ByteArrayOutputStream baos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
byte[] imageBytes = baos.toByteArray();
// String length = String.valueOf((imgString.length()));
// textView.setText(length);
PyObject obj = pythonn(imageBytes);
textView.setText(obj.toString());
@Override
public void onError(@NonNull ImageCaptureException error)
error.printStackTrace();
);
);
cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector, imageCapture,
preview);
private PyObject pythonn(byte[] imageBytes)
Python py = Python.getInstance();
final PyObject pyobj = py.getModule("shape");
PyObject obj = pyobj.callAttr("main",imageBytes);
return obj;
Python: '''
import cv2
import numpy as np
def main(imageBytes):
np_data = np.asarray(imageBytes,np.uint8)
img = cv2.imdecode(np_data,cv2.IMREAD_UNCHANGED)
if img.shape[0] > img.shape[1]:
img = cv2.transpose(img)
img = cv2.resize(img, (1600, 1200))
return str(str(img.shape[0]) + "_" + str(img.shape[1]))
更新了github:https://github.com/kintipu/Camerax_ImageCapture_Chaquopy_OpenCV_ImageResolution/blob/master/README.md
【问题讨论】:
【参考方案1】:不需要使用base64,你可以发送PNG编码的byte[]
数组,它会在Python中作为bytes-like对象接收。我不确定这个对象是否会被cv2.imdecode
直接接受,但它肯定会被np.array
或np.frombuffer
接受。 (不推荐将二进制数据传递给np.fromstring
,正如其documentation page 所解释的那样。)
如果您不确定哪一行代码导致延迟,您可以在行之间添加Log
调用(或Python 中的print
调用),然后检查Logcat 中的时间戳。
【讨论】:
谢谢mhsmith,我会试试的。以上是关于Camerax ImageCapture 传递给 Chaquopy 太慢了的主要内容,如果未能解决你的问题,请参考以下文章
为 CameraX ImageAnalysis 进行 YUV 到 RGB 的转换
Android CameraX 仿一甜相机(录像拍照可调节尺寸聚焦照明网格线),最全的CameraX教程