Objective C中iOS的图像模糊检测
Posted
技术标签:
【中文标题】Objective C中iOS的图像模糊检测【英文标题】：Image blur detection for iOS in Objective C 【发布时间】：2021-04-26 07:36:11 【问题描述】：我正在尝试确定从相机拍摄的 ios 图像是否模糊。 我已经在拍照前检查了相机焦距，但是如果图像模糊，这似乎有所不同。
我使用 Open CV 在 android 上完成了这项工作， OpenCV with Laplacian formula to detect image is blur or not in Android
结果是，
int soglia = -6118750;
if (maxLap <= soglia) // blurry
我玩了一下这个并降低到-6718750。
对于 iOS，关于执行此操作的信息似乎较少。我看到一些人试图在 iOS 上使用 Open CV 来解决这个问题，但他们似乎并不成功。
我看到这篇文章在 iOS 上使用 Metal 来做到这一点， https://medium.com/better-programming/blur-detection-via-metal-on-ios-16dd02cb1558
这是用 Swift 编写的，所以我手动将其逐行转换为 Objective C。 我认为可能代码是正确的翻译，但不确定原始代码是否正确或一般适用于相机捕获的图像？
在我的测试中，它总是给我一个 2 的结果，无论是平均值还是方差，这如何用于检测模糊图像或任何其他想法？
- (BOOL) detectBlur: (CGImageRef)image
NSLog(@"detectBlur: %@", image);
// Initialize MTL
device = MTLCreateSystemDefaultDevice();
queue = [device newCommandQueue];
// Create a command buffer for the transformation pipeline
id <MTLCommandBuffer> commandBuffer = [queue commandBuffer];
// These are the two built-in shaders we will use
MPSImageLaplacian* laplacian = [[MPSImageLaplacian alloc] initWithDevice: device];
MPSImageStatisticsMeanAndVariance* meanAndVariance = [[MPSImageStatisticsMeanAndVariance alloc] initWithDevice: device];
// Load the captured pixel buffer as a texture
MTKTextureLoader* textureLoader = [[MTKTextureLoader alloc] initWithDevice: device];
id <MTLTexture> sourceTexture = [textureLoader newTextureWithCGImage: image options: nil error: nil];
// Create the destination texture for the laplacian transformation
MTLTextureDescriptor* lapDesc = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: sourceTexture.pixelFormat width: sourceTexture.width height: sourceTexture.height mipmapped: false];
lapDesc.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id <MTLTexture> lapTex = [device newTextureWithDescriptor: lapDesc];
// Encode this as the first transformation to perform
[laplacian encodeToCommandBuffer: commandBuffer sourceTexture: sourceTexture destinationTexture: lapTex];
// Create the destination texture for storing the variance.
MTLTextureDescriptor* varianceTextureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat: sourceTexture.pixelFormat width: 2 height: 1 mipmapped: false];
varianceTextureDescriptor.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id <MTLTexture> varianceTexture = [device newTextureWithDescriptor: varianceTextureDescriptor];
// Encode this as the second transformation
[meanAndVariance encodeToCommandBuffer: commandBuffer sourceTexture: lapTex destinationTexture: varianceTexture];
// Run the command buffer on the GPU and wait for the results
[commandBuffer commit];
[commandBuffer waitUntilCompleted];
// The output will be just 2 pixels, one with the mean, the other the variance.
NSMutableData* result = [NSMutableData dataWithLength: 2];
void* resultBytes = result.mutableBytes;
//var result = [Int8](repeatElement(0, count: 2));
MTLRegion region = MTLRegionMake2D(0, 0, 2, 1);
const char* bytes = resultBytes;
NSLog(@"***resultBytes: %d", bytes[0]);
NSLog(@"***resultBytes: %d", bytes[1]);
[varianceTexture getBytes: resultBytes bytesPerRow: 1 * 2 * 4 fromRegion: region mipmapLevel: 0];
NSLog(@"resultBytes: %d", bytes[0]);
NSLog(@"resultBytes: %d", bytes[1]);
int variance = (int)bytes[1];
return variance < 2;
【问题讨论】：
这是一种模糊检测方法：***.com/questions/60587428/… 然而，答案是用 C++ 实现的，你可能会发现它很有用。 谢谢，但是上面的 iOS Metal 代码有什么输入吗？它似乎给了我一个值 1-5，如果模糊，通常为 2，如果不是，则为 2-5，但我想我需要更精细的粒度，为什么上述代码的方差如此之低？您的链接似乎给出了 0-500 的差异 能写出如此精细的快速版本吗？ 【参考方案1】：您的代码暗示您假设一个带有 4 个通道的变量纹理，每个通道一个字节。但是对于您的 varianceTextureDescriptor，您可能希望使用浮点值，这也是由于方差的值范围，请参见下面的代码。此外，您似乎想与 OpenCV 进行比较并具有可比较的值。
无论如何，让我们从 Apple 的 MPSImageLaplacian 文档开始：
此过滤器使用具有 3x3 内核的优化卷积过滤器，其权重如下：
在 Python 中可以这样做，例如喜欢：
import cv2
import np
from PIL import Image
img = np.array(Image.open('forrest.jpg'))
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
laplacian_kernel = np.array([[0, 1, 0], [1, -4, 1], [0, 1, 0]])
print(img.dtype)
print(img.shape)
laplacian = cv2.filter2D(img, -1, laplacian_kernel)
print('mean', np.mean(laplacian))
print('variance', np.var(laplacian, axis=(0, 1)))
cv2.imshow('laplacian', laplacian)
key = cv2.waitKey(0)
请注意，我们完全使用 Apple 文档中给出的值。
它为我的测试图像提供以下输出：
uint8
(4032, 3024)
mean 14.531123203525237
variance 975.6843631756923
MPSImageStatisticsMeanAndVariance
我们现在希望使用 Apple 的 Metal Performance Shader MPSImageStatisticsMeanAndVariance 获得相同的值。
将输入图像转换为灰度图像很有用。然后应用 MPSImageLaplacian 图像内核。
一个字节也只能有 0 到 255 之间的值。因此，对于得到的均值或方差值，我们希望有浮点值。我们可以独立于输入图像的像素格式来指定它。所以我们应该如下使用MTLPixelFormatR32Float：
MTLTextureDescriptor *varianceTextureDescriptor = [MTLTextureDescriptor
texture2DDescriptorWithPixelFormat:MTLPixelFormatR32Float
width:2
height:1
mipmapped:NO];
然后我们想将结果纹理中的 8 个字节解释为两个浮点数。我们可以通过工会很好地做到这一点。这可能如下所示：
union
float f[2];
unsigned char bytes[8];
u1;
MTLRegion region = MTLRegionMake2D(0, 0, 2, 1);
[varianceTexture getBytes:u1.bytes bytesPerRow:2 * 4 fromRegion:region mipmapLevel: 0];
最后，我们需要知道计算是使用介于 0 和 1 之间的浮点值完成的，这实际上意味着我们要乘以 255 或 255*255 以获得方差以使其进入可比较的值范围：
NSLog(@"mean: %f", u1.f[0] * 255);
NSLog(@"variance: %f", u1.f[1] * 255 * 255);
为了完整起见，整个Objective-C代码：
id<MTLDevice> device = MTLCreateSystemDefaultDevice();
id<MTLCommandQueue> queue = [device newCommandQueue];
id<MTLCommandBuffer> commandBuffer = [queue commandBuffer];
MTKTextureLoader *textureLoader = [[MTKTextureLoader alloc] initWithDevice:device];
id<MTLTexture> sourceTexture = [textureLoader newTextureWithCGImage:image.CGImage options:nil error:nil];
CGColorSpaceRef srcColorSpace = CGColorSpaceCreateDeviceRGB();
CGColorSpaceRef dstColorSpace = CGColorSpaceCreateDeviceGray();
CGColorConversionInfoRef conversionInfo = CGColorConversionInfoCreate(srcColorSpace, dstColorSpace);
MPSImageConversion *conversion = [[MPSImageConversion alloc] initWithDevice:device
srcAlpha:MPSAlphaTypeAlphaIsOne
destAlpha:MPSAlphaTypeAlphaIsOne
backgroundColor:nil
conversionInfo:conversionInfo];
MTLTextureDescriptor *grayTextureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatR16Unorm
width:sourceTexture.width
height:sourceTexture.height
mipmapped:false];
grayTextureDescriptor.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id<MTLTexture> grayTexture = [device newTextureWithDescriptor:grayTextureDescriptor];
[conversion encodeToCommandBuffer:commandBuffer sourceTexture:sourceTexture destinationTexture:grayTexture];
MTLTextureDescriptor *textureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:grayTexture.pixelFormat
width:sourceTexture.width
height:sourceTexture.height
mipmapped:false];
textureDescriptor.usage = MTLTextureUsageShaderWrite | MTLTextureUsageShaderRead;
id<MTLTexture> texture = [device newTextureWithDescriptor:textureDescriptor];
MPSImageLaplacian *imageKernel = [[MPSImageLaplacian alloc] initWithDevice:device];
[imageKernel encodeToCommandBuffer:commandBuffer sourceTexture:grayTexture destinationTexture:texture];
MPSImageStatisticsMeanAndVariance *meanAndVariance = [[MPSImageStatisticsMeanAndVariance alloc] initWithDevice:device];
MTLTextureDescriptor *varianceTextureDescriptor = [MTLTextureDescriptor
texture2DDescriptorWithPixelFormat:MTLPixelFormatR32Float
width:2
height:1
mipmapped:NO];
varianceTextureDescriptor.usage = MTLTextureUsageShaderWrite;
id<MTLTexture> varianceTexture = [device newTextureWithDescriptor:varianceTextureDescriptor];
[meanAndVariance encodeToCommandBuffer:commandBuffer sourceTexture:texture destinationTexture:varianceTexture];
[commandBuffer commit];
[commandBuffer waitUntilCompleted];
union
float f[2];
unsigned char bytes[8];
u;
MTLRegion region = MTLRegionMake2D(0, 0, 2, 1);
[varianceTexture getBytes:u.bytes bytesPerRow:2 * 4 fromRegion:region mipmapLevel: 0];
NSLog(@"mean: %f", u.f[0] * 255);
NSLog(@"variance: %f", u.f[1] * 255 * 255);
最终输出给出了与 Python 程序相似的值：
mean: 14.528159
variance: 974.630615
Python 代码和 Objective-C 代码也会为其他图像计算类似的值。
即使没有直接询问，也应该注意方差值当然也非常依赖于主题。如果您有一系列具有相同主题的图像，那么该值肯定是有意义的。为了说明这一点，这里有两个不同的主题的小测试，它们都很清晰，但在方差值上显示出明显的差异：
在上部区域中，您可以看到相应的图像在应用拉普拉斯滤波器后转换为灰色，而在下部区域中。相应的中值或方差值可以在图像的中间看到。
【讨论】：
我试过你的代码，但它不起作用。它崩溃了，[meanAndVariance encodeToCommandBuffer:commandBuffer sourceTexture:texture destinationTexture:varianceTexture]; 在 ___lldb_unnamed_symbol389$$MPSImage () 中的 #8 0x00000001d4a95c94 带有 SIGABRT 在我看到的日志中，detectBlur:Apple 提供了一个示例项目，展示了如何使用 Accelerate 框架find the sharpest image in a sequence of captured images。这可能是一个有用的起点。
【讨论】：
谢谢你的链接，上面的代码有没有cmets？【参考方案3】：斯威夫特版本
需要记住的一些事情。内核大小为 3x3 (MPSImageLaplacian) 的拉普拉斯函数可以根据图像的大小给出一些不同的结果。如果您要比较相似的图像以获得一致的结果，您可能需要将图像缩放为恒定大小。
guard let image = uiImage.cgImage, let device = MTLCreateSystemDefaultDevice(), let queue = device.makeCommandQueue(), let commandBuffer = queue.makeCommandBuffer() else
return
let textureLoader = MTKTextureLoader(device: device)
let sourceColorSpace = CGColorSpaceCreateDeviceRGB()
let destinationColorSpace = CGColorSpaceCreateDeviceGray()
let conversionInfo = CGColorConversionInfo(src: sourceColorSpace, dst: destinationColorSpace)
let conversion = MPSImageConversion(device: device, srcAlpha: MPSAlphaType.alphaIsOne, destAlpha: MPSAlphaType.alphaIsOne, backgroundColor: nil, conversionInfo: conversionInfo)
guard let sourceTexture = try? textureLoader.newTexture(cgImage: image, options: [.origin: MTKTextureLoader.Origin.flippedVertically]) else
return
let grayscaleTextureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.r16Snorm, width: sourceTexture.width, height: sourceTexture.height, mipmapped: false)
grayscaleTextureDescriptor.usage = [.shaderWrite, .shaderRead]
guard let grayscaleTexture = device.makeTexture(descriptor: grayscaleTextureDescriptor) else
return
conversion.encode(commandBuffer: commandBuffer, sourceTexture: sourceTexture, destinationTexture: grayscaleTexture)
let textureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: grayscaleTexture.pixelFormat, width: sourceTexture.width, height: sourceTexture.height, mipmapped: false)
textureDescriptor.usage = [.shaderWrite, .shaderRead]
guard let texture = device.makeTexture(descriptor: textureDescriptor) else
return
let laplacian = MPSImageLaplacian(device: device)
laplacian.encode(commandBuffer: commandBuffer, sourceTexture: grayscaleTexture, destinationTexture: texture)
let meanAndVariance = MPSImageStatisticsMeanAndVariance(device: device)
let varianceTextureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.r32Float, width: 2, height: 1, mipmapped: false)
varianceTextureDescriptor.usage = [.shaderWrite]
guard let varianceTexture = device.makeTexture(descriptor: varianceTextureDescriptor) else
return
meanAndVariance.encode(commandBuffer: commandBuffer, sourceTexture: texture, destinationTexture: varianceTexture)
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
var bytes = [Int8](repeatElement(0, count: 8))
let region = MTLRegionMake2D(0, 0, 2, 1)
varianceTexture.getBytes(&bytes, bytesPerRow: 4 * 2, from: region, mipmapLevel: 0)
var result = [Float32](repeating: 0, count: 2)
memcpy(&result, &bytes, 8)
let mean = Double(result[0] * 255.0)
let variance = Double(result[1] * 255.0 * 255.0)
let standardDeviation = sqrt(variance)
【讨论】：
以上是关于Objective C中iOS的图像模糊检测的主要内容，如果未能解决你的问题，请参考以下文章
从 iOS 中的共享按钮启动应用程序 - Objective C 和 React Native
如何在 iOS Swift 或 Objective C 中检测 iPhone 卷访问 [关闭]