代码之家 › 专栏 › 技术社区 › Carlos Scheidegger

从CMSampleBufferRef创建的UIImage是否未显示在UIImageView中?

avfoundation ios4 uiimage uiimageview iphone

23

Carlos Scheidegger · 技术社区 · 16 年前

AVCaptureVideoDataOutputSampleBufferDelegate 必须实施

- (void)captureOutput:(AVCaptureOutput *)captureOutput 
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer 
       fromConnection:(AVCaptureConnection *)connection
{ 
    // Create a UIImage from the sample buffer data
    UIImage *theImage = [self imageFromSampleBuffer:sampleBuffer];
//    NSLog(@"Got an image! %f %f", theImage.size.width, theImage.size.height);
//    NSLog(@"The image view is %@", imageView);
//    UIImage *theImage = [[UIImage alloc] initWithData:[NSData 
//        dataWithContentsOfURL:[NSURL 
//        URLWithString:@"http://farm4.static.flickr.com/3092/2915896504_a88b69c9de.jpg"]]];
    [self.session stopRunning];
    [imageView setImage: theImage];
}

要解决容易出现的问题:

使用UIImagePickerController不是一个选项(最终我们将实际处理图像)
我知道我已经正确的设置了。如果我使用上面的注释代码从web加载任意图像,而不是简单地发送 setImage:theImage 对于imageView,映像已正确加载(第二次调用NSLog时报告一个非nil对象)。
至少在基本程度上,我从 imageFromSampleBuffer: 很好,因为NSLog报告的大小是360x480,这是我预期的大小。

我使用的代码是最近发布的 AVFoundation 苹果提供的代码片段 here .

特别是,这是我用来设置 AVCaptureSession imageFromSampleBuffer 方法)。

drawInRect: 使用 UIImage imageFromSamplerBuffer ,而如果我使用 UIImage公司

#0  0x34a977ee in decode_swap ()
#1  0x34a8f80e in decode_data ()
#2  0x34a8f674 in img_decode_read ()
#3  0x34a8a76e in img_interpolate_read ()
#4  0x34a63b46 in img_data_lock ()
#5  0x34a62302 in CGSImageDataLock ()
#6  0x351ab812 in ripc_AcquireImage ()
#7  0x351a8f28 in ripc_DrawImage ()
#8  0x34a620f6 in CGContextDelegateDrawImage ()
#9  0x34a61fb4 in CGContextDrawImage ()
#10 0x321fd0d0 in -[UIImage drawInRect:blendMode:alpha:] ()
#11 0x321fcc38 in -[UIImage drawInRect:] ()

编辑:下面是关于该位代码返回的UIImage的更多信息。

使用所描述的方法 here ,我可以得到像素并打印它们,它们乍一看看起来很正常(例如,alpha通道中的每个值都是255)。不过,缓冲区大小有点不一致。我从Flickr上得到的图片是375x500,它是 [pixelData length] 750000 = 375*500*4 imageFromSampleBuffer: 有大小 691208 = 360*480*4 + 8 ,因此像素数据中有8个额外的字节。 CVPixelBufferGetDataSize 额外的

5 回复 | 直到 9 年前

1

41

Ken Pletzer 15 年前

我也有同样的问题。。。但是我发现了这篇老文章,以及它创建CGImageRef的方法!

http://forum.unity3d.com/viewtopic.php?p=300819

下面是一个工作示例:

app has a member UIImage theImage;

// Delegate routine that is called when a sample buffer was written
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer  fromConnection:(AVCaptureConnection *)connection
{
        //... just an example of how to get an image out of this ...

    CGImageRef cgImage = [self imageFromSampleBuffer:sampleBuffer];
    theImage.image =     [UIImage imageWithCGImage: cgImage ];
    CGImageRelease( cgImage );
}

- (CGImageRef) imageFromSampleBuffer:(CMSampleBufferRef) sampleBuffer // Create a CGImageRef from sample buffer data
{
    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 
    CVPixelBufferLockBaseAddress(imageBuffer,0);        // Lock the image buffer 

    uint8_t *baseAddress = (uint8_t *)CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0);   // Get information of the image 
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); 
    size_t width = CVPixelBufferGetWidth(imageBuffer); 
    size_t height = CVPixelBufferGetHeight(imageBuffer); 
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); 

    CGContextRef newContext = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); 
    CGImageRef newImage = CGBitmapContextCreateImage(newContext); 
    CGContextRelease(newContext); 

    CGColorSpaceRelease(colorSpace); 
    CVPixelBufferUnlockBaseAddress(imageBuffer,0); 
    /* CVBufferRelease(imageBuffer); */  // do not call this!

    return newImage;
}

2

10

matt 14 年前

现在,苹果的技术问答很好地解释了视频帧的实时捕获;QA1702:

https://developer.apple.com/library/ios/#qa/qa1702/_index.html

3

7

Vladimir 14 年前

[videoDataOutput setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA] forKey:(NSString*)kCVPixelBufferPixelFormatTypeKey]];

4

5

sroske 13 年前

good write up 如何做到这一点。

我正在使用 his example app imageFromSamplerBuffer 函数使用CGBitmapContextCreate创建CGImageRef,他使用的是主调度队列(通过 dispatch_get_main_queue() )设置输出样本缓冲区委托时。这不是最好的解决方案,因为它需要一个串行队列,而且据我所知,主队列不是串行队列。所以,虽然你不能保证得到正确的顺序帧似乎为我工作到目前为止:)

5

1

Eric 11 年前

另一个要注意的是你是否真的在更新你的 UIImageView 在主线上:如果你不是,很可能它不会反映任何变化。

captureOutput:didOutputSampleBuffer:fromConnection 委托方法通常在后台线程上调用。所以你想这样做:

- (void)captureOutput:(AVCaptureOutput *)captureOutput
didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer
       fromConnection:(AVCaptureConnection *)connection
{   
    CFRetain(sampleBuffer);

    [[NSOperationQueue mainQueue] addOperationWithBlock:^{

        //Now we're definitely on the main thread, so update the imageView:
        UIImage *capturedImage = [self imageFromSampleBuffer:sampleBuffer];

        //Display the image currently being captured:
        imageView.image = capturedImage;

        CFRelease(sampleBuffer);
    }];
}