我有一个金属应用程序,我正在尝试将帧导出到QuickTime电影。我在用超高分辨率渲染帧,然后在写之前缩小它们,以消除场景的锯齿。
要缩放它,我将采用高分辨率纹理并将其转换为CGIMAGE,然后调整图像大小并写出较小的版本。我在网上找到了这个将mtltexture转换为cgimage的扩展名:
extension MTLTexture {
func bytes() -> UnsafeMutableRawPointer {
let width = self.width
let height = self.height
let rowBytes = self.width * 4
let p = malloc(width * height * 4)
self.getBytes(p!, bytesPerRow: rowBytes, from: MTLRegionMake2D(0, 0, width, height), mipmapLevel: 0)
return p!
}
func toImage() -> CGImage? {
let p = bytes()
let pColorSpace = CGColorSpaceCreateDeviceRGB()
let rawBitmapInfo = CGImageAlphaInfo.premultipliedFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue // noneSkipFirst
let bitmapInfo:CGBitmapInfo = CGBitmapInfo(rawValue: rawBitmapInfo)
let size = self.width * self.height * 4
let rowBytes = self.width * 4
let releaseMaskImagePixelData: CGDataProviderReleaseDataCallback = { (info: UnsafeMutableRawPointer?, data: UnsafeRawPointer, size: Int) -> () in
// https://developer.apple.com/reference/coregraphics/cgdataproviderreleasedatacallback
// N.B. 'CGDataProviderRelease' is unavailable: Core Foundation objects are automatically memory managed
return
}
if let provider = CGDataProvider(dataInfo: nil, data: p, size: size, releaseData: releaseMaskImagePixelData) {
let cgImageRef = CGImage(width: self.width, height: self.height, bitsPerComponent: 8, bitsPerPixel: 32, bytesPerRow: rowBytes, space: pColorSpace, bitmapInfo: bitmapInfo, provider: provider, decode: nil, shouldInterpolate: true, intent: CGColorRenderingIntent.defaultIntent)!
return cgImageRef
}
return nil
}
} // end extension
我不是肯定的,但是这个函数中的某些东西似乎导致了内存泄漏——它所保存的每一帧都与巨大纹理/cgimage中的内存量有关,并且没有释放它。
cgdataprovider初始化采用“releasedata”回调参数,但我觉得不再需要它。
我也有一个CGIMAGE的扩展——这也可能导致泄漏,我不知道。但是,我可以对帧的大小调整和写入进行注释,并且内存泄漏仍然存在,因此在我看来,到CGIMAGE的转换是主要问题。
extension CGImage {
func resize(_ scale:Float) -> CGImage? {
let imageWidth = Float(width)
let imageHeight = Float(height)
let w = Int(imageWidth * scale)
let h = Int(imageHeight * scale)
guard let colorSpace = colorSpace else { return nil }
guard let context = CGContext(data: nil, width: w, height: h, bitsPerComponent: bitsPerComponent, bytesPerRow: Int(Float(bytesPerRow)*scale), space: colorSpace, bitmapInfo: alphaInfo.rawValue) else { return nil }
// draw image to context (resizing it)
context.interpolationQuality = .high
let r = CGRect(x: 0, y: 0, width: w, height: h)
context.clear(r)
context.draw(self, in:r)
// extract resulting image from context
return context.makeImage()
}
}
最后,这里是导出时调用每个帧的大函数。很抱歉,提供太多信息可能比提供太少信息要好。因此,基本上在渲染开始时,我分配了一个巨大的MTL纹理(“exportTextureBig”),即正常屏幕的大小乘以每个方向上的“缩放子场景”。我将场景渲染成块,每个点对应一个网格,然后使用blitcommandencoder.copy()将每个小块复制到大纹理上,组装大框架。一旦整个帧被填满,然后我尝试从中生成一个CGIMAGE,将其缩小到另一个CGIMAGE,然后写出来。
我在导出时对每一帧调用commandBuffer.waitUntilCompleted(),希望避免让渲染器保留它仍在使用的纹理。
func exportFrame2(_ commandBuffer:MTLCommandBuffer, _ texture:MTLTexture) { // texture is the offscreen render target for the screen-size chunks
if zoom_index < zoom_subdivisions*zoom_subdivisions { // copy screen-size chunk to large texture
if let blitCommandEncoder = commandBuffer.makeBlitCommandEncoder() {
let dx = Int(BigRender.globals_L.displaySize.x) * (zoom_index%zoom_subdivisions)
let dy = Int(BigRender.globals_L.displaySize.y) * (zoom_index/zoom_subdivisions)
blitCommandEncoder.copy(from:texture,
sourceSlice: 0,
sourceLevel: 0,
sourceOrigin: MTLOrigin(x:0,y:0,z:0),
sourceSize: MTLSize(width:Int(BigRender.globals_L.displaySize.x),height:Int(BigRender.globals_L.displaySize.y), depth:1),
to:BigVideoWriter!.exportTextureBig!,
destinationSlice: 0,
destinationLevel: 0,
destinationOrigin: MTLOrigin(x:dx,y:dy,z:0))
blitCommandEncoder.synchronize(resource: BigVideoWriter!.exportTextureBig!)
blitCommandEncoder.endEncoding()
}
}
commandBuffer.commit()
commandBuffer.waitUntilCompleted() // do this instead
// is big frame complete?
if (zoom_index == zoom_subdivisions*zoom_subdivisions-1) {
// shrink the big texture here
if let cgImage = self.exportTextureBig!.toImage() { // memory leak here?
// this can be commented out and memory leak still happens
if let smallImage = cgImage.resize(1.0/Float(zoom_subdivisions)) {
writeFrame(nil, smallImage)
}
}
}
}
除了巨大的内存泄漏,这一切都可以工作。我能做些什么让它释放每帧的CGIMAGE数据吗?为什么它会抓住它?
非常感谢您的建议!
