代码之家  ›  专栏  ›  技术社区  ›  uckelman

为什么BufferedImage需要的内存超过其数据数组的大小?

  •  4
  • uckelman  · 技术社区  · 14 年前

    我正在尝试确定任何给定类型的堆有多少个int argb BufferedImage 这样,对于一个正在进行一些图像处理的程序,我可以根据我们提供的图像大小设置一个合理的最大堆。

    我编写了以下程序作为一个测试,然后用它来确定在没有 OutOfMemoryError :

    import java.awt.image.BufferedImage;
    
    public class Test {
      public static void main(String[] args) {
        final int w = Integer.parseInt(args[0]);
        final int h = Integer.parseInt(args[1]);
    
        final BufferedImage img =
          new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
    
        System.out.println((4*w*h) >> 20);
      }
    }
    

    (打印值是 int[] 其中 缓冲图像 存储的像素数据。)我希望找到的是所需的最大堆类似于 x + c 在哪里 x 是数据数组的大小,并且 c 是由加载的类的大小组成的常量, 缓冲图像 对象等。这是我找到的(所有值均以MB为单位):

    4*w*h   min max heap
    -----   ------------
      5          -
     10         15
     20         31
     40         61
     80        121
    160        241
    

    1.5x 很适合观察。(注意,我没有找到最小的5兆图像。)我不明白我看到了什么。这些额外的字节是什么?

    3 回复  |  直到 6 年前
        1
  •  4
  •   jarnbjo    14 年前

    甲骨文的虚拟机在1.6.0-1.6.0之间引入了一个漏洞。甚至可以减少分配int数组的问题,因为该问题不仅与BufferedImage相关。

    对于1.6.0_16,我需要至少413MB堆来分配一个包含100000000个元素的int数组,这似乎是合理的。对于1.6.0_20,相同的操作至少需要573MB堆空间,尽管在分配数组之后实际只使用了大约400000000字节。

        2
  •  1
  •   Community CDub    8 年前

    在进一步调查中,问题似乎是堆中的旧代无法充分扩展以容纳映像的数据数组,尽管堆中有足够的可用内存。

    有关如何扩展旧一代的详细信息,请参阅 this question .

        3
  •  0
  •   JHead    6 年前

    问题是BufferedImage对象以未还原的格式将图像存储在内存中。有一个有效的解决方案:您可以将映像存储在硬盘上,而不必担心堆大小或物理内存限制。它最多可存储2147483647像素(或46340 x 46340像素)。 大缓冲区图像 解决了这个问题。

    创建az清空bigBufferedImage:

    BigBufferedImage image = BigBufferedImage.create(
            tempDir, width, height, type);
    

    将现有图像加载到BigBufferedImage:

    BigBufferedImage image = BigBufferedImage.create(
            imagePath, tempDir, type);
    

    渲染图像部分:

    part = image.getSubimage(x, y, width, height);
    

    BigBufferedImage的实现:

    package com.pulispace.mc.ui.panorama.util;
    
    /*
     * This class is part of MCFS (Mission Control - Flight Software) a development 
     * of Team Puli Space, official Google Lunar XPRIZE contestant. 
     * This class is released under Creative Commons CC0.
     * @author Zsolt Pocze, Dimitry Polivaev
     * Please like us on facebook, and/or join our Small Step Club.
     * http://www.pulispace.com
     * https://www.facebook.com/pulispace
     * http://nyomdmegteis.hu/en/
     */
    import java.awt.Point;
    import java.awt.Rectangle;
    import java.awt.color.ColorSpace;
    import java.awt.image.BandedSampleModel;
    import java.awt.image.BufferedImage;
    import java.awt.image.ColorModel;
    import java.awt.image.ComponentColorModel;
    import java.awt.image.DataBuffer;
    import java.awt.image.Raster;
    import java.awt.image.RenderedImage;
    import java.awt.image.SampleModel;
    import java.awt.image.WritableRaster;
    import java.io.File;
    import java.io.FileNotFoundException;
    import java.io.IOException;
    import java.io.RandomAccessFile;
    import java.nio.MappedByteBuffer;
    import java.nio.channels.FileChannel;
    import java.util.ArrayList;
    import java.util.HashSet;
    import java.util.Hashtable;
    import java.util.Iterator;
    import java.util.List;
    import java.util.concurrent.Callable;
    import java.util.concurrent.ExecutorService;
    import java.util.concurrent.Executors;
    import java.util.logging.Level;
    import java.util.logging.Logger;
    import javax.imageio.ImageIO;
    import javax.imageio.ImageReadParam;
    import javax.imageio.ImageReader;
    import javax.imageio.stream.ImageInputStream;
    import sun.nio.ch.DirectBuffer;
    
    public class BigBufferedImage extends BufferedImage {
    
        private static final String TMP_DIR = System.getProperty("java.io.tmpdir");
        public static final int MAX_PIXELS_IN_MEMORY =  1024 * 1024;
    
        public static BufferedImage create(int width, int height, int imageType) {
            if (width * height > MAX_PIXELS_IN_MEMORY) {
                try {
                    final File tempDir = new File(TMP_DIR);
                    return createBigBufferedImage(tempDir, width, height, imageType);
                } catch (IOException e) {
                    throw new RuntimeException(e);
                }
            } else {
                return new BufferedImage(width, height, imageType);
            }
        }
    
        public static BufferedImage create(File inputFile, int imageType) throws IOException {
            try (ImageInputStream stream = ImageIO.createImageInputStream(inputFile);) {
                Iterator<ImageReader> readers = ImageIO.getImageReaders(stream);
                if (readers.hasNext()) {
                    try {
                        ImageReader reader = readers.next();
                        reader.setInput(stream, true, true);
                        int width = reader.getWidth(reader.getMinIndex());
                        int height = reader.getHeight(reader.getMinIndex());
                        BufferedImage image = create(width, height, imageType);
                        int cores = Math.max(1, Runtime.getRuntime().availableProcessors() / 2);
                        int block = Math.min(MAX_PIXELS_IN_MEMORY / cores / width, (int) (Math.ceil(height / (double) cores)));
                        ExecutorService generalExecutor = Executors.newFixedThreadPool(cores);
                        List<Callable<ImagePartLoader>> partLoaders = new ArrayList<>();
                        for (int y = 0; y < height; y += block) {
                            partLoaders.add(new ImagePartLoader(
                                y, width, Math.min(block, height - y), inputFile, image));
                        }
                        generalExecutor.invokeAll(partLoaders);
                        generalExecutor.shutdown();
                        return image;
                    } catch (InterruptedException ex) {
                        Logger.getLogger(BigBufferedImage.class.getName()).log(Level.SEVERE, null, ex);
                    }
                }
            }
            return null;
        }
    
        private static BufferedImage createBigBufferedImage(File tempDir, int width, int height, int imageType)
            throws FileNotFoundException, IOException {
            FileDataBuffer buffer = new FileDataBuffer(tempDir, width * height, 4);
            ColorModel colorModel = null;
            BandedSampleModel sampleModel = null;
            switch (imageType) {
                case TYPE_INT_RGB:
                    colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
                        new int[]{8, 8, 8, 0},
                        false,
                        false,
                        ComponentColorModel.TRANSLUCENT,
                        DataBuffer.TYPE_BYTE);
                    sampleModel = new BandedSampleModel(DataBuffer.TYPE_BYTE, width, height, 3);
                    break;
                case TYPE_INT_ARGB:
                    colorModel = new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_sRGB),
                        new int[]{8, 8, 8, 8},
                        true,
                        false,
                        ComponentColorModel.TRANSLUCENT,
                        DataBuffer.TYPE_BYTE);
                    sampleModel = new BandedSampleModel(DataBuffer.TYPE_BYTE, width, height, 4);
                    break;
                default:
                    throw new IllegalArgumentException("Unsupported image type: " + imageType);
            }
            SimpleRaster raster = new SimpleRaster(sampleModel, buffer, new Point(0, 0));
            BigBufferedImage image = new BigBufferedImage(colorModel, raster, colorModel.isAlphaPremultiplied(), null);
            return image;
        }
    
        private static class ImagePartLoader implements Callable<ImagePartLoader> {
    
            private final int y;
            private final BufferedImage image;
            private final Rectangle region;
            private final File file;
    
            public ImagePartLoader(int y, int width, int height, File file, BufferedImage image) {
                this.y = y;
                this.image = image;
                this.file = file;
                region = new Rectangle(0, y, width, height);
            }
    
            @Override
            public ImagePartLoader call() throws Exception {
                Thread.currentThread().setPriority((Thread.MIN_PRIORITY + Thread.NORM_PRIORITY) / 2);
                try (ImageInputStream stream = ImageIO.createImageInputStream(file);) {
                    Iterator<ImageReader> readers = ImageIO.getImageReaders(stream);
                    if (readers.hasNext()) {
                        ImageReader reader = readers.next();
                        reader.setInput(stream, true, true);
                        ImageReadParam param = reader.getDefaultReadParam();
                        param.setSourceRegion(region);
                        BufferedImage part = reader.read(0, param);
                        Raster source = part.getRaster();
                        WritableRaster target = image.getRaster();
                        target.setRect(0, y, source);
                    }
                }
                return ImagePartLoader.this;
            }
        }
    
        private BigBufferedImage(ColorModel cm, SimpleRaster raster, boolean isRasterPremultiplied, Hashtable<?, ?> properties) {
            super(cm, raster, isRasterPremultiplied, properties);
        }
    
        public void dispose() {
            ((SimpleRaster) getRaster()).dispose();
        }
    
        public static void dispose(RenderedImage image) {
            if (image instanceof BigBufferedImage) {
                ((BigBufferedImage) image).dispose();
            }
        }
    
        private static class SimpleRaster extends WritableRaster {
    
            public SimpleRaster(SampleModel sampleModel, FileDataBuffer dataBuffer, Point origin) {
                super(sampleModel, dataBuffer, origin);
            }
    
            public void dispose() {
                ((FileDataBuffer) getDataBuffer()).dispose();
            }
    
        }
    
        private static final class FileDataBufferDeleterHook extends Thread {
    
            static {
                Runtime.getRuntime().addShutdownHook(new FileDataBufferDeleterHook());
            }
    
            private static final HashSet<FileDataBuffer> undisposedBuffers = new HashSet<>();
    
            @Override
            public void run() {
                final FileDataBuffer[] buffers = undisposedBuffers.toArray(new FileDataBuffer[0]);
                for (FileDataBuffer b : buffers) {
                    b.disposeNow();
                }
            }
        }
    
        private static class FileDataBuffer extends DataBuffer {
    
            private final String id = "buffer-" + System.currentTimeMillis() + "-" + ((int) (Math.random() * 1000));
            private File dir;
            private String path;
            private File[] files;
            private RandomAccessFile[] accessFiles;
            private MappedByteBuffer[] buffer;
    
            public FileDataBuffer(File dir, int size) throws FileNotFoundException, IOException {
                super(TYPE_BYTE, size);
                this.dir = dir;
                init();
            }
    
            public FileDataBuffer(File dir, int size, int numBanks) throws FileNotFoundException, IOException {
                super(TYPE_BYTE, size, numBanks);
                this.dir = dir;
                init();
            }
    
            private void init() throws FileNotFoundException, IOException {
                FileDataBufferDeleterHook.undisposedBuffers.add(this);
                if (dir == null) {
                    dir = new File(".");
                }
                if (!dir.exists()) {
                    throw new RuntimeException("FileDataBuffer constructor parameter dir does not exist: " + dir);
                }
                if (!dir.isDirectory()) {
                    throw new RuntimeException("FileDataBuffer constructor parameter dir is not a directory: " + dir);
                }
                path = dir.getPath() + "/" + id;
                File subDir = new File(path);
                subDir.mkdir();
                buffer = new MappedByteBuffer[banks];
                accessFiles = new RandomAccessFile[banks];
                files = new File[banks];
                for (int i = 0; i < banks; i++) {
                    File file = files[i] = new File(path + "/bank" + i + ".dat");
                    final RandomAccessFile randomAccessFile = accessFiles[i] = new RandomAccessFile(file, "rw");
                    buffer[i] = randomAccessFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, getSize());
                }
            }
    
            @Override
            public int getElem(int bank, int i) {
                return buffer[bank].get(i) & 0xff;
            }
    
            @Override
            public void setElem(int bank, int i, int val) {
                buffer[bank].put(i, (byte) val);
            }
    
            @Override
            protected void finalize() throws Throwable {
                dispose();
            }
    
            private void disposeNow() {
                final MappedByteBuffer[] disposedBuffer = this.buffer;
                this.buffer = null;
                disposeNow(disposedBuffer);
            }
    
            public void dispose() {
                final MappedByteBuffer[] disposedBuffer = this.buffer;
                this.buffer = null;
                new Thread() {
                    @Override
                    public void run() {
                        disposeNow(disposedBuffer);
                    }
                }.start();
            }
    
            private void disposeNow(final MappedByteBuffer[] disposedBuffer) {
                FileDataBufferDeleterHook.undisposedBuffers.remove(this);
                if (disposedBuffer != null) {
                    for (MappedByteBuffer b : disposedBuffer) {
                        ((DirectBuffer) b).cleaner().clean();
                    }
                }
                if (accessFiles != null) {
                    for (RandomAccessFile file : accessFiles) {
                        try {
                            file.close();
                        } catch (IOException e) {
                            e.printStackTrace();
                        }
                    }
                    accessFiles = null;
                }
                if (files != null) {
                    for (File file : files) {
                        file.delete();
                    }
                    files = null;
                }
                if (path != null) {
                    new File(path).delete();
                    path = null;
                }
            }
    
        }
    }
    
    推荐文章