代码之家 › 专栏 › 技术社区 › Mahesha999

在spring boot应用程序中上载1+GB文件时出现“java.lang.OutOfMemory错误:java堆空间”

tomcat spring-boot spring java

Mahesha999 · 技术社区 · 7 年前

我完全按照 this 上传单个文件的教程。我所做的更改如下所述。

我已将以下属性添加到 application.properties

spring.http.multipart.max-file-size=2048MB
spring.http.multipart.max-request-size=2048MB

然后我将以下jvm参数添加到项目运行配置中,以将jvm堆大小限制为2GB,如中所述 this answer :

-Xmx2048m

接下来,我还确保我的spring boot应用程序运行在64位java上 it seems that

但当我试图上传大文件时,它还是给了我同样的错误。

首先我试着上传20MB的文件。成功了。接下来我一直在尝试UbuntuISO,它值1GB,但它一直给我以下异常:

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3236) ~[na:1.8.0_74]
    at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118) ~[na:1.8.0_74]
    at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) ~[na:1.8.0_74]
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153) ~[na:1.8.0_74]
    at org.springframework.util.StreamUtils.copy(StreamUtils.java:128) ~[spring-core-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.util.FileCopyUtils.copy(FileCopyUtils.java:109) ~[spring-core-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.util.FileCopyUtils.copyToByteArray(FileCopyUtils.java:156) ~[spring-core-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile.getBytes(StandardMultipartHttpServletRequest.java:291) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at com.digitate.ignio.spring_boot_hdfs_file_upload.controller.UploadController.singleFileUpload(UploadController.java:73) ~[classes/:na]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_74]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_74]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_74]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_74]
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) ~[tomcat-embed-core-8.5.6.jar:8.5.6]
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) ~[spring-webmvc-4.3.5.RELEASE.jar:4.3.5.RELEASE]
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) ~[tomcat-embed-core-8.5.6.jar:8.5.6]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) ~[tomcat-embed-core-8.5.6.jar:8.5.6]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) ~[tomcat-embed-core-8.5.6.jar:8.5.6]
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) ~[tomcat-embed-websocket-8.5.6.jar:8.5.6]
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) ~[tomcat-embed-core-8.5.6.jar:8.5.6]
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) ~[tomcat-embed-core-8.5.6.jar:8.5.6]
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) ~[spring-web-4.3.5.RELEASE.jar:4.3.5.RELEASE]

目前我已经在机器上安装了8GB内存,其中1.5GB是免费的。当有2+GB的空闲内存时,我也试着运行SpringBoot应用程序。

在调试过程中,我可以看到上传文件的大小确实是1+GB:

file.getBytes() ,它抛出上述异常。

我错过了什么?

2 回复 | 直到 7 年前

RalleYTN 7 年前

处理大文件时,将整个文件加载到内存中永远不是一个好主意。相反,只要读一点,然后把它写在OutputStream上。这里有一个小例子,可以上传你的文件,而每次只使用4KB的内存。

File source = new File("mySourceFile.txt");
File target = new File("myTargetFile.txt");
int readByteCount = 0;
byte[] buffer = new byte[4096];

try(FileInputStream in = new FileInputStream(source);
    FileOutputStream out = new FileOutputStream(target)) {

    while((readByteCount = in.read(buffer)) != -1) {

        out.write(buffer, 0, readByteCount);
    }
}

Charlie 6 年前

对于大文件,最好使用while循环将1024字节读取到缓冲区,然后将缓冲区写入服务器。此代码已通过5GB文件上载进行测试。如果要尝试此测试,则需要相应地调整应用程序属性,即至少5GB

spring.http.multipart.max-file-size=6000MB
spring.http.multipart.max-request-size=6000MB

下面是使用spring框架编写的代码片段。

@Override
public ResponseEntity<?> uploadFile(@RequestParam("file") MultipartFile multipartfile) {
    ResponseEntity<String> response;
    multipartfile.getOriginalFilename();
    byte [] bufferedbytes= new byte[1024];
    File file= new File("/home/david/Music/"+multipartfile.getOriginalFilename());
    FileOutputStream outStream = null;
    int count=0;
    try {
        BufferedInputStream fileInputStream= new BufferedInputStream(multipartfile.getInputStream());
        outStream=new FileOutputStream(file);
        while((count=fileInputStream.read(bufferedbytes))!=-1) {
         outStream.write(bufferedbytes,0,count);

        }
         outStream.close();
    } catch (IOException e) {
         response= new ResponseEntity<String>("File failed to upload"+multipartfile.getOriginalFilename(),HttpStatus.PAYLOAD_TOO_LARGE);

        return response;

    }

     response= new ResponseEntity<String>("File uploaded sucessifully"+multipartfile.getOriginalFilename(),HttpStatus.OK);

    return response;
}