下载文件时内存不足
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了下载文件时内存不足相关的知识,希望对你有一定的参考价值。
我想在android下载一个CSV文件。下载后,我使用Jackson进行映射,然后继续使用sqlite3 db。我面临的问题是当我拨打下载文件的电话时,我得到了一个OutOfMemoryError
。文件大小在12MB
附近。当我在浏览器中尝试相同时,我能够下载该文件,因此我不认为它与端点有关。
要下载,我正在使用AsyncTask。
class DownloadCSVTask extends AsyncTask<Void, String, Void> {
private final Call<ResponseBody> csvDump;
private final Context context;
public DownloadCSVTask(Call<ResponseBody> csvDump, Context context) {
this.csvDump = csvDump;
this.context = context;
}
@Override
protected Void doInBackground(Void... voids) {
try {
Response<ResponseBody> response = csvDump.execute();
File fileFromResponse = createFileFromResponse(response);
mapFileContentsToModelAndPersist(fileFromResponse);
} catch (IOException e) {
Log.e(getClass().getSimpleName(), "CSV pull failed");
}
return null;
}
private File createFileFromResponse(Response<ResponseBody> response) throws IOException {
File path = context.getFilesDir();
File csvFile = File.createTempFile(TEMP_PREFIX, CSV_EXTENSION, path);
FileOutputStream fileOutputStream = context.openFileOutput(csvFile.getName(), Context.MODE_PRIVATE);
fileOutputStream.write(response.body().bytes());
Log.i(getClass().getCanonicalName(), "Writing file completed" + csvFile.getAbsolutePath());
fileOutputStream.close();
return csvFile;
}
private void mapFileContentsToModelAndPersist(File fileFromResponse) {
try {
MappingIterator<City> cityIter = new CsvMapper().readerWithTypedSchemaFor(City.class).readValues(fileFromResponse);
List<City> cities = cityIter.readAll();
Stream.of(cities).forEach(City::persist);
} catch (IOException e) {
Log.e(getClass().getSimpleName(), "Failed to map contents of file to City Class");
}
}
}
我得到的错误日志:
java.lang.RuntimeException: An error occured while executing doInBackground()
E at android.os.AsyncTask$3.done(AsyncTask.java:300)
E at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
E at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
E at java.util.concurrent.FutureTask.run(FutureTask.java:242)
E at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
E at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
E at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
E at java.lang.Thread.run(Thread.java:841)
E Caused by: java.lang.OutOfMemoryError
E at java.lang.String.<init>(String.java:354)
E at java.lang.String.<init>(String.java:393)
E at okio.Buffer.readString(Buffer.java:616)
E at okio.Buffer.readString(Buffer.java:599)
E at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:263)
E at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
E at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
E at com.someclass.somepath.network.util.CustomHeadersInterceptor.intercept(CustomHeadersInterceptor.java:42)
E at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
E at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
E at com.someclass.somepath.network.util.NetworkInterceptor.intercept(NetworkInterceptor.java:23)
E at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
E at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
E at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:170)
E at okhttp3.RealCall.execute(RealCall.java:60)
E at retrofit2.OkHttpCall.execute(OkHttpCall.java:174)
E at retrofit2.ExecutorCallAdapterFactory$ExecutorCallbackCall.execute(ExecutorCallAdapterFactory.java:89)
E at com.someclass.somepath.service.DownloadCSVTask.doInBackground(CsvPullHandler.java:60)
E at com.someclass.somepath.service.DownloadCSVTask.doInBackground(CsvPullHandler.java:47)
E at android.os.AsyncTask$2.call(AsyncTask.java:288)
E at java.util.concurrent.FutureTask.run(FutureTask.java:237)
我在这里做错了吗?还有另一种方法吗?
答案
Android上Java代码的内存量限制为16 Mb(因设备而异)。因此,您需要将此部分移动到本机NDK代码,其中malloc()可以与设备一样多的内存,或者将文件直接保存到磁盘,然后处理它而不将整个文件加载到Java缓冲区中。在带有16Kb字节数组的循环中使用URLConnection.getInputStream()
或URLConnection.read()
,而不是将整个文件复制到response
数组中。或者,您可以使用BufferedReader.readLine()
直接从HTTP流中逐行读取,并逐行填充数据库。
以上是关于下载文件时内存不足的主要内容,如果未能解决你的问题,请参考以下文章