猿创征文｜技术成长之路-Java编程系列之文件OSS存储实践：Amazon S3实现文件上传下载
Posted 善良勤劳勇敢而又聪明的老杨
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了猿创征文｜技术成长之路-Java编程系列之文件OSS存储实践：Amazon S3实现文件上传下载相关的知识，希望对你有一定的参考价值。
热门系列：
【Java编程系列】Minio实现文件上传下载
1、前言
最近有一客户，需要独立部署文件存储到自己的私有化服务器，于是选择了让我们使用在AWS服务器的s3存储桶方式来实现。现做一下实践后的开发经验记录~~~
2、Amazon S3实战代码
2.1 S3的Sdk对接实现
2.1.1 S3 Maven依赖
这里需要说明一下，目前封装好的java可用的sdk有两种实现依赖，第一种：
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.12.276</version>
</dependency>以上是第一种封装依赖，但是这里面会有部分依赖缺失，在文末的总结中，可以看到完整的依赖方式。
第二种封装依赖：
<!--amazon s3-->
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>2.17.100</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>kms</artifactId>
<version>2.17.100</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3control</artifactId>
<version>2.17.100</version>
</dependency>本来是使用第一种依赖来做的开发编码。后来因为一个403权限拒绝原因，一开始不确定是否是api问题，所以后来又使用了s3官方文档推荐的第二种依赖完成了开发实现，因此以下示例代码，全都是引用第二种maven依赖来完成的！！！
2.1.2 s3 client配置与初始化
在开始实践编码前，先列一下本次开发中，我参考的一些api文档，希望能给有需要的朋友一些参考和帮助。
S3官方开发指南文档：使用 AWS SDK for Java 的 Amazon S3示例 - AWS SDK for Java
开发示例代码github地址：aws-doc-sdk-examples/javav2/example_code/s3/src/main/java/com/example/s3 at main · awsdocs/aws-doc-sdk-examples · GitHub
还有第一种依赖的实现参考代码：如何使用AWS SDK for Java操作MinIO Server | Minio中文文档
s3存储桶的配置信息，我这边是使用数据表存储配置的，因为我们系统的多元化接入。可能会有多套配置，所以请按需配置。
初始化代码如下，PrivateUploadConfig类是我的s3配置信息实体类：
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.core.waiters.WaiterResponse;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadBucketRequest;
import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
import software.amazon.awssdk.services.s3.waiters.S3Waiter;
import javax.annotation.Resource;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Duration;
import java.util.Objects;
/**
* @Date: 2022/8/19 19:40
* @Description
*/
@Component
@Slf4j
public class S3ClientUtil
private S3Client s3Client;
// private S3AsyncClient s3AsyncClient;
@Resource(name = CommonBeanNames.STRING_REDIS_HELPER)
private RedisHelper redisHelper;
/**
* @Description
* @Date 19:53 2022/8/19
* @Param [uploadConfig=s3配置信息实体类]
* @return software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain
**/
private AwsCredentialsProviderChain getProvider(PrivateUploadConfig uploadConfig)
return AwsCredentialsProviderChain
.builder()
.addCredentialsProvider(new AwsCredentialsProvider()
@Override
public AwsCredentials resolveCredentials()
return AwsBasicCredentials.create(uploadConfig.getAccessKey(), uploadConfig.getSecretKey());
).build();
/**
* @return void
* @Description
* @Date 19:47 2022/8/19
* @Param [uploadConfig=s3配置信息实体类]
**/
private void init(PrivateUploadConfig uploadConfig)
if (null != s3Client)
return;
s3Client = S3Client.builder()
.credentialsProvider(getProvider(uploadConfig))
.region(Region.of(uploadConfig.getRegion()))
.build();
/**
* @Description
* @Date 19:53 2022/8/19
* @Param [bucket=存储桶名称]
* @return void
**/
public void makeBucket(String bucket)
S3Waiter s3Waiter = s3Client.waiter();
CreateBucketRequest bucketRequest = CreateBucketRequest.builder()
.bucket(bucket)
.build();
s3Client.createBucket(bucketRequest);
HeadBucketRequest bucketRequestWait = HeadBucketRequest.builder()
.bucket(bucket)
.build();
// Wait until the bucket is created and print out the response.
WaiterResponse<HeadBucketResponse> waiterResponse = s3Waiter.waitUntilBucketExists(bucketRequestWait);
waiterResponse.matched().response().ifPresent(r->log.info(r.toString()));
2.1.3 上传文件
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.core.waiters.WaiterResponse;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.HeadBucketRequest;
import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
import software.amazon.awssdk.services.s3.waiters.S3Waiter;
import javax.annotation.Resource;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Duration;
import java.util.Objects;
/**
* @Date: 2022/8/19 19:40
* @Description
*/
@Component
@Slf4j
public class S3ClientUtil
private S3Client s3Client;
// private S3AsyncClient s3AsyncClient;
@Resource(name = CommonBeanNames.STRING_REDIS_HELPER)
private RedisHelper redisHelper;
public void uploadFile(PrivateUploadConfig uploadConfig, String fileName, MultipartFile multipartFile, String contentType)
throws IOException
if (null == s3Client)
init(uploadConfig);
PutObjectRequest.Builder putOb = PutObjectRequest.builder();
putOb.bucket(uploadConfig.getBucket());
putOb.key(fileName);
if (StringUtils.isNotEmpty(contentType))
putOb.contentType(contentType);
long begin = System.currentTimeMillis();
s3Client.putObject(putOb.build(), RequestBody.fromInputStream(multipartFile.getInputStream(), multipartFile.getInputStream().available()));
log.info("S3 putObject time :", System.currentTimeMillis() - begin);
2.1.4 下载文件
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
import javax.annotation.Resource;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Duration;
import java.util.Objects;
/**
* @Date: 2022/8/19 19:40
* @Description
*/
@Component
@Slf4j
public class S3ClientUtil
private S3Client s3Client;
// private S3AsyncClient s3AsyncClient;
private InputStream amazonS3Download(PrivateUploadConfig uploadConfig, String fileName)
if (null == s3Client)
init(uploadConfig);
//此处需要注意，如果fileName第一位有/，则需要剔除掉，否则会报错找不到
if (fileName.indexOf("/") == 0)
fileName = fileName.replaceFirst("/", "");
GetObjectRequest objectRequest = GetObjectRequest
.builder()
.key(fileName)
.bucket(uploadConfig.getBucket())
.build();
long begin = System.currentTimeMillis();
InputStream is = s3Client.getObject(objectRequest);
log.info("S3 getObject time :", System.currentTimeMillis() - begin);
return is;
// byte [] bytes = s3AsyncClient.getObject(objectRequest,AsyncResponseTransformer.toBytes()).thenApply(BytesWrapper::asByteArray).join();
// return new ByteArrayInputStream(bytes);
2.1.5 生成文件预签名url
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentials;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;
import javax.annotation.Resource;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Duration;
import java.util.Objects;
/**
* @Date: 2022/8/19 19:40
* @Description
*/
@Component
@Slf4j
public class S3ClientUtil
private S3Client s3Client;
// private S3AsyncClient s3AsyncClient;
@Resource(name = CommonBeanNames.STRING_REDIS_HELPER)
private RedisHelper redisHelper;
public String getObjectURL(PrivateUploadConfig uploadConfig, String fileName)
if (fileName.indexOf("/") == 0)
fileName = fileName.replaceFirst("/", "");
S3Presigner presigner = S3Presigner.builder()
.credentialsProvider(getProvider(uploadConfig))
.region(Region.of(uploadConfig.getRegion()))
.build();
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(uploadConfig.getBucket())
.key(fileName)
.build();
//默认生成的预览url最大时效时间为：7天，aws s3能允许的最大时限也是7天
GetObjectPresignRequest getObjectPresignRequest = GetObjectPresignRequest.builder()
.signatureDuration(Duration.ofDays(7))
.getObjectRequest(getObjectRequest)
.build();
PresignedGetObjectRequest presignedGetObjectRequest = presigner.presignGetObject(getObjectPresignRequest);
String theUrl = presignedGetObjectRequest.url().toString();
log.info("Origin path : , Presigned URL: ", fileName, theUrl);
return theUrl;
2.1.6 文件“永久性”访问实现
其实S3存储桶，通过策略配置，是可以直接通过 域名+桶名称+文件路径名称 ，来直接永久访问到的。但是，这么做的话，安全性就大大降低了。所以，我们需要采取其他的方式来间接实现永久访问的功能。于是，有了一下曲线救国的方针：
通过定时任务，根据你生成预签名url的最大时效来，往后推一点时间，来生成需要更新的时间周期。
例如，我选择最大时效为7天，那么我会定时任务，每天查询图片url时效已达过6天的记录，然后重新生成新的url更新到记录里，如此，就可以间接性的永久有效啦~~~
2.1.7 其他操作（解决下载慢）
还有很多其他api，如删除文件、获取存储桶列表、获取文件列表；另外还有一些，针对不同region或是高版本api，还有transfer等高效高性能方法可用。
另外，使用第一种依赖，还能实现一些分段上传、下载。官方建议，当单个文件大小超过100M时，则需要使用分段上传下载处理。
但是，我这里遇到了一个其他问题：上传和下载很慢的问题 ，这个问题我分析主要是来自于网络传输效率低所导致，不在代码层面。测试用例：一个10M左右的文件，上传需要10+秒，下载更是达到60秒左右。
得出以上结论的原因有：
1、同样的s3实现，我们其他使用aws服务器的朋友，他们使用上传下载时，就没有这个慢的情况出现，至少不会像我这个情况。
2、我使用本地文件，对代码进行测试，除了s3的api调用处，其他代码运行效率都很高，只在毫秒级别，使用的是同一个大小的原文件测试的。
3、使用断点调试时，发现s3Client.putObject(）方法，阻塞所占时间为主要耗时；另外，对于s3Client.getObject(）下载方法，虽然程序调用在毫秒级，但最终将文件流输出时，底层实际是通过s3服务端的url实时传输的，所以也是传出效率低所导致！
最终，我先想了一个方式，解决下载慢的问题。因为一个文件下载要1min多，实在是太不能接受了。上传10s左右，还得过且过吧。
对于下载慢的问题，我使用的方式时，在上传完成后，使用redis缓存文件流来解决！
String fis = new String(multipartFile.getBytes(),"ISO8859-1");
redisHelper.hset(cacheKey,fileId,fis);以上是缓存实现的关键代码。对于为什么字符串编码选择使用 ISO8859-1 ，因为也有与base64转码做过性能对比，发现前者的性能更优。大家可以参考使用！
3、总结
整体实践完成后，有一些开发过程中遇到的问题，在这里总结一下。帮助大家避坑~~~
3.1 异常报错
错误一：关键依赖类缺失（EndpointToRegion类找不到）
运行时异常主要信息为：
Caused by: java.lang.ClassNotFoundException: com.amazonaws.regions.EndpointToRegion
全部报错信息如下：
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: com/amazonaws/regions/EndpointToRegion
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1055)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at ***.GzipFilter.doFilter(GzipFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at ***.AbstractCorsFilter.doFilter(AbstractCorsFilter.java:63)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at ***.ChannelFilter.doFilter(ChannelFilter.java:33)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: com/amazonaws/regions/EndpointToRegion
at com.amazonaws.services.s3.internal.auth.S3SignerProvider.getSigner(S3SignerProvider.java:62)
at com.amazonaws.http.ExecutionContext.getSigner(ExecutionContext.java:135)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor$ExecOneRequestParams.newSigner(AmazonHttpClient.java:1843)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1216)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1074)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:4062)
at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1278)
at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1268)
at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1406)
at ***.***.PrivateMinioClientHelper.makeBucket(PrivateMinioClientHelper.java:56)
at ***.***.PrivateMinioClientHelper.amazonS3Init(PrivateMinioClientHelper.java:140)
at ***.***.PrivateMinioClientHelper.init(PrivateMinioClientHelper.java:110)
at ***.***.PrivateMinioClientHelper.amazonS3Upload(PrivateMinioClientHelper.java:163)
at ***.***.PrivateMinioClientHelper.uploadFile(PrivateMinioClientHelper.java:72)
at ***.***.service.impl.PrivateMinioserviceImpl.uploadFile(PrivateMinioServiceImpl.java:79)
at ***.***.service.impl.AbstractFileServiceImpl.upload(AbstractFileServiceImpl.java:269)
at ***.***.controller.FileController.upload(FileController.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
... 52 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.amazonaws.regions.EndpointToRegion
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 88 common frames omitted解决方式：因为缺少部分依赖，只需要引入以下maven依赖即可
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.12.276</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-kms</artifactId>
<version>1.12.276</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-core -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-core</artifactId>
<version>1.12.276</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.amazonaws/jmespath-java -->
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>jmespath-java</artifactId>
<version>1.12.276</version>
</dependency>错误二：权限配置问题，拒绝访问
另一个运行异常报错信息如下：
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 7GR03AT1BWSMPNYM; S3 Extended Request ID: 8fu4lS0r0N6hNhKQbqunAnkmx3HxxRwjqA9oHobv61P7U8vYra23L4MuNlfC6sKjdFr73U1EmX4=; Proxy: null)
整体错误日志如下：
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 7GR03AT1BWSMPNYM; S3 Extended Request ID: 8fu4lS0r0N6hNhKQbqunAnkmx3HxxRwjqA9oHobv61P7U8vYra23L4MuNlfC6sKjdFr73U1EmX4=; Proxy: null)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5456)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5403)
at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:421)
at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6531)
at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1861)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1821)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1753)
at ***.helper.PrivateMinioClientHelper.amazonS3Upload(PrivateMinioClientHelper.java:174)
at ***.helper.PrivateMinioClientHelper.uploadFile(PrivateMinioClientHelper.java:73)
at ***.service.impl.PrivateMinioServiceImpl.uploadFile(PrivateMinioServiceImpl.java:79)
at ***.service.impl.AbstractFileServiceImpl.upload(AbstractFileServiceImpl.java:269)
at ***.controller.FileController.upload(FileController.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at ***.GzipFilter.doFilter(GzipFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at ***.AbstractCorsFilter.doFilter(AbstractCorsFilter.java:63)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at ***.ChannelFilter.doFilter(ChannelFilter.java:33)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2022-08-10 10:37:52.535 [5m7a7kaVC5xwggeT] ERROR (GlobalExceptionHandler.java:135) - custom exception: code: 30006, message: null, data:null, Exception: ***.service.impl.PrivateMinioServiceImpl.uploadFile(PrivateMinioServiceImpl.java:89)
***.service.impl.AbstractFileServiceImpl.upload(AbstractFileServiceImpl.java:269)
***.controller.FileController.upload(FileController.java:108)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)这个问题，提示已经很明显了，就是s3服务端的拒绝通过啦。当时，还一阵以为少了什么配置所致，其实不然，就是权限配置的问题。
解决方式：1、将IAM权限设置好，对指定账户放开哪些权限，如读写等等
2、将指定存储桶的权限放开，保证该桶对指定用户可以上传、下载、删除等等
分享内容就到这里啦，欢迎大家留言探讨。有问题，也会第一时间回复~~
以上是关于猿创征文｜技术成长之路-Java编程系列之文件OSS存储实践：Amazon S3实现文件上传下载的主要内容，如果未能解决你的问题，请参考以下文章
猿创征文｜技术成长之路-Java编程系列文件存储实践：Amazon S3实现文件上传下载，总结坑点，积累成长经验
猿创征文｜弃文从工，从小白到蚂蚁工程师，我的 Java 成长之路
猿创征文｜弃文从工，从小白到蚂蚁工程师，我的 Java 成长之路