AmazonS3-Java对接
1、Amazons3？
S3 是一个全球存储区域网络(SAN)，它表现为一个超大的硬盘，可以存储AWS用户上传的资源文件。S3可根据AWS用户需求不同创建不同区域、不同名称的Bucket（桶）,代码调用的时候可直接根据BucketNamed对Bucket进行访问（前提是AWS凭证认证通过）。
2、基础配置
1、AWS用户凭证设置
AWS用户凭证有一下集中方式可配置：
1：LocalFile（本地配置文件）
2：Java.profile/.java文件中设置变量
3：配置环境变量（暂未测试）
1、LoaclFile
1、AWS官网查看用户Id/Key:
2、Windows设置：
指定目录下创建文件夹（.aws)及下属文件（config/credentials)
config文件：
区域可根据自己需求（以及服务所在区域）设置
credentials文件：
3：CLI设置凭证
2、pom.xml配置
只需要引入如下依赖即可，版本可根据自行需求设定
<dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk-s3</artifactId> <version>1.11.759</version> </dependency>
3、代码示例
1、AmazonS3连接（正常连接）
package com.stefanie.sun.bean.AWS.Rekognition; import com.amazonaws.AmazonClientException; import com.amazonaws.AmazonServiceException; import com.amazonaws.ClientConfiguration; import com.amazonaws.Protocol; import com.amazonaws.auth.AWSCredentials; import com.amazonaws.auth.AWSStaticCredentialsProvider; import com.amazonaws.auth.BasicAWSCredentials; import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; import com.amazonaws.auth.profile.ProfileCredentialsProvider; import com.amazonaws.regions.Regions; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3Client; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.*; import com.amazonaws.util.IOUtils; import lombok.extern.slf4j.Slf4j; import org.springframework.mock.web.MockMultipartFile; import org.springframework.web.multipart.MultipartFile; import java.io.File; import java.io.FileInputStream; import java.io.IOException; import java.util.List; /** * Created with IntelliJ IDEA * StefanieSun * * @Description: AWS S3桶连接 * @Author: zy * @Date: 2020-06-28 15:51 * @Version: 1.0.0 */ @Slf4j public class S3client { private final static String REGION = "us-east-2"; private final static String ACCESSKEY = "your id"; private final static String SECREYKEY = "your key"; public static void main(String[] s) throws Exception { AmazonS3 s3 = amazonS31(); try { //遍历bucket信息 for (Bucket bucket : s3.listBuckets()) { System.out.println("bucker名称:" + bucket.getName() + "/bucker创建时间：" + bucket.getCreationDate() + "/bucker-owner:" + bucket.getOwner()); } } catch (AmazonClientException ace) { log.error(ace.getMessage()); } } //1-读取默认配置连接AmazonS3 public static AmazonS3 amazonS31() { AWSCredentials credentials = null; try { credentials = new ProfileCredentialsProvider("default").getCredentials(); } catch (Exception e) { throw new AmazonClientException("Exception on credentials.", e); } AmazonS3 s3 = new AmazonS3Client(credentials); return s3; } //2-默认连接方式 public static AmazonS3 amazonS32() { AmazonS3 s3 = AmazonS3ClientBuilder.defaultClient(); return s3; } //3-手动设置区域连接 public static AmazonS3 amazonS33(){ AWSCredentials credentials = null; try { //获取凭证信息 credentials = new ProfileCredentialsProvider().getCredentials(); } catch (Exception e) { throw new AmazonClientException( "Cannot load the credentials from the credential profiles file. " + "Please make sure that your credentials file is at the correct " + "location (~/.aws/credentials), and is in valid format.", e); } //通过美国区域设置获取 AmazonS3 s3 = AmazonS3ClientBuilder.standard() .withCredentials(new AWSStaticCredentialsProvider(credentials)) .withRegion(REGION) .build(); return s3; } //4-使用S3客户端创建连接 public static AmazonS3 amazonS34(){ AmazonS3 s3 = new AmazonS3Client(new DefaultAWSCredentialsProviderChain()); return s3; } //5-使用Java.profile/.Java变量连接 public static AmazonS3 amazonS35(){ BasicAWSCredentials awsCreds= new BashicAWSCredentials(ACCESSKEY,SECRETKEY); AmazonS3 s3 = AmazonS3ClientBuilder.standard() .withCredentials(new AWSStaticCredentialsProvider(awsCreds)) .withRegion(Region.US-EASE-2) .build(); return s3; } }
2、异常区域无法访问
1、错误信息
场景：在使用 "us-east-1"区域访问的时候无法连接，其他区域均可正常连接
java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76) at com.amazonaws.http.conn.$Proxy2.connect(Unknown Source) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1323) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5054) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5000) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4994) at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:993) at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:999) at com.stefanie.sun.bean.AWS.Rekognition.S3client.main(S3client.java:49) Caused by: java.net.UnknownHostException: s3.amazonaws.com at java.net.InetAddress.getAllByName0(InetAddress.java:1280) at java.net.InetAddress.getAllByName(InetAddress.java:1192) at java.net.InetAddress.getAllByName(InetAddress.java:1126) at com.amazonaws.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:27) at com.amazonaws.http.DelegatingDnsResolver.resolve(DelegatingDnsResolver.java:38) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ... 28 common frames omitted 10:14:49.175 [main] DEBUG org.apache.http.impl.conn.DefaultManagedHttpClientConnection - http-outgoing-3: Shutdown connection 10:14:49.175 [main] DEBUG org.apache.http.impl.execchain.MainClientExec - Connection discarded 10:14:49.175 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 3][route: {s}->https://s3.amazonaws.com:443][total available: 0; route allocated: 0 of 50; total allocated: 0 of 50] 10:14:49.175 [main] ERROR com.stefanie.sun.bean.AWS.Rekognition.S3client - Unable to execute HTTP request: s3.amazonaws.com Process finished with exit code 0
2、原因分析
1：需要调用的服务在US-EAST-1区，Bucket也在US-EAST-1区，但是连接时无法访问，提示如上所示为请求有问题
2：通过日志比对访问成功时与访问失败的日志可以看出https://------后面的连接有问题，如下图所示：
由上图可以看出https请求地址不同。
3：在找出时http请求地址有问题的情况下，查看源代码，找出问题，但是并未找到哪里设置有问题。
4：在源码中查出AmazonS3.endpoint属性存储http请求地址，故更改属性即可。
5：使用原有生成AmazonS3的方式无法更改属性（在构建的时候无法更改属性）。
3、解决办法
public static AmazonS3 amazonS36() { AWSCredentials credentials = new BasicAWSCredentials(ACCESSKEY, SECRETKEY); ClientConfiguration clientConfig = new ClientConfiguration(); clientConfig.setProtocol(Protocol.HTTP); clientConfig.withSignerOverride("S3SignerType"); //noinspection deprecation AmazonS3 s3 = new AmazonS3Client(credentials, clientConfig); s3.setEndpoint("https://s3.us-east-1.amazonaws.com");//访问地址类似，只是区别与用户凭证的验证 return s3; }
4、上传图片
/** * 上传文件 * * @param filePath 本地文件路径 * @param bucketName 桶名称 * @param keyName 桶路径以及名称 实例："assets/test/1513132153/1.png"; * @throws IOException */ private static void uploadFileToBucket(String filePath, String bucketName, String keyName) throws IOException { AmazonS3 s3Client = amazonS36(); File imageFile = new File(filePath); FileInputStream in = new FileInputStream(imageFile); MultipartFile file = new MockMultipartFile("file", imageFile.getName(), "text/plain", IOUtils.toByteArray (in)); ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentType(file.getContentType()); metadata.setContentLength(file.getSize()); try { s3Client.putObject(new PutObjectRequest( bucketName, keyName, file.getInputStream(), metadata)); } catch (AmazonServiceException ase) { ase.getMessage(); } }
5、查看Bucket中资源
/** * 获取s3桶中数据 */ public static void getS3Object(String bucketName) { AmazonS3 amazonS3 = amazonS36(); ListObjectsV2Result result = amazonS3.listObjectsV2(bucketName); List<S3ObjectSummary> objects = result.getObjectSummaries(); for (S3ObjectSummary os : objects) { System.out.println("* " + os.getKey()); } }