由网友(旺仔小樱桃)分享简介：我试图使用Amazon的SDK（Java）的样品code称为S3TransferProgressSample.java上传大文件到Amazon S3存储（也张贴在这里http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?HLuploadFi...

我试图使用Amazon的SDK（Java）的样品code称为 S3TransferProgressSample.java上传大文件到Amazon S3 存储（也张贴在这里http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?HLuploadFileJava.html).但是，当我试图上传11 GB的文件上传卡住的不同点，并显示错误消息无法上传文件到Amazon S3：无法上传部分：无法toexecute HTTP请求：无缓冲实体内附的要求不能重复 （附screesnhot）。它看起来像发生后IOException异常SDK是无法重试的要求（见下文）。有没有人遇到这样的，什么是最好的做法来解决此问题？任何code是AP preciated

I tried to use Amazon SDK (Java) sample code called S3TransferProgressSample.java to upload large files to Amazon s3 storage (also posted here http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?HLuploadFileJava.html). But when I am trying to upload 11 GB files the upload is getting stuck at different points with the error message "Unable to upload file to Amazon S3: Unable to upload part: Unable toexecute HTTP request: Unbuffered entity enclosing request can not be repeated " (attached screesnhot). It looks like after IOException occurs SDK is not able to retry the request (see below). Does anyone encounter this and what is the best-practise to resolve this? Any code is appreciated

信息：收到成功响应：200，AWS请求ID：   2B66E7669E24DA75 2011年1月15日上午06时44分46秒   com.amazonaws.http.HttpClient执行信息：发送请求：PUT   s3.amazonaws.com /test_file_upload/autogenerated.txt参数：   （uploadId：   m9MqxzD484Ys1nifnX._IzJBGbCFIoT_zBg0xdd6kkZ4TAtmcG0lXQOE.LeiSEuqn6NjcosIQLXJeKzSnKllmw--，型号：1494，） 2011年1月15日上午06时45分10秒   org.apache.commons.httpclient.HttpMethodDirector executeWithRetry   信息：I / O异常（产生java.net.SocketException）处理请求时陷入：将连接复位：套接字写入错误   2011年1月15日上午06点45分10秒   org.apache.commons.httpclient.HttpMethodDirector executeWithRetry   信息：重试请求 2011年1月15日上午6点45分12秒   com.amazonaws.http.HttpClient执行警告：无法执行   HTTP请求：无缓冲实体内附请求不能被   反复。照片2011年1月15日上午06点45分12秒   org.apache.commons.httpclient.HttpMethodDirector executeWithRetry   信息：I / O异常（产生java.net.SocketException）处理请求时陷入：将连接复位：套接字写入错误   2011年1月15日上午06时45分十二秒   org.apache.commons.httpclient.HttpMethodDirector executeWithRetry   信息：重试请求 2011年1月15日上午六时45分13秒   org.apache.commons.httpclient.HttpMethodDirector executeWithRetry   信息：I / O异常（产生java.net.SocketException）处理请求时陷入：将连接复位：套接字写入错误   2011年1月15日上午06点45分13秒   org.apache.commons.httpclient.HttpMethodDirector executeWithRetry   信息：重试请求 2011年1月15日上午六时45分13秒   com.amazonaws.http.HttpClient执行   警告：无法执行HTTP请求：无缓冲实体内附的要求不能重复 2011年1月15号上午六时45分14秒   com.amazonaws.http.HttpClient执行警告：无法执行   HTTP请求：无缓冲实体内附请求不能被   反复。照片2011年1月15日上午06时45分14秒com.amazonaws.http.HttpClient   执行警告：无法执行HTTP请求：无缓冲的实体   封闭请求不能被重复。照片二〇一一年一月十五日上午06时45分14秒   com.amazonaws.http.HttpClient执行警告：无法执行   HTTP请求：无缓冲实体内附请求不能被   反复。照片2011年1月15日上午6点45分15秒com.amazonaws.http.HttpClient   执行警告：无法执行HTTP请求：无缓冲的实体   封闭请求不能被重复。照片二○一一年一月十五日上午06点45分16秒   com.amazonaws.http.HttpClient执行警告：无法执行   HTTP请求：无缓冲实体内附请求不能被   反复。照片2011年1月15日上午06时45分十六​​秒com.amazonaws.http.HttpClient   执行警告：无法执行HTTP请求：无缓冲的实体   封闭请求不能被重复。照片二零一一年一月十五日上午06点45分17秒   com.amazonaws.http.HttpClient执行警告：无法执行   HTTP请求：无缓冲实体内附请求不能被   反复。照片2011年1月15日上午6点45分19秒com.amazonaws.http.HttpClient   执行警告：无法执行HTTP请求：无缓冲的实体   封闭请求不能被重复。照片二零一一年一月十五日上午06点45分19秒   com.amazonaws.http.HttpClient执行 .... 2011年1月15日   上午06时45分21秒com.amazonaws.http.HttpClient用handleResponse   信息：收到成功响应：204，AWS请求ID：E794B8FCA4C3D007 2011年1月15日上午06时45分21秒   com.amazonaws.http.HttpClient执行 ... 2011年1月15日6时45分十九秒   AM com.amazonaws.http.HttpClient执行信息：发送请求：   DELETE s3.amazonaws.com /test_file_upload/autogenerated.txt   参数： ... 2011年1月15日上午06点47分零一秒   com.amazonaws.http.HttpClient handleErrorResponse 信息：收到   错误响应：状态code：404，AWS请求ID：0CE25DFE767CC595，   AWS错误code：NoSuchUpload，AWS错误消息：指定上传   不存在。上载的ID可能是无效的，或上载可以具有   被中止或已完成。

INFO: Received successful response: 200, AWS Request ID: 2B66E7669E24DA75 Jan 15, 2011 6:44:46 AM com.amazonaws.http.HttpClient execute INFO: Sending Request: PUT s3.amazonaws.com /test_file_upload/autogenerated.txt Parameters: (uploadId: m9MqxzD484Ys1nifnX._IzJBGbCFIoT_zBg0xdd6kkZ4TAtmcG0lXQOE.LeiSEuqn6NjcosIQLXJeKzSnKllmw--, partNumber: 1494, ) Jan 15, 2011 6:45:10 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error Jan 15, 2011 6:45:10 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request Jan 15, 2011 6:45:12 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:12 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error Jan 15, 2011 6:45:12 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request Jan 15, 2011 6:45:13 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error Jan 15, 2011 6:45:13 AM org.apache.commons.httpclient.HttpMethodDirector executeWithRetry INFO: Retrying request Jan 15, 2011 6:45:13 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:14 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:14 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:14 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:15 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:16 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:16 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:17 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:19 AM com.amazonaws.http.HttpClient execute WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated. Jan 15, 2011 6:45:19 AM com.amazonaws.http.HttpClient execute .... Jan 15, 2011 6:45:21 AM com.amazonaws.http.HttpClient handleResponse INFO: Received successful response: 204, AWS Request ID: E794B8FCA4C3D007 Jan 15, 2011 6:45:21 AM com.amazonaws.http.HttpClient execute ... Jan 15, 2011 6:45:19 AM com.amazonaws.http.HttpClient execute INFO: Sending Request: DELETE s3.amazonaws.com /test_file_upload/autogenerated.txt Parameters: ... Jan 15, 2011 6:47:01 AM com.amazonaws.http.HttpClient handleErrorResponse INFO: Received error response: Status Code: 404, AWS Request ID: 0CE25DFE767CC595, AWS Error Code: NoSuchUpload, AWS Error Message: The specified upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.

感谢您！

-Alexey

推荐答案

尝试使用底层API 。

这会给你远远更多的控制，当事情出错，因为它们很可能做一个11GB的文件。

This will give you far more control when things go wrong, as they are likely to do with an 11GB file.

要求，并从S3做不成不时。与低级别的API，你就可以如果失败重试上载的一部分。

Requests to and from S3 do fail from time to time. With the low level API, you'll be able to retry a part of the upload if it fails.

重构的例子在亚马逊的文档了一下：

Refactoring the example in the Amazon docs a bit:

// Step 2: Upload parts.
long filePosition = 0;
for (int i = 1; filePosition < contentLength; i++) {
    // Last part can be less than 5 MB. Adjust part size.
    partSize = Math.min(partSize, (contentLength - filePosition));

    // Create request to upload a part.
    UploadPartRequest uploadRequest = new UploadPartRequest()
                .withBucketName(existingBucketName).withKey(keyName)
                .withUploadId(initResponse.getUploadId()).withPartNumber(i)
                .withFileOffset(filePosition)
                .withFile(file)
                .withPartSize(partSize);

    // repeat the upload until it succeeds.
    boolean anotherPass;  
        do {
              anotherPass = false;  // assume everythings ok
              try {
                  // Upload part and add response to our list.
                  partETags.add(s3Client.uploadPart(uploadRequest).getPartETag());
              } catch (Exception e) {
                    anotherPass = true; // repeat
              }
           }
        } while (anotherPass);

     filePosition += partSize;
}

   // Step 3: complete.
   CompleteMultipartUploadRequest compRequest = new 
                     CompleteMultipartUploadRequest(
                                existingBucketName, 
                                keyName, 
                                initResponse.getUploadId(), 
                                partETags);

   s3Client.completeMultipartUpload(compRequest);

注：我不是一个Java开发人员，所以我可以把事情搞乱语法，但希望这可以让你在正确的方向前进。另外，你要在一个重试计数器为prevent添加一个无限循环如果上传失败重复。

Note: I am not a java developer so I could have messed things up syntactically, but hopefully this gets you going in the right direction. Also, you'll want to add in a 'retry counter' to prevent an endless loop if the upload repeatedly fails.