CopyObject in SDK V2- how to specify the copy version

[damogallagher]

Expected Behavior

In version 1 of the sdk - you could specify the following code to perform a copy on a particular version of a file

 CopyObjectRequest copyObjectRequest =new CopyObjectRequest(fromBucketName, fromObjectKey, toBucketName, toObjectKey).withSourceVersionId(versionId);

s3Client.copyObject(copyObjectRequest);

Current Behavior

I cant seem to figure out with version 2 of the SDK how to specify the version when performing a copy

Possible Solution

Is it possible to specify a version on the copyObject command? If so could the documentation be updated with the correct syntax to use

Your Environment

• AWS Java SDK version used: 2.10.65
• JDK version used: Amazon 11 Corretto
• Operating System and version: Mac OS Catalina 10.15.3

[debora-ito]

Hi @damogallagher, to copy a specified version of an object you can pass the version id in the CopyObjectRequest copySource:

/bucket/my-image.jpg?versionId=<myVersionId>

We've requested the documentation team to add this info to the docs some time ago, I'll ask them again.

[damogallagher]

Does the versionId need to be encoded?
I have the following code

  try {
            String encodedUrl = null;
            try {
                if (StringUtils.isEmpty(versionId)) {
                    encodedUrl = URLEncoder.encode(fromBucketName + "/" + fromObjectKey, StandardCharsets.UTF_8.toString());
                } else {
                    String url = fromBucketName + "/" + fromObjectKey + "?versionId=" + versionId;
                     encodedUrl = URLEncoder.encode(url, StandardCharsets.UTF_8.toString());
                }
                logger.debug("encodedUrl:{}", encodedUrl);
            } catch (UnsupportedEncodingException e) {
                logger.error("An UnsupportedEncodingException has occured. Exception is: {}", e.getMessage());
                return fileCopied;
            }

            CopyObjectRequest copyObjectRequest = CopyObjectRequest.builder()
                    .copySource(encodedUrl)
                    .destinationBucket(toBucketName)
                    .destinationKey(toObjectKey)
                    .build();

            CopyObjectResponse copyObjectResponse = s3Client.copyObject(copyObjectRequest);

            if (copyObjectResponse == null) {
                logger.error("CopyObject response is null");
                fileCopied = false;
            } else {
                logger.debug("copySourceVersionId:{}, versionId:{}", copyObjectResponse.copySourceVersionId(), copyObjectResponse.versionId());
                fileCopied = true;
            }
            logger.debug("fileCopied:{}", fileCopied);
            return fileCopied;
        } catch (S3Exception exc) {
            logger.error("An S3Exception has occured copying an object. Exception: {}", exc);
        }

When I execute this code - I get the following error

2020-02-18 20:12:58 b21c952a-1b2e-16c0-9c09-225fec20a032 ERROR S3ServiceImpl - An S3Exception has occured copying an object. Exception: {}
software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. (Service: S3, Status Code: 404, Request ID: E364B4C42FAECBEF)
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:158) ~[aws-xml-protocol-2.10.65.jar:?]
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108) ~[aws-xml-protocol-2.10.65.jar:?]
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:86) ~[aws-xml-protocol-2.10.65.jar:?]
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:44) ~[aws-xml-protocol-2.10.65.jar:?]
        at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:94) ~[aws-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$4(BaseClientHandler.java:215) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:74) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:43) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.doExecute(RetryableStage.java:114) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage$RetryExecutor.execute(RetryableStage.java:87) ~[sdk-core-2.10.65.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:63) ~[sdk-core-2.10.65.jar:?]


        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:43) ~[sdk-core-2.10.65.jar:?]

I have validated that the bucket, file and version do in fact exist

[damogallagher]

Hi @debora-ito

As per the documentation https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/examples-s3-objects.html#copy-an-object

If I use the following code for copying an object without a version - it works fine
encodedUrl = URLEncoder.encode(fromBucketName + "/" + fromObjectKey , StandardCharsets.UTF_8.toString());

If I use this code where I specify a version - it fails with an S3 Exception that states The specified key does not exist
encodedUrl = URLEncoder.encode(fromBucketName + "/" + fromObjectKey + "?versionId=" + versionId, StandardCharsets.UTF_8.toString());

However, if I use this line with a version specified and not URLEncoding the string, the file copies as expected
encodedUrl = fromBucketName + "/" + fromObjectKey + "?versionId=" + versionId;

Can you please advise on the correct approach to use here. For now I will update my code to not use the URLEncoder

[debora-ito]

Can you try encoding only the value of versionId, not encoding the "?versionId=" portion? Something like:

encodedUrl = URLEncoder.encode(fromBucketName + "/" + fromObjectKey, StandardCharsets.UTF_8.toString());
encodeUrl +=  "?versionId=" + URLEncoder.encode(versionId, StandardCharsets.UTF_8.toString());

[damogallagher]

@debora-ito that works also
Would encoding the versionId make much of a difference? The versionIds I can see in my buckets are alphanumeric with periods and underscores just.
Can you advise on the recommended approach to follow?

I think it would be useful for others to update the example here to show how to perform a copy on a specific version

[debora-ito]

The copySource() documentation says it must be URL-encoded, so yes I would encode the versionId too.

I'll go ahead and close this, feel free to reach out if you have any further questions.