AWS S3 Copying empty success file

30 Apr 2018

rchamart@rchamart-ubuntu:~$ aws s3 cp "/tmp/test_folder/_SUCCESS" "s3://my-test-bucket/prefix/_SUCCESS"
  upload failed: /tmp/test_folder/_SUCCESS to s3://my-test-bucket/prefix/_SUCCESS seek() takes 2 positional arguments but 3 were given

The issue and related pull request for the fix are present here. https://github.com/aws/aws-cli/issues/2403

Onw way to get around the issue is to use s3api

aws s3api put-object \
    --bucket=my-test-bucket \
    --key=_SUCCESS \
    --body=/tmp/_SUCCESS

Or if you can make an SDK call, you can use boto3.set_contents_from_string and pass in an empty string.

import boto3
s3 = boto3.resource(
    's3',
    region_name='us-east-1',
    aws_access_key_id=KEY_ID,
    aws_secret_access_key=ACCESS_KEY
)
s3.Object('my-test-bucket', '_SUCCESS').put(Body="")

who am i

I am a data engineer with interests in databases, data science, algorithms and programming in general.

where am i

linkedin//rchamarthi
stackoverflow/users/237939/rchamarthi
Content available under Creative Commons (BY-NC-SA) unless otherwise noted.
This site is hosted at Github Pages and created with Jekyll. It uses the papyrus theme