In this post we show examples of how to download files and images from an aws S3 bucket using Python and Boto 3 library.
Boto is an AWS SDK for Python. It provides easy to use functions that can interact with AWS services such as EC2 and S3 buckets.
Dowload S3 Objects With Python and Boto 3
In the following example, we download one file from a specified S3 bucket.
First we have to create an S3 client using
import boto3 BUCKET_NAME = 'my_s3_bucket' BUCKET_FILE_NAME = 'my_file.json' LOCAL_FILE_NAME = 'downloaded.json' def download_s3_file(): s3 = boto3.client('s3') s3.download_file(BUCKET_NAME, BUCKET_FILE_NAME, LOCAL_FILE_NAME)
download_file method takes three parameters:
The first parameter is the bucket name in S3. The second is the file (name and extension) we want to download and the third parameter is the name of the file we want to save as.
Download All S3 Objects in a Specified Bucket
In the following example, we download all objects in a specified S3 bucket.
The code snippet assumes the files are directly in the root of the bucket and not in a sub-folder.
import boto3 def download_all_files(): #initiate s3 resource s3 = boto3.resource('s3') # select bucket my_bucket = s3.Bucket('bucket_name') # download file into current directory for s3_object in my_bucket.objects.all(): filename = s3_object.key my_bucket.download_file(s3_object.key, filename)
Download All Objects in A Sub-Folder S3 Bucket
The following code shows how to download files that are in a sub-folder in an S3 bucket.
Suppose the files are in the following bucket and location:
BUCKET_NAME = 'images'
PATH = pets/cats/
import boto3 import os def download_all_objects_in_folder(): s3_resource = boto3.resource('s3') my_bucket = s3_resource.Bucket('images') objects = my_bucket.objects.filter(Prefix='pets/cats/') for obj in objects: path, filename = os.path.split(obj.key) my_bucket.download_file(obj.key, filename)