Tag: boto3

Boto 3: Basic and setup

By amit

Boto : Boto is a SDK designed to improve the use of the python programming language in aws.

Setup requirement:

  • Aws signup
  • Python version 2.7
  • Pycharm 3.3 [ide]
  • Pip setup

For programmatic access: We need to enable the access in the aws iam:

For programmatically user access , secret key and access id.

Setup awscli on windows

  • Awscli
  • Prerequires:
    • Check your system has Python 2.7
    • Pip is configured

Configure awscli

https://docs.aws.amazon.com/cli/latest/userguide/install-windows.html

Configure the access key and id

C:\Users\amitm>aws configure

AWS Access Key ID [****************Z2FA]:

AWS Secret Access Key [****************V270]:

Default region name [ap-south-1]:

Default output format [test]:

Check setup?

C:\Users\amitm>aws s3 ls

2018-10-15 13:08:20 cf-templates-106h68kzl5m34-us-east-2

2018-11-08 23:39:48 openwriteup

2018-11-08 23:46:12 openwriteup-1

2018-11-09 00:16:44 test-openwriteup

What is awscli??

  • This is a command line tool
  • If we are writing script we can use it
  • Testing purpose or want to use shell or powershell it is useful that

Setup boto3

  • Pip install boto3
  • Test boto3
    • Python
    • Import boto3
    • Help(boto3)

Botocore

  • A low-level interface to a growing number of Amazon Web Services. The botocore package is the foundation for the AWS CLIas well as boto3.
  • Botocore provides the low level clients, session, and credential & configuration data. Boto 3 builds on top of Botocore by providing its own session, resources and collections.
  • botocore does not provide higher-level abstractions on top of these services, operations and responses. That is left to the application layer. The goal of botocore is to handle all of the low-level details of making requests and getting results from a service

Core concepts of boto3

Resources

  • higher-level, object-oriented API
  • generated from resource description
  • uses identifiers and attributes
  • has actions (operations on resources)
  • exposes subresources and collections

example:

import boto3

s3 = boto3.resource('s3')

bucket = s3.Bucket('mybucket')

for obj in bucket.objects.all():

print(obj.key, obj.last_modified)

Boto Client:

  • low-level service access
  • generated from service description
  • exposes botocore client to the developer
  • typically maps 1:1 with the service API
  • snake-cased method names (e.g. ListBuckets API => list_buckets method)

example:

import boto3

client = boto3.client('s3')

response = client.list_objects(Bucket='mybucket')

for content in response['Contents']:

obj_dict = client.get_object(Bucket='mybucket', Key=content['Key'])

print(content['Key'], obj_dict['LastModified'])

 

Difference Between resource and client:

Resource object is very high level object, every operation with resource object would be high level operation. We may not have all the operation with resource.

Client is low level object, so whatever operation we want to perform its always be available. Client operations are mostly dictionary operation.

Session:

  • stores configuration information (primarily credentials and selected region)
  • allows you to create service clients and resources

Simple object to get it connected to particular aws account or iam account. If i want to connect any iam acocunt, session object will be used.

Pagination

  • Automatically handles pagination
  • Yields individual pages
  • You must process each pages

Example: I have three thousand object in my s3 bucket, which i want to list. Boto3 Api can only list till a limit (1000 object). In such cases paginator can be used to list all the 3k objects. It will be using 3 pages to list .

Waiter

Waiter are used for reach waiting to reach certain state

Example: I have ec2 instance, which i newly launched, it takes some time to reach running state. For that purpose we can use waiter