Boto : Boto is a SDK designed to improve the use of the python programming language in aws.
Setup requirement:
- Aws signup
- Python version 2.7
- Pycharm 3.3 [ide]
- Pip setup
For programmatic access: We need to enable the access in the aws iam:
For programmatically user access , secret key and access id.
Setup awscli on windows
- Awscli
- Prerequires:
- Check your system has Python 2.7
- Pip is configured
Configure awscli
https://docs.aws.amazon.com/cli/latest/userguide/install-windows.html
Configure the access key and id
C:\Users\amitm>aws configure
AWS Access Key ID [****************Z2FA]:
AWS Secret Access Key [****************V270]:
Default region name [ap-south-1]:
Default output format [test]:
Check setup?
C:\Users\amitm>aws s3 ls
2018-10-15 13:08:20 cf-templates-106h68kzl5m34-us-east-2
2018-11-08 23:39:48 openwriteup
2018-11-08 23:46:12 openwriteup-1
2018-11-09 00:16:44 test-openwriteup
What is awscli??
- This is a command line tool
- If we are writing script we can use it
- Testing purpose or want to use shell or powershell it is useful that
Setup boto3
- Pip install boto3
- Test boto3
- Python
- Import boto3
- Help(boto3)
Botocore
- A low-level interface to a growing number of Amazon Web Services. The botocore package is the foundation for the AWS CLIas well as boto3.
- Botocore provides the low level clients, session, and credential & configuration data. Boto 3 builds on top of Botocore by providing its own session, resources and collections.
- botocore does not provide higher-level abstractions on top of these services, operations and responses. That is left to the application layer. The goal of botocore is to handle all of the low-level details of making requests and getting results from a service
Core concepts of boto3
Resources
- higher-level, object-oriented API
- generated from resource description
- uses identifiers and attributes
- has actions (operations on resources)
- exposes subresources and collections
example:
import boto3
s3 = boto3.resource('s3')
bucket = s3.Bucket('mybucket')
for obj in bucket.objects.all():
print(obj.key, obj.last_modified)
Boto Client:
- low-level service access
- generated from service description
- exposes botocore client to the developer
- typically maps 1:1 with the service API
- snake-cased method names (e.g. ListBuckets API => list_buckets method)
example:
import boto3
client = boto3.client('s3')
response = client.list_objects(Bucket='mybucket')
for content in response['Contents']:
obj_dict = client.get_object(Bucket='mybucket', Key=content['Key'])
print(content['Key'], obj_dict['LastModified'])
Difference Between resource and client:
Resource object is very high level object, every operation with resource object would be high level operation. We may not have all the operation with resource.
Client is low level object, so whatever operation we want to perform its always be available. Client operations are mostly dictionary operation.
Session:
- stores configuration information (primarily credentials and selected region)
- allows you to create service clients and resources
Simple object to get it connected to particular aws account or iam account. If i want to connect any iam acocunt, session object will be used.
Pagination
- Automatically handles pagination
- Yields individual pages
- You must process each pages
Example: I have three thousand object in my s3 bucket, which i want to list. Boto3 Api can only list till a limit (1000 object). In such cases paginator can be used to list all the 3k objects. It will be using 3 pages to list .
Waiter
Waiter are used for reach waiting to reach certain state
Example: I have ec2 instance, which i newly launched, it takes some time to reach running state. For that purpose we can use waiter
