Part1. Learning AWS CLI

As an application developer, you must be familiar with the CLI. Using the CLI (instead of UI) has the benefit that the operations can be documented and then they become reproducible and shareable. Fortunately, AWS provides both API and the unified CLI tool named aws-cli.

You must use and understand AWS CLI, especially when you want to control anything, AWS UI doesn’t provide yet; for example, Scheduled Scaling – Auto Scaling can be available only via AWS CLI.

Before explaining the full process, I will assume that you are using AWS VPC & S3. Also, ensure to have all of your network resources like security group inside VPC, and you know an access key and a secret key of your own AWS account or IAM account.

Let’s see how we can control EC2 instances and S3 :

Install aws-cli package

The first thing you need to do is to install aws-cli package on your machine.

# Install pip if your machine doesn't have pip yet
$ sudo easy_install pip

# Install awscli with pip
$ sudo pip install awscli

# Configure AWS credential and config
$ aws configure
AWS Access Key ID: foo
AWS Secret Access Key: bar
Default region name [us-west-2]: us-west-2
Default output format [None]: json

Note: You have to configure AWS Access Key ID and Secret Access Key to which an IAM account is attached by necessary but minimum policies. For now, I recommend you create an IAM account attached AmazonEC2FullAccess-AMI-201412181939 and AmazonS3FullAccess-AMI-201502041017.

# AmazonEC2FullAccess-AMI-201412181939
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "ec2:*",
      "Effect": "Allow",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "elasticloadbalancing:*",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "cloudwatch:*",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "autoscaling:*",
      "Resource": "*"
    }
  ]
}

# AmazonS3FullAccess-AMI-201502041017
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "s3:*",
      "Resource": "*"
    }
  ]
}

Run an EC2 instance

Okay, you are ready to control AWS resources via CLI. The most important thing to run an EC2 instance is preparing option parameters. You can confirm these details in run-instances — AWS CLI documentation.

This command generates a JSON file which has skeleton option parameters:

$ aws ec2 run-instances --generate-cli-skeleton > /tmp/run-instances_base.json

# We overwrite this skeleton file to be shorter and easier to understand
$ vi /tmp/run-instances_base.json
$ cat /tmp/run-instances_base.json 
{
    "ImageId": "ami-936d9d93",
    "KeyName": "YOUR Key pair name",
    "InstanceType": "t2.micro",
    "Placement": {
        "AvailabilityZone": "us-west-2"
    },
    "NetworkInterfaces": [
        {
            "DeviceIndex": 0,
            "SubnetId": "subnet-***",
            "Groups": [
                "sg-***"
            ],
            "DeleteOnTermination": true,
            "AssociatePublicIpAddress": true
        }
    ]
}

# Run an instance
$ aws ec2 run-instances --cli-input-json file:///tmp/run-instances_base.json

List running EC2 instances

Now confirm your running EC2 instances. The detail of using the command is here : describe-instances — AWS CLI documentation.

I recommend you use the jq tool because the output is formatted as JSON and you might be overwhelmed by its volume. You can install jq via brew or make the tool.

# Install jq if your machine doesn't have it yet and you want to use it on MacOSX
$ brew install jq

# List EC2 instances
$ aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | [.LaunchTime, .State.Name, .InstanceId,  .InstanceType, .PrivateIpAddress, (.Tags[] | select(.Key=="Name").Value)] | join("t")'
2015-09-22T10:16:41.000Z        running i-f19f6e54      t2.micro        10.0.1.61

Terminate an EC2 instance

Well, it’s time to terminate an EC2 instance to save money. The detail of using the command is here: terminate-instances — AWS CLI documentation

# DryRun the command
$ aws ec2 terminate-instances --instance-ids i-f19f6e54 --dry-run

# Terminate an EC2 instance
$ aws ec2 terminate-instances --instance-ids i-f19f6e54

List S3 directory contents

You want to find and grep AWS ELB access logs, especially if you are an operations engineer and have some problems. To start, find the specific file. The detail of using the command is here: ls — AWS CLI documentation.

# List ELB access logs created at 2015/09/18
$ aws s3 ls s3://example-elb-log/example-app-elb/AWSLogs/717669809617/elasticloadbalancing/us-west-2/2015/09/18/

Download a S3 content

Then you can download a concerned file and grep with a specific keyword. The detail of using the command is here: cp — AWS CLI documentation.

# Find access logs whose SSL cipher are ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2
$ aws s3 cp s3://example-elb-log/example-app-elb/AWSLogs/717669809617/elasticloadbalancing/us-west-2/2015/09/18/717669809617_elasticloadbalancing_us-west-2_example-app-elb_20150918T0230Z_54.92.79.213_5wo8k1of.log - | head -n 1000 | grep 'ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2'

Conclusion

AWS CLI is a very useful tool. It supports extensive and important services; for example, I recently upgraded an SSL certificate of ELB from a SHA-1 signed to a SHA-2 before the iOS9 release due to iOS9 ATS. During these operations, I received a peer review for the planned aws-cli commands asynchronously. It’s one of AWS CLI’s benefits.

About the author

Yohei Yoshimuta is a software engineer with a proven record of delivering high quality software in both game and advertising industries. He has extensive experience building products from scratch in both small and large teams. His primary focuses are Perl, Go, and AWS technologies. You can reach him at @yoheimuta on GitHub and Twitter.