Automated EBS Snapshots using AWS Lambda & CloudWatch

Posted 2021-01-03 qiandavi

Overview

In this post, we‘ll cover how to automate EBS snapshots for your AWS infrastructure using Lambda and CloudWatch.   We‘ll build a solution that creates nightly snapshots for volumes attached to EC2 instances and deletes any snapshots older than 10 days.   This will work across all AWS regions.

Lambda offers the ability to execute "serverless" code which means that AWS will provide the run-time platform for us.   It currently supports the following languages: Node.js, Java, C# and Python.   We‘ll be using Python to write our functions in this article.

We‘ll use a CloudWatch rule to trigger the execution of the Lambda functions based on a cron expression.

 

Create IAM Role

Before we write any code, we need to create an IAM role that has permissions to do the following:

• Retrieve information about volumes and snapshots from EC2
• Take new snapshots using the CreateSnapshot API call
• Delete snapshots using the DeleteSnapshot API call
• Write logs to CloudWatch for debugging

In the AWS management console, we‘ll go to IAM > Roles > Create New Role.   We name our role "ebs-snapshots-role".

For Role Type, we select AWS Lambda.   This will grant the Lambda service permissions to assume the role.

On the next page, we won‘t select any of the managed policies so move on to Next Step.

Go back to the Roles page and select the newly created role.   Under the Permissions tab, you‘ll find a link to create a custom inline policy.

Paste the JSON below for the policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": "ec2:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSnapshot",
                "ec2:DeleteSnapshot",
                "ec2:CreateTags",
                "ec2:ModifySnapshotAttribute",
                "ec2:ResetSnapshotAttribute"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

 

Create Snapshots Function in Lambda

Now, we can move on to writing the code to create snapshots.   In the Lambda console, go to Functions > Create a Lambda Function -> Configure function and use the following parameters:

 

In our code, we‘ll be using Boto library which is the AWS SDK for Python. 

Paste the code below into the code pane:

# Backup all in-use volumes in all regions

import boto3

def lambda_handler(event, context):
    ec2 = boto3.client(‘ec2‘)
    
    # Get list of regions
    regions = ec2.describe_regions().get(‘Regions‘,[] )

    # Iterate over regions
    for region in regions:
        print "Checking region %s " % region[‘RegionName‘]
        reg=region[‘RegionName‘]

        # Connect to region
        ec2 = boto3.client(‘ec2‘, region_name=reg)
    
        # Get all in-use volumes in all regions  
        result = ec2.describe_volumes( Filters=[{‘Name‘: ‘status‘, ‘Values‘: [‘in-use‘]}])
        
        for volume in result[‘Volumes‘]:
            print "Backing up %s in %s" % (volume[‘VolumeId‘], volume[‘AvailabilityZone‘])
        
            # Create snapshot
            result = ec2.create_snapshot(VolumeId=volume[‘VolumeId‘],Description=‘Created by Lambda backup function ebs-snapshots‘)
        
            # Get snapshot resource 
            ec2resource = boto3.resource(‘ec2‘, region_name=reg)
            snapshot = ec2resource.Snapshot(result[‘SnapshotId‘])
        
            volumename = ‘N/A‘
        
            # Find name tag for volume if it exists
            if ‘Tags‘ in volume:
                for tags in volume[‘Tags‘]:
                    if tags["Key"] == ‘Name‘:
                        volumename = tags["Value"]
        
            # Add volume name to snapshot for easier identification
            snapshot.create_tags(Tags=[{‘Key‘: ‘Name‘,‘Value‘: volumename}])

 

The code will create snapshots for any in-use volumes across all regions.   It will also add the name of the volume to the snapshot name tag so it‘s easier for us to identify whenever we view the list of snapshots. 

Next, select the role we created in the Lamba function handler and role section.

The default timeout for Lambda functions is 3 seconds, which is too short for our task.  Let‘s increase the timeout to 1 minute under Advanced Settings.    This will give our function enough time to kick off the snapshot process for each volume.

Click Next then Create Function in the review page to finish.

Schedule Trigger as CloudWatch Rule

Navigate to the Triggers tab and click on Add Trigger which brings up the following window:

 

Selecting CloudWatch Event - Schedule from the dropdown list allows us to configure a rule based on a schedule. It‘s important to note that the times listed in the cron entry are in UTC.   

You‘ll be prompted to enter a name, description, and schedule for the rule.

It‘s important to note that the times listed for the cron expression are in UTC.  In the example below, we are scheduling the Lambda function to run each weeknight at 11pm UTC.

 

Testing

We can test our function immediately by click on the Save and Test button in the function page.   This will execute the function and show the results in the console at the bottom of the page.

Logging

After verifying that the function runs successfully, we can take a look at the CloudWatch logs by clicking on the link shown in the Log Output section.

You‘ll notice a Log Group was created with the name /aws/lambda/ebs-create-snapshots.  Select the most recent Log Stream to view individual messages:

11:00:19  START RequestId: bb6def8d-f26d-11e6-8983-89eca50275e0 Version: $LATEST
11:00:21  Backing up volume vol-0c0b66f7fd875964a in us-east-2a
11:00:22  END RequestId: bb6def8d-f26d-11e6-8983-89eca50275e0
11:00:22  REPORT RequestId: bb6def8d-f26d-11e6-8983-89eca50275e0 Duration: 3256.15 ms Billed Duration: 3300 ms Memory Size: 128 MB Max Memory Used: 40 MB

 

Delete Snapshots Function in Lambda

Let‘s take a look at how we can delete snapshots older than the retention period which we‘ll say is 10 days.

Before using the code below, you‘ll want to replace account_id with your AWS account number and adjust retention_days according to your needs. 

 

# Delete snapshots older than retention period

import boto3
from botocore.exceptions import ClientError

from datetime import datetime,timedelta

def delete_snapshot(snapshot_id, reg):
    print "Deleting snapshot %s " % (snapshot_id)
    try:  
        ec2resource = boto3.resource(‘ec2‘, region_name=reg)
        snapshot = ec2resource.Snapshot(snapshot_id)
        snapshot.delete()
    except ClientError as e:
        print "Caught exception: %s" % e
        
    return
    
def lambda_handler(event, context):
    
    # Get current timestamp in UTC
    now = datetime.now()

    # AWS Account ID    
    account_id = ‘1234567890‘
    
    # Define retention period in days
    retention_days = 10
    
    # Create EC2 client
    ec2 = boto3.client(‘ec2‘)
    
    # Get list of regions
    regions = ec2.describe_regions().get(‘Regions‘,[] )

    # Iterate over regions
    for region in regions:
        print "Checking region %s " % region[‘RegionName‘]
        reg=region[‘RegionName‘]
        
        # Connect to region
        ec2 = boto3.client(‘ec2‘, region_name=reg)
        
        # Filtering by snapshot timestamp comparison is not supported
        # So we grab all snapshot id‘s
        result = ec2.describe_snapshots( OwnerIds=[account_id] )
    
        for snapshot in result[‘Snapshots‘]:
            print "Checking snapshot %s which was created on %s" % (snapshot[‘SnapshotId‘],snapshot[‘StartTime‘])
       
            # Remove timezone info from snapshot in order for comparison to work below
            snapshot_time = snapshot[‘StartTime‘].replace(tzinfo=None)
        
            # Subtract snapshot time from now returns a timedelta 
            # Check if the timedelta is greater than retention days
            if (now - snapshot_time) > timedelta(retention_days):
                print "Snapshot is older than configured retention of %d days" % (retention_days)
                delete_snapshot(snapshot[‘SnapshotId‘], reg)
            else:
                print "Snapshot is newer than configured retention of %d days so we keep it" % (retention_days)          
  

 

FAQ

How do I include or exclude certain volumes from the create snapshots function?

The function we wrote for creating snapshots used a filter when calling ec2.describe_volumes that looked for status of in-use:

result = ec2.describe_volumes( Filters=[{‘Name‘: ‘status‘, ‘Values‘: [‘in-use‘]}])

Including Volumes using Tags

We can also create tags on volumes and filter by tag. 

Suppose we wanted to backup only the volumes that had a specific tag named "Backup" with a value of "Yes".

First, we create a tag for each volume by right-clicking on the volume and selecting Add/Edit Tags

 

Next, we modify the script and use the following line for describe_volumes:

result = ec2.describe_volumes( Filters=[{‘Name‘:‘tag:Backup‘, ‘Values‘:[‘Yes‘]}] )

Excluding Volumes using Tags

Excluding certain volumes using tags is a bit different.   The Filters parameter of describe_volumes is for inclusion only.   So we cannot use that to tell the script to exclude specific volumes.

Suppose we wanted to backup all volumes EXCEPT ones with a tag named "Backup" with a value of "No".

First, we create a tag for each volume by right-clicking on the volume and selecting Add/Edit Tags

Next, we modify the script and manually filter the volumes inside the loop:

# Backup all volumes all regions
# Skip those volumes with tag of Backup=No

import boto3

def lambda_handler(event, context):
    ec2 = boto3.client(‘ec2‘)
    
    # Get list of regions
    regions = ec2.describe_regions().get(‘Regions‘,[] )

    # Iterate over regions
    for region in regions:
        print "Checking region %s " % region[‘RegionName‘]
        reg=region[‘RegionName‘]

        # Connect to region
        ec2 = boto3.client(‘ec2‘, region_name=reg)
    
        # Get all volumes in all regions  
        result = ec2.describe_volumes()
        
        for volume in result[‘Volumes‘]:
            
            backup = ‘Yes‘
            
            # Get volume tag of Backup if it exists
            for tag in volume[‘Tags‘]:
                if tag[‘Key‘] == ‘Backup‘:
                    backup = tag.get(‘Value‘)
                    
            # Skip volume if Backup tag is No
            if backup == ‘No‘:
                break
            
            print "Backing up %s in %s" % (volume[‘VolumeId‘], volume[‘AvailabilityZone‘])
        
            # Create snapshot
            result = ec2.create_snapshot(VolumeId=volume[‘VolumeId‘],Description=‘Created by Lambda backup function ebs-snapshots‘)
        
            # Get snapshot resource 
            ec2resource = boto3.resource(‘ec2‘, region_name=reg)
            snapshot = ec2resource.Snapshot(result[‘SnapshotId‘])
        
            instance_name = ‘N/A‘
            
            # Fetch instance ID
            instance_id = volume[‘Attachments‘][0][‘InstanceId‘]
            
            # Get instance object using ID
            result = ec2.describe_instances(InstanceIds=[instance_id])
            
            instance = result[‘Reservations‘][0][‘Instances‘][0]
            
            print instance
            
             # Find name tag for instance
            if ‘Tags‘ in instance:
                for tags in instance[‘Tags‘]:
                    if tags["Key"] == ‘Name‘:
                        instance_name = tags["Value"]
            
            # Add volume name to snapshot for easier identification
            snapshot.create_tags(Tags=[{‘Key‘: ‘Name‘,‘Value‘: instance_name}])