Triggering an image upload to an S3 bucket and invoking a Lambda function to create a thumbnail
One of the functions provided by S3 is the event notification feature.
You can use the Amazon S3 Event Notifications feature to receive notifications when certain events happen in your S3 bucket.
Amazon S3 Event Notifications
In this case, we will consider triggering S3 event notifications and executing a Lambda function.
Specifically, we will reproduce the configuration covered in the following official AWS tutorial.
https://docs.aws.amazon.com/lambda/latest/dg/with-s3-tutorial.html
When an image is uploaded to the S3 bucket, the event notification triggers a Lambda function that creates a thumbnail.
Environment
Create two S3 buckets.
The other bucket is for storing images uploaded by users.
This bucket activates the notification feature.
The other is the bucket that stores the thumbnails created by the function.
Create a Lambda function.
The function’s action is to read images and create thumbnails.
This function is triggered by the notification of the first bucket.
CloudFormation template files
The above configuration is built with CloudFormation.
The CloudFormation templates are placed at the following URL
https://github.com/awstut-an-r/awstut-dva/tree/main/03/005
Explanation of key points of template files
S3 bucket
Bucket 1
Resources:
UploadBucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: Private
BucketName: !Ref UploadBucketName
NotificationConfiguration:
LambdaConfigurations:
- Event: "s3:ObjectCreated:*"
Function: !Ref ThumbnailFunctionArn
Code language: YAML (yaml)This is the bucket where the original thumbnail images will be placed.
The NotificationConfiguration property is used to configure notification settings.
In this configuration, the Lambda function is specified as the notification destination, so the LambdaConfigurations property is used.
To notify the Lambda function of an event when an image is placed in the bucket, specify “s3:ObjectCreated:*” in the Event property and the function described below in the Function property.
Bucket 2
Resources:
ResizeBucket:
Type: AWS::S3::Bucket
Properties:
AccessControl: Private
BucketName: !Ref ResizeBucketName
Code language: YAML (yaml)The bucket in which to place the thumbnail images generated by the function.
No special settings are made.
Lambda
Function
Resources:
ThumbnailFunction:
Type: AWS::Lambda::Function
Properties:
Architectures:
- !Ref Architecture
Code:
ZipFile: |
import boto3
import os
import sys
import uuid
from urllib.parse import unquote_plus
from PIL import Image
import PIL.Image
s3_client = boto3.client('s3')
def resize_image(image_path, resized_path):
with Image.open(image_path) as image:
image.thumbnail(tuple(x / 2 for x in image.size))
image.save(resized_path)
def lambda_handler(event, context):
for record in event['Records']:
bucket = record['s3']['bucket']['name']
key = unquote_plus(record['s3']['object']['key'])
tmpkey = key.replace('/', '')
download_path = '/tmp/{}{}'.format(uuid.uuid4(), tmpkey)
upload_path = '/tmp/resized-{}'.format(tmpkey)
s3_client.download_file(bucket, key, download_path)
resize_image(download_path, upload_path)
s3_client.upload_file(upload_path, '{}-resized'.format(bucket), key)
FunctionName: !Sub "${Prefix}-ThumbnailFunction"
Handler: !Ref Handler
Layers:
- !Ref LambdaLayer
Runtime: !Ref Runtime
Role: !GetAtt ThumbnailFunctionRole.Arn
Timeout: !Ref Timeout
Code language: YAML (yaml)The code to be executed by the Lambda function in inline format.
For more information, please refer to the following page.
https://awstut.com/2022/02/02/3-patterns-for-creating-lambda-with-cloudformation
The code to be executed, we will use the code described in the following official AWS page.
https://docs.aws.amazon.com/lambda/latest/dg/with-s3-tutorial.html
The code is as follows.
- From the received event notification, obtain the name of the bucket from which the notification originated and the name of the placed object.
- Download the object from the S3 bucket to local storage.
- Resize the downloaded object and create a thumbnail.
- Place the thumbnail in the bucket.
Check the IAM role for this function.
Resources:
ThumbnailFunctionRole:
Type: AWS::IAM::Role
DeletionPolicy: Delete
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action: sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: CreateThumbnailPolicy
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- s3:GetObject
Resource:
- !Sub "arn:aws:s3:::${UploadBucketName}/*"
- Effect: Allow
Action:
- s3:PutObject
Resource:
- !Sub "arn:aws:s3:::${ResizeBucketName}/*"
Code language: YAML (yaml)The contents allow actions on two buckets.
For the bucket that uploads images, the function downloads the images, so it grants “s3:GetObject” permission.
For the bucket to place the thumbnails, the function will upload the images, so the “s3:PutObject” permission is granted.
Lambda Layer
The Pillow package is used by the Lambda function in the process of creating thumbnails.
The external package creates a Lambda layer and includes the package here so that the Lambda function can import the client module.
In this case, we will use a CloudFormation custom resource to automatically perform the creation of this Lambda layer.
For more information, please see the following pages
Architecting
Use CloudFormation to build this environment and check its actual behavior.
Create CloudFormation stacks and check the resources in the stacks
Create CloudFormation stacks.
For information on how to create stacks and check each stack, please refer to the following pages.
https://awstut.com/en/2021/12/11/cloudformations-nested-stack
After reviewing the resources in each stack, information on the main resources created in this case is as follows
- S3 bucket for uploading images: dva-03-005 S3 bucket for installing thumbnails: dva-03-005-resized Lambda function for creating thumbnails: dva-03-005-ThumbnailFunction
Check each resource from the AWS Management Console.
First, check the image upload bucket.
There is a setting for event notification.
This setting is to execute the Lambda function described below for all events related to object creation.
Check the Lambda function.
function has been successfully created. Execute the code shown on the official AWS page.
Operation Check
Now that you are ready, place the images in the bucket. Place the images in the bucket from the AWS Management Console.
Upload this configuration diagram as a test image.
Upload is complete.
By uploading the object, the Lambda function should have been notified of the event. Check the execution log of the function.
Function is automatically executed.
Check the bucket for thumbnails.
The image is installed. An event notification automatically executes a Lambda function to create a thumbnail image and place it in this bucket.
Check the thumbnails created at the end.
It is indeed resized.
Thus, by using S3 event notification, a Lambda function can be executed when any event occurs.
summary
When an image is uploaded to the S3 bucket, the event notification triggers a configuration that executes a Lambda function that creates a thumbnail image.