A Blueprint for AWS-Secured Webhook Listeners for Cloudinary

tl;dr: An AWS-secured and optimized Cloudinary webhook listener for extending the Cloudinary service

Code: Github

A webhook is a communication medium for sending notifications from one platform to another about events that occurred. In place are user-defined HTTP callbacks that are triggered by specific events. When a triggered event takes place on the source site, the webhook listens to the event, collects the data, and sends it to the URL you specified in the form of an HTTP request.

With Cloudinary webhooks, you can augment Cloudinary capabilities with home-grown code or third-party APIs by triggering actions in your application when certain events occur in your Cloudinary account. In essence, a Cloudinary webhook is a small piece of code that takes action upon a Cloudinary event, such as a file upload, a metadata modification, or the complete processing of a media asset.

The sections below describe a blueprint of a Hello World! Cloudinary webhook listener that is deploy-ready and customizable with a few clicks or lines of code.

Real-Life Use Case

A prominent digital-transformation agency that specializes in luxury fashion migrated a legacy DAM to Cloudinary to leverage our platform’s new capabilities but continue to use the current middleware. Here’s what the agency aimed to accomplish:

• Poll Cloudinary on a fixed frequency for changes in content and specific metadata fields.
• Preprocess short videos with an optimized format and quality compression.
• Preprocess long videos with adaptive bitrate (ABR) streaming.

Accordingly, the agency did the following:

1. Adopted the template in this post to quickly deploy a global web listener that checks if an asset or the nominated metadata has been changed and, if so, update another metadata field to reflect the last update timestamp (last_update_timestamp).

2. Fulfilled the polling requirement with Cloudinary’s search API.

3. Through the same web listener, determined if an event is an upload of a video. If so, gauge if the video size is large or small.

4. Executed the required preprocessing with Cloudinary’s explicit method.

Vincenzo Cerbone, technical consultant of openmind, said:

“We had a previous experience in batch synchronization with a legacy DAM based on polling techniques. Despite the pros and cons of that approach, we valued the user's asset state on Cloudinary the most. That means that we let the CMS act more as a ‘slave’ of the user's decisions on the Cloudinary side rather than the opposite. The connection point between the two worlds is the mandatory CMS_ID SMD custom field. The rest of the ‘magic’ is done through AWS Lambda functions (WebListeners) Yuval suggested. Ultimately, that mechanism took care of all the essential transformations.

We look forward to other upcoming features from Cloudinary that this use case can take advantage of. It’s truly exciting to be part of this process!”

Use Cases

Cloudinary offers an end-to-end media-management solution for images and videos, including upload, storage, administration, transformation, optimization, and delivery. The many features open up numerous use cases for a webhook listener, such as the following:

• Updates to an activity feed on a front-end system with asset-related events and details, such as when someone uploads an asset to the Cloudinary Media Library or deletes one from there.
• Bulk processing of records listed in an uploaded CSV file.
• Updates to the asset metadata stored in an external database in the event of updates to an asset’s metadata in the Cloudinary Media Library
• Extraction of metadata from assets uploaded with Cloudinary’s upload widget—for tagging in an external system.
• Dispatch of a text message or email to a customer in case of asset uploads.

Prerequisites

To create a blueprint for a Cloudinary webhook listener, you need the following:

• A Cloudinary account
• An AWS account
• The AWS command-line interface (CLI)
• A functional S3 bucket

Installation and Setup

Upon installation, the blueprint for a Cloudinary webhook listener calls for the following:

1. Install a Lambda listener that adds the tag Hello World! to any asset you upload to your Cloudinary account.
2. Store your Cloudinary account’s credentials in the Amazon Web Services (AWS) Secrets Manager, logging the web-listener activity in Cloudwatch.

The installation and setup process takes a few minutes. The boilerplate code is under 50 lines of Python script, easily accessible through AWS Lambda’s code editor.

To set up the tutorial, follow the instructions in the README.md file.

Code Tutorial

1. Write to your preferred logging solution: either use the default AWS logger with CloudWatch or override set_logger. The entry point is where the Cloudinary event was first received.

   validate_signature ensures that the events received are authentic. Note the mandatory return syntax, which informs Cloudinary that the function has been completed successfully. If the function fails, Cloudinary resends the notification three more times: once after three minutes, then after nine minutes, and then after 27 minutes. exception_wrapper adheres to that syntax as well.

2. Place your code in the main_process function. The example below shows how to write back a tag to the uploaded asset. Here, to avoid additional I/O operations, the utility function config_cloudinary_instance() validates the event's authenticity with cached credentials.

def lambda_handler(event, context):
    try:
        set_logger()
        logger.info(json.dumps(event))
        validate_signature(event)
        main_process(json.loads(event['body']))
    except Exception as e:
        return exception_wrapper(logger, e)
    return {'statusCode': 200, 'body': 'OK', 'isBase64Encoded': False}

The main_process function is where you’ll place your own code. The example shows how to write back a tag to the uploaded asset. To prevent additional I/O operations, the utility function config_cloudinary_instance() ensures that the cached credentials that validated the event's authenticity are in use.

Further Options and Resources

Check out the resources below on creating and managing webhook listeners.

When uploading assets:

• Modify the original asset with incoming transformations.
• Eagerly transform assets with eager transformations.
• Calculate upload parameters in JavaScript with the eval parameter.

When delivering assets:

• Apply different transformations to different assets with conditional transformations.
• Customize client-side functions with WASM code.
• Customize image transformations with remote functions.

Summary

As always, our goal at Cloudinary is to efficiently manage, transform, and optimize your web and mobile app's media assets, freeing you up to focus on your core business logic. Oftentimes, real-world use cases are complex and diverse, involving front-end and back-end systems as well as asynchronous workflows. Cloudinary’s webhooks with their ability to perform custom computing tasks promise to be useful tools for handling even the most demanding media-management tasks.

We’d much appreciate your feedback on this new blueprint.