Cloudinary is a cloud-based service that provides solutions for image and video management, including server or client-side upload, on-the-fly image and video transformations, quick CDN delivery, and a variety of asset management options.
The Cloudinary AI Background Removal add-on combines a variety of deep-learning algorithms to recognize the primary foreground object(s) in a photo and accurately remove the background in a matter of seconds. You can optionally specify one of a set of object names to instruct the add-on to remove everything except that object.
Before you can use the Cloudinary AI Background Removal add-on:
You must have a Cloudinary account. If you don't already have one, you can sign up for a free account.
Register for the add-on: make sure you're logged in to your account and then go to the Add-ons page. For more information about add-on registrations, see Registering for add-ons.
Keep in mind that many of the examples on this page use our SDKs. For SDK installation and configuration details, see the relevant SDK guide.
If you are new to Cloudinary, you may want to take a look at How to integrate Cloudinary in your app for a walk through on the basics of creating and setting up your account, working with SDKs, and then uploading, transforming and delivering assets.
The default background removal behavior detects the foreground object(s) of the image and removes the background. You can activate this behavior in one of two ways:
By setting the background_removal parameter to cloudinary_ai when uploading an image (Upload method), or by using the Update method of the Admin API for existing images.
On the fly, by specifying the background_removal effect transformation parameter when delivering images (e_background_removal in URLs).
You can also include additional directives for indicating what the AI algorithm should treat as the foreground object to keep. For details see: Specifying the foreground object below.
The code below uploads the photo shown below on the left and activates the Cloudinary AI background removal add-on. The photo that ultimately gets stored in the product environment is the dog image shown on the right, with a fully transparent background. The original image is not stored.
var uploadParams = new ImageUploadParams(){
File = new FileDescription(@"dog_couch.jpg"),
PublicId = "dog_couch",
BackgroundRemoval = "cloudinary_ai",
NotificationUrl = "https://mysite.example.com/hooks"};
var uploadResult = cloudinary.Upload(uploadParams);
iOS (cloudinary 3.x):
Copy to clipboard
let params = CLDUploadRequestParams()
.setPublicId("dog_couch")
.setBackgroundRemoval("cloudinary_ai")
.setNotificationUrl("https://mysite.example.com/hooks")
var mySig = MyFunction(params) // your own function that returns a signature generated on your backend
params.setSignature(CLDSignature(signature: mySig.signature, timestamp: mySig.timestamp))
let request = cloudinary.createUploader().signedUpload(
url: "dog_couch.jpg", params: params)
Original image to uploadUploaded image with the add-on applied
Tip
You can use upload presets to centrally define a set of upload options including add-on operations to apply, instead of specifying them in each upload call. You can define multiple upload presets, and apply different presets in different upload scenarios. You can create new upload presets in the Upload page of the Console Settings or using the upload_presets Admin API method. From the Upload page of the Console Settings, you can also select default upload presets to use for image, video, and raw API uploads (respectively) as well as default presets for image, video, and raw uploads performed via the Media Library UI.
You can remove the background from existing images using the Update Admin API method. The existing image is overwritten with the background-removed image.
var updateParams = new UpdateParams("woman"){
BackgroundRemoval = "cloudinary_ai",
NotificationUrl = "https://mysite.example.com/hooks"};
var updateResult = cloudinary.UpdateResource(updateParams);
Unlike the other methods of removing the background on upload and by updating an existing asset, removing the background on the fly keeps the original image intact and creates a derived version of the image with the background removed.
To remove the background of an image on the fly, specify the background_removal effect (e_background_removal in URLs). For example:
To replace the background with a different image, apply the new background as an underlay. For example, replacing the couch with a road, scaling and positioning the dog appropriately:
In some cases, the AI algorithm may identify more than one object in the foreground. If you know which type of item you expect in the photos you are applying the add-on to, and that item is among the supported objects listed below, you can request to keep only object(s) of that type and treat everything else as background.
To do this when uploading or updating an image, set the background_removal parameter to cloudinary_ai:[object_to_keep].
To do this on the fly, set the effect parameter to background_removal:hints_([object_to_keep]).
For example, in the original image below, both the cars and the road sign are identified by the add-on algorithm as foreground objects. However, by specifying the object to keep as car using background_removal:"cloudinary_ai:car" or e_background_removal:hints_(car), the sign is also treated as part of the background, and only the cars remain:
OriginalDefault background removalSpecifying car
Similarly, if you want to deliver images containing only dogs or only cats, you can ensure you only keep the animal you want:
Original Default background removalSpecifying catSpecifying dog
You can also specify more than one object to keep, for example to keep both cats and dogs, where there may be other objects in the picture:
On upload/update: background_removal:"cloudinary_ai:cat:dog"
var uploadParams = new ImageUploadParams(){
File = new FileDescription(@"girl_hair.jpg"),
PublicId = "girl_hair_fine",
BackgroundRemoval = "cloudinary_ai:fine_edges",
NotificationUrl = "https://mysite.example.com/hooks"};
var uploadResult = cloudinary.Upload(uploadParams);
iOS (cloudinary 3.x):
Copy to clipboard
let params = CLDUploadRequestParams()
.setPublicId("girl_hair_fine")
.setBackgroundRemoval("cloudinary_ai:fine_edges")
.setNotificationUrl("https://mysite.example.com/hooks")
var mySig = MyFunction(params) // your own function that returns a signature generated on your backend
params.setSignature(CLDSignature(signature: mySig.signature, timestamp: mySig.timestamp))
let request = cloudinary.createUploader().signedUpload(
url: "girl_hair.jpg", params: params)
Although the background removal algorithm usually takes only a few seconds, it is still handled asynchronously after the original file is uploaded or updated. Therefore, the immediate upload response indicates that the background_removal status is pending. For example, here's the upload response from the above dog-on-couch image.
Immediately after you perform the upload or update method, the image in your product environment is your original image. When the background removal process is complete, the original image is overwritten with a PNG containing the foreground image and transparent background.
If you requested a notification in your upload or update call by adding the notification_url parameter, the endpoint you specified will receive a JSON POST request when the background removal is complete, including the new url & secure_url with the updated .png file extension and new version number:
Similarly, when background removal is requested on the fly, initially a 423 error response is returned, until the background removal process has completed. This only happens for the first request. After this, the image is cached, so subsequent requests to the same URL will return the background-removed image.
The notification response returns a confidence score, which is a measure of how successful the background removal, and hence the resulting image, is likely to be. For low scores, you could for example, send the image for manual moderation using the explicit method, or perhaps use the Pixelz - Remove the Background add-on instead.
In addition to the notification response, the confidence score is also reported in responses to calls to the Admin API resources endpoint where asset details are returned.
After the background has been removed from your image, you can take advantage of a variety of image transformations to deliver the new image to your users.
If all the product images in your online store have a shadow, then after you've applied the Cloudinary AI Remove the Background add-on to this router photo, you could use the transformation code below to apply a shadow to the delivered image.
//React SDK transformations are created using @cloudinary/url-gen.new CloudinaryImage("docs/rmv_bgd/router.png")
.resize(scale().height(105))
.effect(shadow().strength(50).offsetX(10).offsetY(10));
//Angular SDK transformations are created using @cloudinary/url-gen.new CloudinaryImage("docs/rmv_bgd/router.png")
.resize(scale().height(105))
.effect(shadow().strength(50).offsetX(10).offsetY(10));
You can take advantage of the Cloudinary AI Remove the Background add-on to provide an app with fun effects. For example, you could allow users to upload images of themselves and then select new background locations by adding the desired scenery as an underlay layer behind your transparent background.
For example, after running the add-on on the original image to remove the background, you could use the following delivery code to relocate this woman from her office to the beach:
//React SDK transformations are created using @cloudinary/url-gen.new CloudinaryImage("docs/rmv_bgd/woman.png")
.resize(scale().height(375))
.underlay(
source(
image("docs/bg_beach").transformation(
new Transformation().resize(scale().width(800))
)
).position(new Position().gravity(compass("south")))
);
//Angular SDK transformations are created using @cloudinary/url-gen.new CloudinaryImage("docs/rmv_bgd/woman.png")
.resize(scale().height(375))
.underlay(
source(
image("docs/bg_beach").transformation(
new Transformation().resize(scale().width(800))
)
).position(new Position().gravity(compass("south")))
);
This add-on is based on a combination of neural networks that were trained on a large dataset to create precise segmentation maps of salient objects and refine those maps to accurately separate the edges of the foreground from the background. These neural networks were also optimized to enable returning the result within seconds regardless of the image size.1
In the great majority of cases, these deep-learning algorithms return high quality results. The following best practices can further increase the likelihood of good results:
Make sure there is enough background around the foreground object(s) on all sides.
Try to avoid large shadows. This is especially relevant if the background is a single color. For example, if the main subject is photographed against a plain light colored background, the object's shadow might be classified as part of the foreground.
In some cases, hair or other very small or blurry parts on the edges of a foreground object might be blended with the background. For example, try to avoid hairs blowing in the wind.
In some cases, strong changes in contrast, such as a bright lamp or a bright sun included in the photo, can impact the results.
Light-colored reflections on a shiny object (e.g. shine from the camera flash) may be treated as part of the background and removed, especially if the color of the reflection is similar to the main background color.
Images containing transparent objects, such as a glass jug, may not give good results. The confidence score can help you to determine how successful the background removal was.
However, as with any Deep Learning-based algorithm, even if you follow these guidelines, the results may not always be perfect. Additionally, keep in mind that the neural networks are continually training on new data and thus results may be different over time.
If you are not satisfied with the results of a background removal operation, you can use one (or both) of the following options:
Restore from backup: When the newly generated (background-removed) image overwrites the old one, both the original image and background-removed image are backed up and are accessible from the View backed up versions button, available when you open the Edit Transformation page for that image in the Media Library. This backup is performed for all images where the Cloudinary AI Remove the Background add-on is applied, even if you haven't enabled backups globally for your product environment.
Note
If you don't want the images to be backed up, contact support to apply a setting to your product environment that disables this specific backup. With this setting applied:
If your product environment does not have global backup configured and backup was not requested during the upload (either in the request or in the upload preset), then none of the images are backed up.
If your product environment does have global backup configured, or backup was requested during the upload (either in the request or in the upload preset), then only the original image is backed up, and not the background-removed one.
Pixelz Remove the background add-on: The Pixelz Remove the Background add-on automatically uploads your original image to the Pixelz team of human experts who manually remove the background of your image within 24 hours. When complete, the new image replaces your original one in your product environment similar to the behavior of the Cloudinary AI Remove the Background add-on. You can monitor the status using a notification_url.
1The add-on does not have any specific file or resolution size limitations. But keep in mind that the returned file is always a PNG with transparent background. Even if the originally uploaded image is within your file size product environment limits, large resolution images may result in very large file sizes for the returned PNG, and may exceed the max file size limits for your product environment. If the returned file exceeds your product environment limits, then it will not be stored in your product environment and the original will remain.
✔️ Feedback sent!
✖️
Error
Unfortunately there's been an error sending your feedback.
Original image to upload
Uploaded image with the add-on applied
Original
Default background removal
Specifying car
Original
Default background removal
Specifying cat
Specifying dog
Original
Default background removal
Specifying fine edges
Original
Default background removal
Specifying fine edges
Original router image
No background
No background + shadow
Original personal photo
No background
No background + rainbow scene
No background + beach scene
