Muted Videos and Subtitles

The bane of our existence is the lack of efficient ways for tackling the plethora of recurring tasks in our lives. One of those tasks is surfing the internet. We consume a lot of web content daily, of which a large percentage are images and videos. We’re constantly quickly scrolling through 30-second videos or checking out pictures of cute items we’d like to buy in our free time.

Some of those videos are in our native language; others are in foreign languages that we don’t understand. Sometimes we still enjoy the latter, however, because of the video’s graphical appeal. Other times, we might want to watch muted videos because of a loud immediate environment or an absence of headsets.

The Value of Subtitles

Subtitles are a superb feature for videos, muted or not.

Have you ever downloaded or watched someone download an SRT (.srt) file just to have the subtitles show up while watching a movie? As a software developer who builds apps that involve videos, you must inculcate the capacity that enables users to watch subtitles in videos.

Addition of Subtitles in Videos

How can you add video subtitles? Just as you can manually download and add SRT files to videos for subtitles, you can programmatically add subtitles to videos with Cloudinary. In fact, you can upload multiple SRT or WebVTT files to Cloudinary as normal raw files.

Making the content of those files show up as subtitles on your videos that have already been uploaded to Cloudinary is as straightforward as overlaying the subtitles on them. See this demo:

cl_video_tag("dog", :overlay=>{:resource_type=>"subtitles", :public_id=>"sample_sub_en.srt"})

cl_video_tag("dog", array("overlay"=>array("resource_type"=>"subtitles", "public_id"=>"sample_sub_en.srt")))

(new VideoTag('dog.mp4'))
  ->overlay(Overlay::source(Source::subtitles('sample_sub_en.srt')));

CloudinaryVideo("dog").video(overlay={'resource_type': "subtitles", 'public_id': "sample_sub_en.srt"})

cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})

cloudinary.url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).videoTag("dog");

cloudinary.videoTag('dog', {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")}).toHtml();

$.cloudinary.video("dog", {overlay: new cloudinary.SubtitlesLayer().publicId("sample_sub_en.srt")})

<Video publicId="dog" >
  <Transformation overlay={{publicId: "sample_sub_en.srt", resourceType: "subtitles"}} />
</Video>

<cld-video publicId="dog" >
  <cld-transformation :overlay="{publicId: 'sample_sub_en.srt', resourceType: 'subtitles'}" />
</cld-video>

<cl-video public-id="dog" >
  <cl-transformation overlay="subtitles:sample_sub_en.srt">
  </cl-transformation>
</cl-video>

cloudinary.Api.UrlVideoUp.Transform(new Transformation().Overlay(new SubtitlesLayer().PublicId("sample_sub_en.srt"))).BuildVideoTag("dog")

MediaManager.get().url().transformation(new Transformation().overlay(new SubtitlesLayer().publicId("sample_sub_en.srt"))).resourceType("video").generate("dog.mp4");

cloudinary.createUrl().setResourceType("video").setTransformation(CLDTransformation().setOverlay("subtitles:sample_sub_en.srt")).generate("dog.mp4")

Simply specify the overlay parameter l_subtitles, followed by the public ID of the raw file and its extension. That syntax works in the URL transformation as shown above.

Programmatically, you can code it this way:

cloudinary.video("dog", {overlay: {resource_type: "subtitles", public_id: "sample_sub_en.srt"}})

Node.js

Furthermore, you can customize the way subtitles appear in the video: the color, appearance, font, etc. For example, here’s how to make subtitles yellow in color with a red background and a larger font:

cloudinary.video("dog", {overlay: {font_family: "arial", font_size: 100, resource_type: "subtitles", public_id: "sample_sub_en.srt"}, color: "#ffff00", background: "red"})

Node.js

Generation of Transcripts and Subtitles for Videos

Another important step is to generate transcripts for your videos on the fly. If your platform contains a lot of user-generated video content, you cannot count on your users to also upload the SRT files for the videos they upload. Instead, your users depend on your platform to generate subtitles for their content. For instance, social-media users with numerous followers would expect that the app can and would generate subtitles for their uploaded videos and that, as soon as they turn on the subtitle feature, followers can immediately view the subtitles.

Cloudinary’s Google Video Transcript add-on enables you to automate the generation of speech-to-text transcripts of user-generated videos that have been uploaded to your site. For the best possible speech-recognition results, the add-on, which can translate videos in almost any language, applies powerful neural-network models to videos with Google's Cloud Speech API.

After a transcript becomes available, you can do the following:

1. Automatically insert the transcript into your video in the form of subtitles.
2. Parse the content of the transcript and display it on a webpage. That way, users can skim the content instead of watching the videos. This approach is extremely popular on learning platforms.

Here’s the code for generating transcripts:

cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech" },
  function(error, result) {console.log(result, error) });

Node.js

Once generation is complete, the add-on creates a new file with a .transcript extension in your Cloudinary account with the same public ID as your video file.

Depending on the size of the video file, the process might take a while.

You might wonder, “Why a .transcript file? Why not a .srt or .vtt file?” Well, you do have the option to have Cloudinary generate a .srt and a .vtt file. Just specify your preference, like this:

cloudinary.v2.uploader.upload("learning_javascript.mp4", 
  { resource_type: "video", 
    raw_convert: "google_speech:srt:vtt" },
  function(error, result) {console.log(result, error) });

Node.js

When the process is complete, the add-on automatically uploads these three generated files to your account:

.../raw/upload/learning_javascript.transcript
.../raw/upload/learning_javascript.srt
.../raw/upload/learning_javascript.vtt

Display of Generated Transcripts and Subtitles on Videos

The final step is to display the generated transcripts and subtitles on the video for delivery. It’s as simple as programmatically overlaying the files on the video, like this:

cloudinary.video("learning_javascript", {overlay: {resource_type: "subtitles", public_id: "learning_javascript.transcript"}})

Node.js

The URL reads—

https://res.cloudinary.com/demo/video/upload/l_subtitles:learning_javascript.transcript/learning_javascript.mp4

References

Below are additional references:

• An excellent article by Jackie Rosenzvieg on auto-generating subtitles based on video transcript
• Cloudinary’s documentation on the many options for video transcoding and transformation
• Documentation on the Cloudinary Video Player