Plug and play image for HTTP Live Streaming and MPEG-DASH
You know that moment when you click a video link on your phone while on your way to grab a coffee from the office kitchen, but then you get that annoying buffering icon, and just give up? That video might have been interesting, maybe even valuable, but it’s just not worth your time.
Millions of users repeat this experience every day: stealing a couple minutes before starting work, stretching the time before going to sleep or getting out of bed, grabbing a quick break in between other planned tasks, etc. In any of these scenarios, a short video can be a great way to learn more about a concept, product, or get the latest news, but our ‘free’ moments are precious and we are not going to waste them waiting for heavy videos to download to our phones and other devices. Even when we do have more time to spare, we find ourselves too impatient to wait for any video that buffers for too long.
If you deliver video on your website, you understand that great video content is of little value if it isn't delivered quickly enough to get your users watching and then keep their attention until the end. You may also know that the best way to minimize these incredibly costly failures is to deliver video using adaptive bitrate streaming.
Adaptive bitrate streaming is a video delivery technique that adjusts the quality of a video stream in real time according to detected bandwidth and CPU capacity of each user.
Two of the most popular adaptive bitrate streaming formats are HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP). If you can deliver video in both of these formats, you can probably minimize buffering time for just about all of your users. But each of these formats requires outputting every video multiple times at different qualities along with a set of additional required files. This process is generally both complex and tedious. Services that help you create these files can be costly, and most don’t provide an end-to-end solution.
You can simplify the creation of these files using Cloudinary’s adaptive streaming profile functionality, which enables you to automatically generate all of the required video files for each of the desired quality and bitrate levels, along with the corresponding index and master files. You just upload a video in any format, plug in the name of one of Cloudinary's predefined streaming profiles in an eager transformation request (or specify a custom streaming profile that you defined with the Admin API), and then deliver the automatically generated, fully packaged .m3u8 or .mpd file using the HTTP Live Streaming or MPEG-DASH player of your choice. Literally Plug... and Play!
How adaptive bitrate streaming works
Providing a video using adaptive bitrate streaming means that you actually provide several versions (known as representations or variants) of your video, each with different qualities, bitrates, and codecs. Each video file must also be accompanied by an index file that specifies predefined segments or chunks of the video. These segments are usually 2, 5, or 10 seconds long. Additionally, there is a master playlist that points to the available representations with additional information about each one.
The adaptive streaming player uses this information to decide which of the available representations best matches a user’s network conditions or preferences at any time. It can switch to another representation with higher or lower quality at each segment if the network conditions change. This helps maintain continuous viewing with the best possible quality to maximize the user experience.
The example below simulates the process that an adaptive streaming player might follow as a user watches a video that’s delivered via adaptive streaming:
|
|
User begins playing while connected to wifi. Video begins playing immediately using the smaller 640x360 representation and simultaneously buffers. |
|
|
After a few seconds, the user expands to full screen, and since the buffering is sufficient, the client delivers the higher resolution 1280x720 representation. |
 |
The user manually selects the 1920x1080 representation from the player’s resolution selection box. |
 |
As the user walks from his home to his car and leaves his wifi range, he changes the player resolution selection back to automatic. The client recognizes the drop in network performance and reverts to 640x360 play. |
HTTP Live Streaming and MPEG-DASH adaptive bitrate streaming formats
The two leading adaptive bitrate streaming formats are HLS (HTTP Live Streaming) and MPEG-DASH (Dynamic Adaptive Streaming over HTTP).
HLS is an adaptive streaming communications protocol created by Apple. The Safari browser can play HLS streams within a web page, iPhones, and iPod touch devices. Since v2, all Apple TV devices also include an HTTP Live Streaming client. Apple’s AppStore requires apps that stream videos longer than 10 minutes over a cellular network to deliver those videos via HLS adaptive streaming. For details, see https://developer.apple.com/library/ios/qa/qa1767/_index.html
MPEG-DASH is an international standard for streaming video, intended to be a standardized replacement for proprietary HTTP streaming technologies, but is currently not natively supported in iOS devices. In other browsers and devices, it requires a javascript library or a video player that supports MPEG-DASH.
As mentioned above, to deliver videos using adaptive bitrate streaming, you must generate multiple video representations, an index file per representation, and a master playlist. The formats and encoding for HLS and MPEG-DASH are different for each of these files. So, if you want to deliver both HTTP Live Streaming and MPEG-DASH formats, to cover both iOS and non-iOS devices, then you need to double the effort for every video you want to deliver. Additionally, for MPEG-DASH, the best practice is to deliver the audio and video separately, so chalk up even more files and complexity.
Cloudinary streaming profiles
Generating so many video outputs and creating all the required supporting files for every video you want to deliver, is a huge and potentially expensive challenge.
To address this challenge, Cloudinary enables you to pick from a set of predefined streaming profiles or create your own custom profile. A Cloudinary streaming profile comprises a set of video representation definitions with various qualities, bitrates, and codecs. For example, the 4k predefined profile specifies 8 different representations in 16*9 aspect ratio, ranging from extremely high quality to audio-only.
Once you pick (or define) a streaming profile, you simply upload your video file with a single eager transformation that instructs Cloudinary to generate all the required files for the requested profile in either HTTP Live Streaming or MPEG-DASH format. If you want to deliver both formats, all it takes is two eager transformations within your upload command, instead of one.
For example, this Ruby code eagerly generates the big_buck_bunny video in both HTTP Live Streaming and MPEG-DASH formats using the full_hd built-in streaming profile, which yields 6 representations of the movie at a variety of quality levels.
Cloudinary::Uploader.upload("big_buck_bunny.mp4", :resource_type => :video, :eager => [ {:streaming_profile => "full_hd", :format => "m3u8"}, {:streaming_profile => "full_hd", :format => "mpd"}], :eager_async => true, :eager_notification_url => "http://mysite/notify_endpoint", :public_id => "bb_bunny")
There are seven different predefined adaptive streaming profiles to choose from. The same profiles are relevant for both HLS and DASH. For full details on the available profiles, see Predefined streaming profiles in the Cloudinary documentation. You can use the Admin API to fine-tune the predefined profiles to fit your needs, or to create your own custom streaming profiles.
When the eager upload is complete, the only thing left for you to do is to embed the relevant adaptive streaming client player(s) in your application. There are a number of open source and closed source HLS and DASH client players available.
While you play an m3u8 or mpd file generated using an adaptive streaming profile, you can take a look at the way the client player selects from your representations using the browser tools, such as the Chrome DevTools console:
For example, you can see above that the highlighted segment played using the 480x360 resolution with 800k bitrate, while the following segment played the representation with 640x480 resolution and 2m bitrate.
The higher-level the streaming profile you select, the more levels the client player has to choose from, to serve up the best possible quality that will enable continuous playback for each user’s current network conditions. However, do keep in mind that higher level profiles will use more storage and transformation resources in your Cloudinary account. If this is an issue, you may want to take advantage of the ‘lean’ predefined profiles, which include fewer representations, with somewhat bigger jumps between lower and higher quality. Also, remember that even though you now only need a single line of code to prepare your HLS and/or MPEG-DASH output, these transformations involve a lot of overhead and must be done eagerly.
(Note: Technically, Cloudinary does support transforming and delivering these files on-the-fly in the case that the total size of all the files together is less than 100MB, but due to the long encoding time, this on-the-fly option should be used only for debugging purposes.)
You can also apply other transformations alongside your streaming profile transformation. For example, the following URL plays the bb_bunny video in HTTP Live Streaming format with a small logo overlaid in the top left corner of the movie.
(Note: The above URL can be viewed only in devices or players that support HLS)
Applying transformations in this manner applies the same transformation to all representations. You can also apply different transformations to different representations in a profile, for example, to include a text overlay showing the user which resolution is currently playing. To do this, you need to create a custom streaming profile.
Custom Streaming Profiles
The predefined adaptive streaming profiles that Cloudinary provides are based on a set of selected best practices and one of them is likely to be a good fit for most standard scenarios.
But what if none of the provided profiles exactly answer your needs? For example, you might want a different number of representations or different divisions of quality available for your profile. Or you might want to apply special transformations for different representations within the profile. To create a customized streaming profile, you use the streaming_profiles method of the Admin API.
For example, below we've used the Ruby implementation of the Admin API to create an hd_debug profile, where each defined representation includes the relevant video transformations as well as text overlay that displays that resolution and bitrate. This helps us debug and watch how the client switches from one representation to another as the video buffer grows, as we resize the screen, or as network conditions change.
Cloudinary::Api.create_streaming_profile("hd_debug", :representations => [{:transformation => [{:width=>320,:height=>240,:bit_rate=>'192k',:crop=>'limit'}, {:overlay=>'text:Arial_24_bold:320x240--bitrate:192KB', :color=>'white',:background=>'rgb:00000020'}, {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1, :flags=>'layer_apply'}]}, {:transformation => [{:width=>480,:height=>270,:video_codec=>'h264:baseline:3.0', :bit_rate=>'800k',:crop=>'limit'}, {:overlay=>'text:Arial_24_bold:480x270--bitrate:800KB', :color=>'white',:background=>'rgb:00000020'} {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1, :flags=>'layer_apply'}]}, {:transformation => [{:width=>640,:height=>360,:video_codec=>'h264:baseline:3.0', :bit_rate=>'2m',:crop=>'limit'}, {:overlay=>'text:Arial_24_bold:640x360--bitrate:2MB', :color=>'white',:background=>'rgb:00000020'}, {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1, :flags=>'layer_apply'}]}, {:transformation => [{:width=>960,:height=>540,:video_codec=>'h264:main:3.1', :bit_rate=>'3500k',:crop=>'limit'}, {:overlay=>'text:Arial_24_bold:960x540--bitrate:3.5MB', :color=>'white',:background=>'rgb:00000020'}, {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1, :flags=>'layer_apply'}]}, {:transformation => [{:width=>1280,:height=>720,:video_codec=>'h264:main:3.1', :bit_rate=>'5500k',:crop=>'limit'}, {:overlay=>'text:Arial_24_bold:1280x720--bitrate:5.5MB', :color=>'white',:background=>'rgb:00000020'}, {:border=>'6px_solid_rgb:00000020',:gravity=>'north',:y=>0.1, :flags=>'layer_apply'}]}])
After uploading the bb_bunny video with an eager transformation requesting to use this hd_debug profile in HLS (.m3u8) format, you can play the HLS video below natively in your mobile device or in an iOS browser, such as Safari. Give it a try!
https://res.cloudinary.com/cloudinary/video/upload/sp_hd_debug/bb_bunny.m3u8
(If you don't have a browser that supports HLS, navigate to hlsplayer.net and paste in the link above.)
Just Imagine...
Remember that "moment when" moment I described at the start of this post? You’ve been there. I’ve been there. But just imagine that with a couple lines of code to eagerly generate all your adaptive streaming files, your website users won’t have to 'be there' anymore.
To learn more, see Adaptive bitrate streaming - HLS and MPEG-DASH and the streaming_profiles method of the Admin API in the Cloudinary documentation .
Adaptive bitrate streaming profiles for HLS and MPEG-DASH are supported for all Cloudinary plans, including the Free Plan. Go ahead, try it out for yourself.
