Deep Dive: Three Tips for Faster, Easier Video Delivery

The best way to deliver video that starts up quickly and doesn’t stall is to make sure that the file downloads quickly. Since we cannot control the network our customers are using – this means we need to make the files smaller.
In a previous post at Cloudinary, I described three techniques to reduce file size: changing dimensions, lowering the video quality and using different codecs. In that post, I didn’t dive very deeply into the technical details. This post is the deeper dive into the analysis and how I created the smaller files for video delivery.

My Analysis:

To test and compare the playback of different video files, I used StreamorNot, a tool I built to measure video playback metrics. This tool plays back a video, and monitors the video startup time, the number of stalls, and the amount of video in the buffer at startup (amongst other things). Using devTools or WebPageTest to change the network speed allows you to test your video on networks of varying speed to understand what your users will see. I used the integration with WebPageTest to test each video 9 times on various network connections (all of the results can be found in this Google Sheet)

Changing Video Dimensions

Simply resizing the video to a smaller dimension can remove thousands (even millions!) of pixels per frame, so it is no surprise that the resulting video has a smaller file size. For example, the following video is hosted at Cloudinary, and is served at 99% quality (and trimmed to be just 30s long):

https://res.cloudinary.com/dougsillars/video/upload/eo_30,q_99/v1562844864/rbsp_launch_720p_gz9zfm.mp4

This video is 1280 pixels wide, and 720 pixels high (commonly referred to as 720p). This 30s long video weighs in at 25 MB. This video took 11.4s to begin playing on a 4G connection. Since this video is hosted at Cloudinary, it is very easy for me to resize the video – I can simply add a width parameter to the url. For example, a 800×600 pixel video is created with the following url:

https://res.cloudinary.com/dougsillars/video/upload/eo_30,w_800,q_99/v1562844864/rbsp_launch_720p_gz9zfm.mp4

Testing four different dimensions shows that changing the dimensions of the video not only reduces file size, but also that the video begins playback faster.

dimensions Size (MB) Video Startup Time (s) S in buffer at startup
1280×720 25 11.4 7.6
960×540 10.7 2.7 1.5
800×450 6.7 2.3 1.0
640×360 4.2 2.11 2.5

Video Quality

Videos often have very high pixel density, and the number of pixels in each frame can be reduced with minimal visual reduction in quality. One way is to re-encode with video with FFMPEG and modifying the Constant Rate Factor (measured from 0 high to 51 low). Typical settings for the web are around 18-28 (the default value is 23). Using Cloudinary, one can simply add the vc_auto parameter (in this case replacing the q_99 parameter in the urls above). The vc_auto parameter saves the video at Cloudinary’s Quality 70 (out of 100), and lowers the audio bitrate to 22k. Retesting the four videos examined above with optimized quality further reduces file size:

dimensions Size (MB) Video Startup Time (s) S in buffer at startup
1280×720 4.1 2 1.2
960×540 2.3 2.1 3.5
800×450 1.8 2.1 7.5
640×360 1.3 2.25 15

Comparing this table with the full quality videos in the first table, the files are 4-5x smaller, and the amount of video in the buffer at video startup has increased (meaning that there is a much smaller chance of a stall during playback). But before we move ahead with these new files – we need to ensure that the visual quality has not dropped. We can measure the change in quality by comparing the full quality video with the reduced quality video. There are 3 metrics commonly used to compare video quality:

Structural SIMilarity (SSIM): measured from 0-1 where 1 is an identical video.

Peak Signal to Noise (PSNR): measured from 30-50 where higher is better.

Video MultiMethod Assessment Fusion (VMAF): measured from 0-100, where 70 is good, and 100 is excellent. In the table below, I am using the VMAF TV model (there is also a phone model – which we will see later).

dimensions SSIM PSNR VMAF
1280×720 0.985 45.4 89.6
960×540 0.985 45.1 88.8
800×450 0.985 44.54 87.33
640×360 0.984 43.8 85.1

The SSIM (for this specific video) remains rather constant, while the PSNR and VMAF drop slightly as the video gets smaller, but all of the values remain relatively high, indicting that the change in quality does not vary the quality apparent to the viewer.

To measure the PSNR and VMAF, the reference video (the original high quality video) and the distorted video must have the same dimensions. With Cloudinary, I can downsize the video with one transformation (eo_30,w_800,vc_auto), and then upsize it to the original size (w_1280,h_720) with a second transformation:
https://res.cloudinary.com/dougsillars/video/upload/eo_30,w_800,vc_auto/w_1280,h_720/v1562844864/rbsp_launch_720p_gz9zfm.mp4

Re-encoding the video

Our final optimisation is to look at the file formats that are available for video. In all of the examples above, I have used h.264 encoding inside the mp4 wrapper, which is the most commonly used video format, and is supported in nearly all browsers. However, by re-encoding to newer formats, we can again drastically reduce the file size of our videos. This is a trivial task of changing a few parameters in Cloudinary:

encoding Video URL
h264 https://res.cloudinary.com/dougsillars/video/upload/eo_30/v1562844864/rbsp_launch_720p_gz9zfm.mp4
VP8 https://res.cloudinary.com/dougsillars/video/upload/eo_30,vc_vp8/v1562844864/rbsp_launch_720p_gz9zfm.webm
VP9 https://res.cloudinary.com/dougsillars/video/upload/eo_30,vc_vp9/v1562844864/rbsp_launch_720p_gz9zfm.webm
H265 https://res.cloudinary.com/dougsillars/video/upload/eo_30,vc_h265/v1562844864/rbsp_launch_720p_gz9zfm.mp4
HLS https://res.cloudinary.com/dougsillars/video/upload/eo_30,sp_hd/v1562844864/rbsp_launch_720p_gz9zfm.m3u8

The links the the table above give the current browser support for VP8/8 and h265. At the time of writing, the former work in all modern browsers but Safari, and the latter works in Safari.
Streaming video cannot compare the VMAF score as each bitrate has different dimensions, unless each different bitrate is compared.

When I apply quality changes, and change the format, here are the results:

encoding Size (MB) Video Startup Time (s) S in buffer at startup VMAF (big screen) VMAF (mobile)
h264 4.1 2.1 1.3 89.6 99.4
VP8 2.0 2.1 2.9 57.58 74.0
VP9 1.7 1.8 4.3 57.2 73.9
H265 1.47 Not tested Not tested 89.2 94.1
HLS 1.7 1.3 5.5 N/A N/A

These video are now more than 90% smaller than the original file we began working with! The startup times remain around 2s, and the time in the buffer at startup doubles or triples depending on the format chosen. However, the VP8 and VP9 have a marked reduction in quality, scoring below ‘good’ for TV, and right at the ‘good’ cutoff for mobile. Depending on the usage of this video, these formats might not be the best choice for video delivery. However, the h265 encoding might be worthwhile for your Safari users..

Conclusion:

To better speed video startup times and to reduce stalls, reducing the size in MB of your video is the most important step. By reducing the dimensions (in pixels), lowering the quality, and changing the encoding, the size of the file can be reduced drastically, with minimal effects on the playback quality. It is important to test your videos to see how the changes in encoding and sizing will affect the quality, but you’ll certainly find the right combination that will deliver the perfect videos to your customers.
Tools like Cloudinary make re-encoding videos at different qualities easy and painless, and StreamorNot helps you understand the way your videos will play back on different devices, and at different network speeds.

One thought on “Deep Dive: Three Tips for Faster, Easier Video Delivery

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s