Video on the web is now reaching far beyond just linking to YouTube.  Thousands of pages are embedding video as an integral part of the website experience: both increasing the interest and engagement of their site to users, but also the size and time to load of the webpage.  This is a delicate balance that deserves a closer exploration.

I thought it would be interesting to see how websites are utilizing video, specifically on mobile devices. In this study, I analyzed videos that autoplay on mobile, as these are the videos that are most likely to impact page load time.

The HTTP Archive August 2019 dataset has request/response data for 5M mobile websites. Filtering for sites that have a video tag in the response with autoplay and muted attributes (for videos to autoplay on mobile, the muted attribute must be present)  resulted in 7063 videos that are set to autoplay on mobile browsers.

Armed with the urls of 7063 videos, I entered them into StreamOrNot to test. This tool will load and playback the video, providing video playback data.  To quickly test all of these videos, I used Andy Davies’ WebPageTest Google sheet and several WebPageTest custom metrics to ascertain the video playback results.  Each video was tested for 200s (over 3 minutes to play back!) on a Motorola 4G in Dulles VA using a 3G connection.  (You can find my results here).

52% of the videos did not play back.  This was because either the file was removed since the HTTP Archive test, or there was JavaScript in the url string that I was unable to parse.  Still, we have 48% of the urls that work. I’ll focus on the 42% of videos that ran in the allocated 200s on a 3G connection, and as an aside, address why 5.8% of videos didn’t complete in the 200s time. (Note: Of the videos that of the that are present, only 88% played in 200s, meaning that 12% completely failed to play).

	videos	percentage
Complete Data	3128	42.05%
Incomplete Data	431	5.79%
no data	3880	52.16%

 

Videos that Played

88% (3128) of the videos that were still available successfully played on a 3G connection.  what characteristics can we discover about these videos?  Let’s look at the shapes of the video, and then the playback characteristics of the videos:

Size

The median mobile webpage is 1.7 MB.  As you might imagine, as soon as video is added into the page weight, this number increases significantly.  By the 25% percentile of video size, the page weight has now doubled (adding an additional 1.7 MB).

The median video delivered on the mobile web is 3.8 MB, and 19% of videos delivered on mobile are over 10MB.

Dimensions

The majority of the videos were 1080×1920 or 1280×720 (and the other larger points are on the 16:9 aspect ratio line), but the are many other sizes present:

Many of the other video sizes stay at 1920 or 1280 wide, but are narrower in height – more like a banner shaped video.  The videos to the bottom left of this chart have more of a portrait aspect ratio (longer height, shorter width).

 

Length

The median video length is 20s long.  As shorter videos will download playback and finish faster, this seems like a reasonable length.  However, 30% of videos are over 30s long, and 15% are over 60s long.

Audio

53% of the videos had sound, and 47% did not.  For background videos, or “animated GIF” looping videos, there is no sound, so removing the audio track slims the video size with no loss of experience.

	Audio	Percentage
TRUE	1888	53.17%
FALSE	1663	46.83%

 

Playback

The median video start time (from onload to the actual video playback commencing) is 12.8s.  Akamai found that beyond a 2s startup delay, viewers abandon videos.  This means that your website loses all the engagement wins you were looking to gain.. because no one waits at the top of a mobile website hoping that the background image might become a video.  especially for 12 (or more) seconds.

That is a long time for a customer on mobile to wait for a background video to start playing, and with 30% of sites showing a delay of 30s or more – the engagement factor drops significantly if the video is not playing quickly for your customers.

 

Once the video starts, it is important to keep enough video in the local buffer that the playback does not stall. Research shows that even the smallest hiccup in video playback has a jarring effect on the user, and can lead to video abandonment – which of course means they are no longer engaging deeply with your website.

Testing over 3G, I found that 40% of videos had at least one stall.  30% of videos had 0.5s of stalled video, 20% had at least 2s of stalled video playback.

 

Why So Many Stalls?

How can be be that 40% of the videos in the data set have at least one stall? Aren’t the players in the browser smart enough to avoid this?  Well, in general, yes they are.  But these tests were run in 3G conditions (1.5 MBPS downlink), and if we look at the video bitrates being served to mobile devices: 50% of videos are above this threshold!  Short videos may scoot through without a stall (but a longer startup time), but longer videos will play back faster than the network can supply content – pretty much ensuring a stall will occur.

All of the analysis in this post is of the 3200 videos that successfully played in 200s on a 3G connection.  But 12% of the videos were unable to play on 3G.  They are large in tonnage, dimension and duration.  But the primary reason that they never play on a 3G connection is the bitrate: 90% of these videos are attempting to download a video that is significantly larger than the available network bandwidth. Combining this with the larger screen sizes, duration (and 81% ship with audio streams), and these videos never had a chance on mobile.

Retesting these videos using WebPageTest on a Moto G4 and a full unthrottled network connection – a full 25% still did not playback in the 200s allotted – mostly because they were longer than 3 minutes..

Percentage Time in Playback

If we sum up the total time for video playback (startup time + playback time + stall time), we can determine the amount of time the video was actually playing:

What this means is that the median video spends just 56% of potential playback time playing the video.  44% of the time the video is starting up, or stuck in a stall.

 

So Which Characteristics Lead to Slow Startup?

With all of this data, let’s look to see what will lead to a slow video startup.  If we break down the metrics and determine the percentage of videos that will startup in under 6 seconds, we come up with some interesting (although not terribly surprising) facts.

 

Larger Videos Tend to Take Longer To Start

84% of videos under 593KB start playback in under 6s, 53% of 2.6 MB videos start in under 5s, but only 30% of 3.7 MB videos start that quickly.  Larger KB videos are more likely to have a delay in startup than shorter videos.

Longer videos are NOT An Indicator of Slower Startup

The length of the video is not a strong indicator into startup time.  The percentage of longer videos starting in under 6s is not much different than that of shorter videos (even those in the 90-100th percentile – over 76 seconds long):

Larger Dimensions Can to Startup Delays

Two thirds of videos with a width of 720 pixels start within 6 seconds of page load, while only 22% of 1280 pixel wide (720p) videos, and 13% of 1920 (1080p) videos start that quickly.

While it is clear that larger pixel dimension videos CAN start quickly, the majority do not.  This indicates the importance of performance optimsations on video encoding. Compression and quality optimisations can enable websites to deliver large videos quickly, but that may websites are not performing these optimizations.  I’ve blogged in the past about video optimisation options to reduce file size.

Conclusion

In examining 3500 videos that are live on mobile websites to autoplay, we find that when testing on a 3G connection, and allowing 200s for the video to download and playback, only 88% of the videos were able to complete playback.

Of the videos that successfully played in 200s, the median video is 3.8 MB, and 20s long. The most common aspect ratio is 16:9, with 720p and 1080p video dimensions dominating.  The median startup time is 13s, and 40% of the videos in the data set experienced a stall during playback.

50% of videos in this study have a bitrate that is greater than the downlink speed of a 3G connection – meaning that video playback will be delayed and contain stalls.

I also found that videos that are larger in KB and with larger dimensions are more likely to have a slow startup, and that the length of the video does not play a large impact in video startup time.