Mobile Web and Video Streaming:  What trends do we see?

In previous posts on video streaming, I’ve looked at how video adaptive streaming works and some of the performance impacts on video on the mobile web.  In this post, I’d like to combine these two ideas by looking at how websites are streaming video, and see if websites are following the best practices I have proposed.

For my dataset, I’ll be using the HTTPArchive crawl from 12/15/2017 of the mobile web, and I’ll specifically be looking at HLS streams (as HLS is currently the predominant streaming mechanism).  In HLS streaming, the first request in a video stream is the manifest (and in HLS streaming, the manifest has with extension m3u8).  Every time the bitrate changes, and additional manifest file is requested and downloaded.  For that reason, it is very common to see multiple manifest files per website, and the data corroborates this.  If I do a query for requests with the m3u8 extension, I find 9464 m3u8 files requested in the dataset, across 1974 different pages.

SELECT
  pageid,
  COUNT(ext) AS cnt
FROM (
  SELECT
    pageid,
    ext,
  FROM
    httparchive:runs.2017_12_15_requests_mobile,
  WHERE
    (ext = "m3u8")
  ORDER BY
    requestid )
GROUP BY
  pageid

The median is 2 manifests per page, but as shown below, there are several outliers with over 68 manifest file requests during page loading:

Picture1.png

Now, once the manifest is downloaded – the browser knows what video files to download to commence the stream (as they are enumerated in the manifest file).  The player can begin downloading the stream, or it can wait until the user clicks the play button.

In the HTTPArchive data, 1187 sites (60%) that have at least one manifest file request, but 0 video segments.  For mobile, this is ideal, saving cellular data by now actually downloading the video files until the video is started by the user. The median site with manifest file downloads, but no video segments has a SpeedIndex of 11233.

The remaining 787 sites (40%) have at least one manifest file, and at least one video segment. The median SpeedIndex of these sites is 12726, indicating about a 1500ms penalty in page load time for these sites.

Comparing Mobile to Desktop 

The desktop data is nearly reversed. 67% of sites (1747 of 2616) download both manifests and segments, while 33% (869) only offer the manifests (with no video segments).  Again, this makes sense, as desktop computers are more likely to be on a faster, non-metered network.

Screen Shot 2018-01-20 at 10.44.37 PM.png

Identifying Trends in Streaming

To better understand how these sites stream video, I need to look at individual sites, and the manifests/video segments being downloaded.  To identify these sites, I ran this query:

SELECT
   pages.speedindex,
    pages.rank,
    pages.url,
    pages.pageid,
    manifests.manifstcnt,
    manifests.segmentcnt
  FROM
    httparchive:runs.2017_12_15_pages_mobile pages
  JOIN (
    SELECT
      pageid,
      SUM(m3u8) AS manifstcnt,
      SUM(ts) AS segmentcnt,
      //count(*) bub
    FROM (
      SELECT
        pageid,
        url,
        respsize,
        mimeType,
        type,
        ext,
        IF(ext ="m3u8",1,0) m3u8,
        IF(ext="ts",1,0) ts
      FROM
        httparchive:runs.2017_12_15_requests_mobile,
      WHERE
        (ext = "ts" || ext = "m3u8") )
    GROUP BY
      pageid ) manifests
  ON
    pages.pageid = manifests.pageid
  WHERE
    (manifests.manifstcnt > 0 && manifests.segmentcnt >0 )
  ORDER BY
    rank ASC

From the resulting list, I manually looked at 18 of the top websites with multiple manifests and segments to understand how these sites set up their streaming videos for mobile consumption.

Looking at Individual Sites 

To understand how these sites download video, I use the pageID to identify all of the manifest and video files downloaded.  I then sort them by the request ID, to give a chronological list of requests:

SELECT
  pageid,
  requestid,
  url,
  respsize,
  mimeType,
  type,
  ext,
  // if(ext ="m3u8",1,0) m3u8,
  // if(ext="ts",1,0) ts
FROM
  httparchive:runs.2017_12_15_requests_mobile,
WHERE
//change the pageid to get a different site
  pageid =19484828
&&(ext = "ts" || ext = "m3u8")
ORDER BY
  Requested

The results look something like this:

Picture11

If we look at the url column in the results above, we see in row 1 the index.m3u8.  This file lists all the bitrates available for the player to download. If I select this url, and open it in Chrome (using the HLS m3u8 Manifest Viewer extension), I can quickly read the manifest file, and identify all of the available bitrates for the stream.

In row 2, the player requests the /500/prog_index.m3u8, and we see index0 and index1 downloaded at bitrate quality “/500/”.  In row 5, the player requests a new manifest: /750/prog_index.m3u8, and we see index2 downloaded at this quality.  Finally, there is a request for a 3rd manifest /1000/prog_index.m3u8, and the next request is for index3 at quality “/1000”.

Each app/player/server combo has a different naming convention for manifests and for the different bitrate video files.  In order quantify these details, I began manually parsing the files for the top 18 streams in the data set.  Let’s see what we can learn!

Compiling the Data

 Each WebPageTest run is slightly different, and each of the webpages I tested behaved slightly differently (and downloaded a different number of segments).  This is a very small sample set, and each one of the sites has a slightly different behavior.  But I think looking at these results can help us understand some basic tenants of video streaming.

For the 18 sites I looked at, I tabulated (from the manifest) the available bitrate streams.  I then looked at the files downloaded, and identified which streams were used.  I color coded the bitrates used red (first), orange (second) and yellow (third).

1Picture1

Initial Bitrates

Let’s look at the initial bitrates set for each of these streams (the first column).  The initial bitrates vary from 0.24 – 3.44 MBPS, indicating that there is a large discrepancy in how initial streaming of video is handled across the web.

A best practice for initial video bitrate is to use an intermediate bitrate for the initial stream – good “enough” quality, but also ensuring a fast download to prevent a long startup delay.

Recall that these HTTPArchive runs use a throughput of 1.5 MBPS for all data traffic.  Since the video is being downloaded at the same time as other content, the max bitrate for streaming is likely around 1.3 MBPS.  11 of the 18 sites begin streaming below this threshold, and 6 are above.  The median initial bitrate is 0.752 MBPS, a good balance between video quality and ensuring good delivery quality.  The videos that start at very low throughputs have a pretty low range of available bitrates (none over 1 MBPS – indicating a desire for the streaming to work on mobile, but not a huge concern on the quality of the stream.

Bitrate Ranges

We can also use the data in the manifest to examine the bitrates that are offered by the streams. What I see is there is no real standard for video bitrates being used – the values are different for every site examined. Some of these sites appear to have mobile specific manifests – with appropriately lower bitrates/video qualities.  Others have a huge range, likely meaning the same manifest is used on all device formats.

The median bitrate range is 2.18 MBPS (average 3.2 MBPS).

To ensure smooth streams and jumps in quality that are less noticeable, it makes sense to have changes in bitrate that are ‘smooth’ across the range offered (there are fewer large jumps in quality).   If we look at the range and the number of bitrates, we can calculate the average bitrate “jump.”  For streams with smaller ranges of available bitrates (likely optimized for mobile) we see that these jumps can be as small as 0.15 MBPS (where users are unlikely able to see any change in quality).  Most of the other streams cluster their jumps in quality to around 0.5 MBPS or 1 MBPS.

2Picture1

Changes in Bitrate

When the player determines that the video cannot be downloaded fast enough to prevent a stall, it will change the bitrate to a lower quality.  Similarly, if the player believes it can increase the quality of the video, it will request a higher quality stream.

Of the 18 streams examined, only 5 never deviate from the initially selected bitrate (28%).  10 (59%) streams change once, and the remaining 4 (22%) change video quality twice. Many changes to video bitrate leads to frequent (and sometimes noticeable) quality changes that degrade the watchability of a movie.

8 of the videos ended at a lower quality stream than initially selected, and 4 jumped to a higher quality video. The largest bitrate increase was 0.65 MBPS, while the largest decrease was -1.94 MBPS.  The fact that more videos dropped in quality indicates that the initial video quality is generally being set too high for mobile devices running on 3G.

Conclusions

Mobile sites have around the same frequency of streaming video as desktop, however, the number of streams with downloaded video segments is much smaller.  This is great, and reduces the data load on resource constrained mobile devices (and saves your customer’s data bucket!). Sites with video have a higher SpeedIndex than those hat do not.

Digging into specific examples of streaming implementations, I find that there are a wide range of video bitrates presented to mobile customers.  33% of the streams analyzed initiate at a speed over 2 MBPS, leading to a large amount of data consumption (assuming that the device has adequate throughput) , or a delay (as the player must request a new bitrate to start the playback).    All videos examined have low bitrates suitable for streaming on a 3G network, even though it may take a bitrate change (or two) to correctly identify it.

Several sites have mobile specific manifests (ensuring that only bitrates appropriate for mobile devices are served), while others offer every possibly bitrate (including high quality videos that would not stream in HTTPArchive tests, but would be too high quality for a mobile device anyway).

Videos that had the fewest bitrate changes tended to start in the 0.6 – 1.2 MBPS range, a nice tradeoff in video quality and mobile data throughput.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s