Video content on the the web is increasing – with such an immersive medium – it is pretty much a . no brainer. However, a few months ago, I looked into how the web is using video on the web, and found that slow video startup times and stalls are much more common than we would all hope.
Video metrics generally focus on 3 attributes: How quickly did the video startup, did the video stall, and did the video have high enough quality. In this post, I’m going to describe StreamOrNot, a tool that helps developers accurately measure the first 2 metrics: will playback be fast and stall free?
Measuring Video Performance
One of the greatest tools for measuring web performance is webpagetest.org. By understanding what assets are being loaded (in which order), one can work to discover inefficient downloads, and work to optimise – thus speeding up the page.
Here’s a WebPageTest waterfall of a website with a background video:
The big long green line? That’s the video. It is 4 MB (75% of the total page weight) and takes 8 seconds to download.
This is all great information – we know this page has a video, how large it is, and how long it took to download – but we don’t get any meaningful data about whether the video started playing. Or if the video had a stall. In order to test how videos play back – we have Stream or Not!
Measuring Video Playback Stats
To run Stream Or Not, Just enter the url of your video and hit go. This will show you how fast your video will load, and if it will stall on your internet connection.
The default test in SON only downloads one file – the video. Your website probably has a bunch of other files that are being downloaded at the same time, and that will affect the video download. StreamOrNot allows you to choose between 0-100 images that can be downloaded at the same time as your video.
Note: Video on StreamOrNot is prioritised with priority hints, giving the video high priority over all other files. This is to ensure that the images and video are downloaded together, Generally – you should not give video files priority over all other downloads on your website.
Adjusting The Network Speed
The test as described is based on your internet connection. Frankly, your connection is probably faster than that of your users. To test your video playback at different speeds, you can use devTools to throttle your connection speed (And yes, I have added custom speeds based on WebPageTest)
I’d recommend testing on the Cable and Fast 3G connection speeds to see how your video plays on slower connections.
As your video plays back, you will receive live information on how long it took for the video to begin playback, how much video is in the buffer (and if it is close to stalling), and a count and time spent in the stall state.
- The size of the video (dimensions and number of bytes)
- On Chrome, the connection speed is shown (navigation.connection is Chrome only)
- Video playback state. Possible options are
- Not Started
- empty buffer
- Video Start time. Measured from start of JS execution to when the video begins playing. The longer this takes, the text color changes to yellow and then red.
- Playback time (in seconds)
- Remaining time in buffer (in seconds)
Video buffer chart:
Under 5s – stalls are imminently possible.
5-10s – warning – stalls might occur soon.
10s – green – You’ve got some video stored, and if you keep it up, you’ll be ok.
The chart is an SVG that is drawn in real time. I think that’s really cool 🙂
In the example above, the video starts up very quickly, but it is very high bitrate, and the buffer is never able to fill up, and there is a short stall every second or so, making the video very janky and unpleasant to watch. The stall report (generated at the end of video playback shows this very clearly:
Automating Video Playback testing
To automate this testing, and to ensure similar conditions for all of your tests, you can utilise WebPageTest and Andy Davies’ Google Sheet to automate the tests. The details on how to do that are all on GitHub.
Videos that are slow to startup or stall generally have a higher bitrate. This means that the video plays back faster than the network can download the video. Lowering the bitrate of your video (lowering the quality, changing the dimensions) will help your video play back faster and with fewer stalls.
There have been no tools to test how well a video plays back on the web. Stream Or Not is a synthetic testing tool that enables you to quickly determine how well your video will play on different types of networks or devices. Give it a go, and please provide feedback or suggestions!