Redirects, and their Effect on Performance or How a (Seemingly Minor) Third Party Change Affected the Website Performance of a Popular JavaScript Bundler

Redirects

When a server redirects a request – the browser has to make a second request for the file, adding (at least) one additional round trip, and delaying the delivery of the content to the end user.

As I was sitting around one day, I wondered ‘what is the maximum number of redirects on a single page?’  HTTP Archive to the rescue. In the summary_pages table, there is a numRedirects field. And, running a quick search in BigQuery, I discovered a page with over 1100 redirects:

Of course, it is doing some weird denial of service attack on Gravatar’s default image – all of the 1100 redirects (ok, all of the ones I looked at – I got bored pretty quickly)  are pointed at the Gravatar skeleton image, which is a little odd.  

An Unexpected Site

As I continued down the list of websites with a large number of redirects, I saw a familiar name in the top 10.  I discovered that the homepage for webpack (https://webpack.js.org) had over 600 redirects on its homepage.

Webpack is a bundling tool for JavaScript, and has many ways to help optimize the delivery of content (see Web Performance Optimization with webpack).  For this reason, I was surprised to see so many redirects on this page! Unfortunately, a quick test with WebPageTest confirmed the matter, and also showed that the 302 redirects were not being cached, and the return visit was suffering the same 600+ 302 redirect penalty.  

 

Screen Shot 2018-11-13 at 9.22.32 PM.png

I’d paste the full waterfall into this post, but then you’d just be scrolling forever, and give up on what I had to say after a while.  If you are curious, you can see the test results here: https://www.webpagetest.org/result/181023_JP_b9476f132f954277f10ab45938be4702/

Diagnosis

What is going on?  Well, Webpack is fortunate to have a lot of sponsors, and they list many of them on their webpage.  And each image was leading to a redirect:

Every request to:

https://opencollective.com/proxy/images/?<id>

Redirected to:

https://images.opencollective.com/proxy/images/?<id>

And the redirect was not being not cached:

cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Meaning that the same redirect occurs on subsequent visits  The webpack page is open source, and digging around, I found an API call to https://opencollective.com/api/groups/webpack/backers

That returns a (really large) JSON file with all of the backers, including a URL to their image.  And this was he source of the redirect.

So, we have found a top tool’s website whose 3rd party API is adding hundreds of extra requests to load the webpage.  

 

Further Digging

We can fix this! Looking in the Opencollective Github repo, I found that the image subdomain change was made on a 9/17 commit on Open Collective.  Nothing nefarious here, just a simple migration.  Could this be the culprit? Luckily, with the HTTP Archive, we have historical data!

Screen Shot 2018-11-13 at 9.29.26 PM.png

Looking at the 9/1/18 HTTPArchive, webpack.js.org has 0 redirects, but there are 649 redirects on the 9/15/2018 HTTP Archive crawl.

My hunch was that the 9/17 commit caused the issue, so this made me worry that my guess was incorrect.  With over 1.2 million webpages tested in each HTTP Archive run – they cannot all be run in one day.  The test for webpack.js.org was run at UNIX time 1537493496, which translates to 9/21/2018 – 4 days after the push. OK, I this pretty much seals the deal.

Filing a Defect – and Resolution

I filed an issue on the opencollective GitHub repo:

https://github.com/opencollective/opencollective-images/issues/10

And it was resolved last week.  Now the webpack.js.org homepage again has 0 redirects, allowing all of those images to load in just one request each!

https://www.webpagetest.org/result/181108_98_7f290d38486ac8ce6505ad553c4abea9/

Screen Shot 2018-11-13 at 9.33.16 PM

Conclusion:

Redirects add extra round trips to deliver content – delaying page load and lowering performance.  Sometimes changes in third parties that your site depends on can lead to catastrophic changes in your sites performance, so regular testing is essential to catch these issues quickly.

Tools like HTTP Archive and WebPageTest are indispensable for helping to pinpoint issues, and should be part of your toolset to identify issues.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s