Domain Sharding Media Requests in Sitecore

There have been a number posts about using Sitecore with a CDN provider and there is even a Sitecore CDN Connector module available Marketplace. Although the benefits of using a CDN provider are fairly well stated, such as reducing load on your server and allowing distributed datacentres, Sitecore itself does a pretty good job with its Media Library and for the vast majority of projects using a CDN may be overkill. Bedsides, there have been various projects I’ve worked on where they did not want the additional expense for little benefit or were unable to use a CDN to due to company policy or project requirements/specifications.

A project I was recently working on was utilising Edge Caching services from Akamai. If you’ve never used this type of service before, you need to re-route requests to your site to Akamai (by changing the DNS entry to point there). You can then configure Akamai as to which type of content should be served from cache and which needs to be forwarded on to your own server. There is nothing else for you to change, i.e. you do not need to provide a different domain prefix for your media items. This was an existing setup to the project from a previous vendor and seemed to work pretty well to meet the clients needs. One of the benefits we lose from not having a different domain name is the ability to domain shard.

What is domain sharding and why is it beneficial?

All web browsers place a maximum limit on the simultaneous connections that can be made to a domain. These connections are used to request your html content, javascript and css file(s), images, AJAX requests etc. Essentially any individual request that needs to be made to your server is placed within this connection limit. Different browsers have different limits, all having grown from the initial 2 simultaneous connections limit established in 1999 in the HTTP/1.1 specification by the Internet Engineering Task Force (IETF). Once the limit is reached any further requests are queued until a slot becomes free again, and this wait time is referred to as blocking.

Chrome DevTools Timing

You can find the connections per hostname limit on Browserscope:

Browser Connections Per Hostname
IE 9 6
IE 10 8
IE 11 13
Safari 7.0.1 6
Chrome 34 6
Firefox 27 6
Android 2.3 8
Android 4 6
Chrome Mobile 18 6

Interestingly, Internet Explorer 11 leads the way with up to 13 parallel connections!

Domain sharding is a technique used to “trick” browsers to open more simultaneous connections than would normally be allowed, thus accelerating page load times. It’s especially beneficial for users with high speed internet connections. This technique works by using multiple sub-domains to serve content. I’m sure you’ve seen this previously on many sites where the images are served from a sub-domain such as http://media.example.com

This technique works because web browsers recognize each domain name as being a different server, even if that is a sub-domain (a.domain.com, b.domain.com etc) and even though those sub-domains may resolve to a single server.

If you have ever run YSlow or run an Audit from within the Chrome Developer tools then one of the recommendations is to Parallelize downloads across hostnames:

Chrome DevTools Audit

Fake It Til You Make It!

Up until recently the technique I had used for CDN or domain shard would have been to change the Media.MediaLinkPrefix. This is the technique that most people seemed to have been using also [here, here and here]…

<!--  MEDIA - MEDIA LINK PREFIX
	The prefix to use when Sitecore generates media links. The setting is used in the front-end as well as the back-end.
	Notice: If you specify a custom media link prefix, you must also add acorresponding entry to the <customHandlers> section.

	If the value is not set, the default media request prefix will be used (which by default is "~/media")
	Default value: ""
-->
<setting name="Media.MediaLinkPrefix" value="/media.domain.com/~/media" />

A recent blog post from Eduardo Moraes about Integrating Sitecore with your content delivery networks alerted me to a setting that I had somehow overlooked. Since Sitecore 7.1 rev. 140324 (Update-2) a config setting was introduced that lets you specify the server URL to use for media links:

<!--  MEDIA - MEDIA LINK SERVER URL
	The server URL to use when Sitecore generates media links and when Media.AlwaysIncludeServerUrl is set to true. This is typically
	used when all media is served from one or more dedicated instances or when your solution is configured to store Sitecore media on
	a content delivery network. 
	The URL must use this format: <protocol>://<hostname>, for example http://example.com 
	If the value is not set, the URL of the current server will be used.
	Default value: ""
-->
<setting name="Media.MediaLinkServerUrl" value="//media.domain.com" />

So a poor mans “CDN” to achieve domain sharding is to set the MediaLinkServerUrl to a sub-domain and point it back to the same server instance as your CD site. None of the server performance benefits but some of the page load benefits. The comments state that a protocol is required but you can use Protocol-relative URLs if you are switching between regular and HTTPS connections.

Other changes required

You need to set Media.AlwaysIncludeServerUrl=”true” and also add the sub-domain to the hostName attribute of your site defintion:

<site name="mysite" ... hostName="domain.com|media.domain.com" />
<!-- or you can use a wilcard to handle all sub-domains: *.domain.com -->

You also need to add a binding of your sub-domain in IIS.

Domain Sharding IIS Bindings

With the settings in place and our media links always served from our sub-domain we now get the benefit of domain sharding and an increased parallel connection limit.

Domain Sharding Chrome Network Panel

As a general guideline, 2 domains is a good number of domains to shard across.

Buyer beware

Although using a dedicated media server or a CDN this technique will cause no issues, I haven’t used this technique for domain sharding from the same server on any production site yet, nor have I run any performance tests yet so it all very theoretical. It is very likely that since you are still serving all requests from the same server (unless you are using Edge Caching) then the concurrent load on your server can be expected to increase. The load over a given period will be the same, I’m talking about simultaneous requests so expect your server traffic to spike whilst serving a page to any given user. You’ll need to decide whether you have enough spare capacity on your servers and any load performance increases is beneficial enough.

In my current situation, using edge caching I expect to see increased page load performance.

Domain sharding is not always beneficial and will depend on a number of factors. The default connection limits may well be more than sufficient, and if most of your connections are used up by media requests then you are simply moving the issue from one domain to a sub-domain. If you have a mixture of images, JS, CSS and AJAX requests then you should see more benefits. You should consider use of other techniques also, including bundling, minification and use of sprites.

Multisite Implementation

In a multisite implementation and for the sake of vanity you make want each site to prefix the media URLs with their own sub-domain. I would use the same technique mentioned by Eduardo and simply set the URL to a sub-domain. In this instance, if you left the default “website” site definition alone then you do not need to specify the sub-domains for each site, since the default site will handle these domains as no hostname is set. Since there is no content, the media will be handled without any issues.

<site name="mysite" hostName="mysite.com" ... />
<site name="website" ... />

Further reading:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s