Sitecore 9 : Config Patches – Layers & Roles

I’ve previously written several posts on Sitecore Configs. Well Sitecore 9 just landed so it time for a big update! This is not going to get your marketers all giddy but will certainly make all you developer and developers super happy.

Over the years the number of configs in Sitecore has grown and grown and grown. Then so did the number of folders. Sitecore has a new trick up it’s sleeve to clean this all up.

What’s new? Rules Based Configuration.

It’s a new configuration approach designed to drastically improve how server roles are configured and how all the config files are now organised, making it both easier to manage and deploy.

Configuration Roles

I hope you all kept your hair that you pulled out cos you can glue it back in now!

One of the biggest headaches when installing was enabling and disabling a whole heap of config files depending on the type of server you were trying to install. The official documentation and excel spreadsheet let to countless variants of PowerShell scripts to parse the files and enable/disable automatically. I’m sure that a lot of us had some variant of these scripts in house as well.

Time to rejoice. Praise the lord. Hallelujah!

In order to configure a server now you simply have to update the following setting in web.config (use a config transform yo!):

<AppSettings>
  <add key="role:define" value="[server role]"/>
</AppSettings>

You can specify an of the following server roles:

• ContentDelivery
• ContentManagement
• Processing
• Reporting
• Standalone

By default, the instance is configured as Standalone, meaning it will perform all roles and making it easier for quickly standing up an instance or for local development purposes.

You can combine server roles, which would be common for a CM/CD server setup, by comma separating the roles. For example:

<AppSettings>
  <add key="role:define" value="ContentManagement, Processing, Reporting"/>
</AppSettings>

Configure Settings for Roles

You can configure whether a particular setting or config node should be patched for a specific server role using role:require attribute. The default Sitecore configuration is all now tagged with the server roles so out of the box simply setting the role:define key is sufficient to configure the server.

Some examples:

<agent type="Sitecore.Tasks.HtmlCacheClearAgent" role:require="ContentManagement">
  <patch:attribute name="interval">00:05:00</patch:attribute>
</agent>

Adds the HtmlCacheClearAgent scheduled task only when server is configured as ContentManagement. There’s plenty of things you don’t want included in CD, such as certain event handlers as well.

<sc.variable name="defaultLinkDatabaseConnectionStringName" value="core" />
<sc.variable name="defaultLinkDatabaseConnectionStringName" role:require="ContentDelivery or ContentManagement">
  <patch:attribute name="value">web</patch:attribute>
</sc.variable>

Sets the variable as core for all server roles, except ContentDelivery or ContentManagement where we patch the attribute to web. Note that you can specify this in the same config file.

<database id="master" 
          singleInstance="true" 
          type="Sitecore.Data.DefaultDatabase, Sitecore.Kernel" 
          role:require="Standalone or Reporting or Processing or ContentManagement">
  ...
</database>

Goodbye zzzSwitchMasterToWeb.config! No need to switch out and patch delete all those references to master database any more. Remember, don’t forget to remove master from connectionString.config, you can’t remove that with a patch file.

 <configuration xmlns:role="http://www.sitecore.net/xmlconfig/role/">
    <sitecore role:require="ContentManagement">
      <events>
        <event name="item:saved">
          <handler type="Website.Class1, Website" method="OnItemSaved" />
        </event>
      <events>
      ...
      <!-- lots of other stuff -->
      ...
    </sitecore>
</configuration>

Want to only enable an entire config file for a particular role? Fine, just set the role:require attribute at the top most sitecore node. This allows you to essentially disable entire sections of configuration for different server roles.

Take note of the new role namespace declared in the configuration role.

Don’t forget to add this to your override configs, in much the same way you would for patch and set.

Amazing job. You no longer need a patch for your patch for your patch for your patch… Just deploy everything and set the server role!

You’re probably wondering that this all seems a bit familiar right? It’s essentially the Sitecore Configuration Roles project by Alen Pelin. Those of you early adopters and eagle eyed viewers may have noticed the option to add configuration roles when installing with the QA version of SIM:

Configure Search Providers

Previously switching search providers has also led to similar issues with having to enable/disable a set of different files. Switching to a different search provider is also now just as easy as changing server role, you just need to change one different setting in web.config:

<AppSettings>
  <!-- SUPPORTED SEARCH PROVIDERS     
       Specify the search provider that you want this server to use. The supported search providers are:
       
       Lucene
       Solr
       Azure
       
  Default value: Lucene
  -->
  <add key="search:define" value="Solr" />
</AppSettings>

Just like the server role configuration, a simple single setting. Within the search configuration files the nodes now have an additional attribute set. This all follows the same pattern to the rest of the configuration but note that the attribute matches the search:define setting above and as such the search configs have an search:require attribute set.

The Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config starts with the following, excluding the entire configuration from being patched in unless SOLR has been set as the search provider:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
  <sitecore search:require="solr">
    <contentSearch>
      <indexConfigurations>
        <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider">

Similarly, the Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config starts with the following:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
  <sitecore search:require="lucene">
    <contentSearch>
      <indexConfigurations>
        <defaultLuceneIndexConfiguration type="Sitecore.ContentSearch.LuceneProvider.LuceneIndexConfiguration, Sitecore.ContentSearch.LuceneProvider">

It’s worth noting that both server roles and search provider configuration can apply to a single node. The Sitecore.ContentSearch.Solr.Index.Web.config file has the following configuration:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
  <sitecore role:require="Standalone or ContentDelivery or ContentManagement" search:require="solr">
    <contentSearch>
      <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">

Note that both role:require and search:require attributes have been set.

Custom Configuration Roles

The role definition are just strings, and the defaults server roles are what Sitecore provides and uses for it’s own config. If you need you can define your own custom roles:

<AppSettings>
  <add key="role:define" value="ContentDelivery, EnvironmentUAT"/>
</AppSettings>

And then you can annotate your custom configuration with the required roles:

<!-- default setting for all server -->
<setting name="MailServer" set:value="smtp.liveserver.com" />
<!-- override for UAT environment -->
<setting name="MailServer" 
         set:value="smtp.testservice.net" role:require="EnvironmentUAT" />

In theory this means you could completely avoid using the Environment layer folder… Use whichever approach you prefer I guess… Some people like having separate environment specific files, other prefer keeping all the settings together…

EDIT: Although this works, I was told of a much better, cleaner and semantically correct way of doing this. Please follow this updated article I wrote instead.

Role Combinators

So far the example for the require annotation have used the OR operator to allow configuration to apply to multiple different server roles:

OR Operator

<setting name="ItemCloning.Enabled" role:require="ContentManagement or Standalone">
  <patch:attribute name="value">true</patch:attribute>
</setting>

AND Operator

It’s also possible to use the AND operator to only apply configuration when the server must match multiple roles:

<AppSettings>
  <add key="role:define" value="ContentDelivery, EnvironmentUAT"/>
</AppSettings>

<setting name="ItemCloning.Enabled" 
         set:value="true"
         role:require="ContentManagement and EnvironmentUAT" />

This allows us to do some nice combinations in a single patch like so:

<sites>
  <site name="fancysite" 
        patch:before="*[@name='website']"
        rootPath="/sitecore/content/fancy-site"
        startItem="Home"
        inherits="website"
        hostName="cms.fancysite.com" />
                    
  <site name="fancysite" 
        set:hostName="www.fancysite.com"
        role:require="ContentDelivery" />
        
  <site name="fancysite" 
        set:hostName="uat.fancysite.com"
        role:require="ContentDelivery and EnvironmentUAT" />
        
  <site name="fancysite" 
        set:hostName="uat-cms.fancysite.com"
        role:require="ContentManagement and EnvironmentUAT" />
</sites>

Grouping Operator

You can use parentheses to Group logical operator combinations, for example:

<setting name="MailServer" 
         set:value="smtp.internal.net" 
         role:require="(Standalone and EnvironmentTest) or (ContentManagement and EnvironmentUAT)"  />

NOT Operator

You can use a NOT operator to exclude the setting from being applied to specific server roles:

<database id="master" type="..." 
          role:require="!ContentDelivery">

The default Sitecore configurations do not use any of these apart from the or operator, opting instead to “or” multiple roles rather than negate a single type. This is a good thing IMO as it allows you add additional roles but not have those settings apply and cause issues.

Folders, Folders Everywhere

About a year or so ago I wrote a post about overriding the Sitecore Config Reader in order to add additional folder locations to load configs from. We now have a very nice folder structure to keep things super clean.

The configuration has now been divided up into 4 different layers: Sitecore, Modules, Custom, Environment

Folder	Location	Purpose
Sitecore	/App_config/Sitecore	Contains all the default Sitecore configurations files. Don’t touch anything in here, if you need to change something then use a patch file in the Custom layer below. Making changes in this layer might cause something to stop working or cause you issues in the future when you try to upgrade.
Modules	/App_config/Modules	Contains configuration files for Sitecore modules, such as Web Forms for Marketers. Modules configs will be loaded in the order which they were installed. Again, do not change any files in here, use a patch in Custom.
Custom	/App_config/Include	This the folder in which you should create your own patch files in order to modify Sitecore settings. You may have noticed that this is the exact same folder as in previous Sitecore versions. So if you had kept things clean, only used patch configs and kept things in a `z.Project` folder, then an upgrade should mean you need to do very little.
Environment	/App_config/Environment	This is the ultimate override folder. It is meant for non-production environment configs, such as QA, Test or Pre-Prod. For example, you may wish to add patch files to specify different SMTP servers, account keys or enable a higher logging level.

Just to re-iterate:

Important
Do not modify or add files in the Sitecore or Modules layers. Changes to files in these layers can be lost during an upgrade.

As you can see, each dedicated layer affects a different part of Sitecore. No more mix-and-match of default and custom configs! Just like before, Sitecore loads config files from each folder alphabetically and then traverses each subfolder alphabetically.

Thank you Sitecore, you can also close the UserVoice request, at least 23 of us will be happy 😁

Controlling the load order of configuration files

Did you ever hear that joke, “what came first, the foundation or the feature”?

One of the “issues” with configs loading alphabetically is Feature configs would load before Foundation, because that’s how the alphabet works… Perhaps it would have been better to have named it Base or Core (UpperCase C) or something… But not really a huge issue when you can name them “1. Foundation”, “2. Feature” and “3. Project”.

Here’s the problem though: default install of Sitecore 9 has almost 50 folders by default in the Sitecore config layer. Numerically naming folders will become tiresome, and then of course someone comes along and wants to stuff in a new folder between 6. and 7. and you’re on a renaming strategy…

To solve this issue, Sitecore now allows you to specify the load order of config folder and files. Load order is specified in /App_Config/Layers.config by adding “ settings:

<layers>
  <layer name="Sitecore" includeFolder="/App_Config/Sitecore/">
  ...
  </layer>
  <layer name="Modules" includeFolder="/App_Config/Modules/" />
  <layer name="Custom" includeFolder="/App_Config/Include/">
    <loadOrder>
      <add path="Foundation" type="Folder" />
      <add path="Feature" type="Folder" />
      <add path="Project" type="Folder" />
      <add path="SomeOtherFolder/loadmefirst.config" type="File" />
      <add path="AnotherFolder" type="Folder" /> 
    </loadOrder>
  </layer>
  <layer name="Environment" includeFolder="/App_Config/Environment/" />
</layers>

Check the default Layers.config and you’ll see the huge list of the Sitecore folders specified by default.

Keep in mind that the layers config is not within the Sitecore node and therefore you cannot patch it. Why? Because it’s a chicken and egg situation, you cannot load a Sitecore setting to specify the location to load that Sitecore setting… It’s the reason I previously used an AppSettings key as well.

You must make modifications to this directly (well, use a Helix style config transform). Be careful with this during upgrades and be careful when you install modules, since modules will update the layers.config when they are installed and add a load order. You should back port the changes into your project… For some modules load order may not matter but other modules may rely on a specific module being present (think SXA having a dependency on PowerShell Extension for example).

Moar, Moar, Moar Folders!

You still want more folders for some reason? Who am I to judge…

<layer name="MoarFoldersPlease" includeFolder="/App_Config/Moar/" />

As simple as that. Add another layer into the layers.config file and Sitecore will automagically load all configs from it. Remember, don’t patch this before the Sitecore or Modules folder otherwise you might get unexpected behaviour.

Disabling Configs

If you need to disable certain configs then this works like it has always worked, Sitecore will only load files with a .config extension so you can rename to something else like .disabled and they will be ignored on load.

You can also disable a config by specifying the path and adding mode="Off" attribute. For example

<add path="MyProject.Custom.config" type="File" mode="Off" />
<add path="Server-UAT" type="Folder" mode="Off" />

Still need more? You can also disable an entire layer using the same attribute, which is an easy and quick way to disable all your customizations in the game of “Is it me or is it Sitecore”:

<layer name="Custom" includeFolder="/App_Config/Include/" mode="Off">
  ...
</layer>

This could also be used to disable (or enable) Environment layers folders for example, instead of having to delete them.

Still too much work? Don’t want to modify any files at all? Remember, you can disable files and folders by making them hidden in Windows.

View Configuration Changes

The roles and layers add an extra dimension of complexity when trying figure out what the final merged config will be. Previously we could use showconfig.aspx page on each server, and now Sitecore has made some changes to allow you to simulate the changes the would be applied per role.

A new admin page has been added at /sitecore/admin/Showconfiglayers.aspx:

In turn, the tool will load the showconfig.aspx page with some additional URL parameters, e.g.

/sitecore/admin/showconfig.aspx?layer=Sitecore|Modules|Custom|Environment&role=ContentDelivery

You can read more in the Sitecore documentation.

Much Options, Such Wow

As you can see, all the additions to the config patching are very powerful, flexible and extensible.

So much awesomeness and a great addition to makes everyone’s lives easier, not just during development and deployment but also during upgrades.

I love the fact the deployments to all servers roles in all environments could all use the exact same build package and just flipping a single switch you can change server role or configure for a different environment.

How easy is it now to include a Developer Performance configs and add a role:require="Standalone and LocalDeveloper" attribute so it only applies on development environments. All we need now is @kamsar to update them 😊

TODO: Everyone update your Modules

Assuming that you module code is still compatible with the Sitecore codebase, for the most part, you shouldn’t need to do anything since the default Custom layer still points to /App_Config/Include. You should consider moving the module configuration to the Modules layer and make use of the Server Roles functionality to make the configuration of your modules easier and more consistent with the new config features.