ESI : Rendering Dynamic Content | AEM

What is ESI ?

Edge Side Includes or ESI is a small markup language for edge level dynamic web content assembly. The purpose of ESI is to tackle the problem of web infrastructure scaling. It is an application of edge computing.

Static Content vs Dynamic Content ?

Let’s assume we have dynamic website having most of the pages are mixture of static and dynamic content. The portion of the page should be un-cacheable and rest of the page should serve from the cache. The question here is, how do you ensure the cache won’t be impacted by having dynamic content on the page. A few possible options are –

Option 1 : Load dynamic content Asynchronously  (AJAX)

Option 2 : Load dynamic content using ESI.

I would say both the options are the best, but you should choose based on type of the website and infra considerations . Having said that, let’s talk about the scenarios when to choose what ?

When to choose AJAX ?

  • When SEO is not a constraint. 
  • Best for SPA type applications.
  • UGC heavy website.
  • Page response time shouldn’t be a constraint. Usually slow.
  • Hard to perform A/B testing when you inject personalised content asynchronously. You should choose when this option is less relevant.  

When to choose ESI ? 

  • Multi Page Application.
  • SEO guidelines has to be met.
  • Dynamic header/footers that are updated frequently.
  • To avoid invalidation of entire site when header/footer keeps changing. 
  • Scalable and dynamic website without writing a new code. 
  • To minimise number of requests per page.
  • Highly scalable CDN server that supports ESI.

There could be more, but i hope above pointers will help you take a decision on which direction you should go with.

NOTE : Suppose you choose esi as an option to render dynamic content, but in few cases you can’t really use esi  because the nature of the component or fragment. Example : Advanced Search( Personalised ) – You can’t really expect the page to be reloaded every-time when user tries to search or apply any filters. In this case AJAX is inevitable and it gives you better user experience. Hybrid option is always a possibility. 

 

How to setup SDI in AEM ?

 

SDI Set-up 

https://experienceleague.adobe.com/docs/experience-manager-learn/foundation/development/set-up-sling-dynamic-include.html?lang=en 

Above like has complete details of how to configure in publishers & Apache. 

 

 

My setup looks like above, but it may not be the same for every implementation. Since all the caching layers like Dispatcher, Varnish & Akamai have the capability to include dynamic content( ESI or SSI ), it can be in any combination. Having varnish onboarded has it’s own advantages since it is also implemented(partially) ESI 1.0 specifications, we can leverage it for lower environments for non-akamai servers and acts like your ESI processing server to perform the testing before it goes to production. 

 

Realtime Problems ?

So far we talked about when to choose esi vs ajax and how it works. Now, let’s see what real problems we get into during implementation. 

#Challenge 1 – Your dispatcher cache ratio might be impacted if you don’t differentiate b/w authenticated and non-authenticated requests. 

In majority of the cases same components has to behave differently when you are authenticated or non-authenticated.  Same component has to render different content based on the context. Since all the esi calls have the selector ‘.nocache‘, these request will be bypassed and delegates to publishers directly. How do you ensure non-authenticated calls to be cached in the dispatcher ?

Let’s take an Example :

Component Name :  Recommended  Products

ESI URL :  <esi:include src=”/content/website/us/en-us/_jcr_content/recommended_products.nocache.html” />

As i said, above call always delegates to publisher irrespective of the user is authenticated or not because of ‘.noache’ selector.

Fix

Implement a simple rewrite rule in Apache to rewrite the url coming with ‘nocache.html’ –> ‘anon.html’ when the user is not authenticated by checking the cookie in the request. Any request with ‘noache’ selector will be interpreted by the below rule and check if the request contains ‘cookie-id'(indicates the user is authenticated), if not exist rewrite condition will be triggered and change the selector to ‘anon’. 

RewriteCond %{HTTP_COOKIE} !cookie-id
RewriteRule /(.*).nocache.html /$1.anon.html

This will ensure the first request will reached out publisher and then on it serves it from dispatcher cache for subsequent request.

Dispatcher Deny Rule

/0001
{
  /glob "*.nocache.html*"
  /type "deny"
}

Akamai/Varnish : 

You need to configure similar kind of rule in both the servers in specified syntax. 

#Challenge 2 – Embedded components leads to 404.

SDI module doesn’t generate proper esi url if you embed the component with in a component or template. The reason being, embedded components won’t generate JCR node by default unless you drag and drop or open the dialog and save. To fix the issue, we need to create the component JCR node to generate proper url’s. 

Fix : Write a simple groovy script to find the references and create the component nodes programatically or Author has to configure the component manually by opening the dialog and save. For few pages, it make sense to author it manually but when you touch/enable exiting component that’s been referenced on 1000+ pages, it makes sense to write a groovy script and bulk update. You need to train the authors on how to run the script again in future. 

Example  : 

Embedded Component :

<div data-sly-resource="${'relatedproducts' @resourceType='website/components/content/related_products'}"></div>

Generated URL : This url leads to 404 error.

<esi:include src="/us/en-us/_jcr_content/related_products.nocache.html/website/components/content/related_products.html" />

Expected  URL :

<esi:include src="/us/en-us/_jcr_content/related_products.nocache.html" />

 

#Challenge 3 – Browser cache headers for *.html leads to in-consistent user behaviour.

Typically in any implementation we do cache *.html files in the client browser by specifying “Cache-Control : max-age=600”. As you know, assembling ESI content is controlled by Akamai or Varnish, each request has to reach out to one of these servers to render personalised content based on the user info. Because of these cache headers or ETag the content(pages) will be cached on the client browser until it expires. This leads to in-consistent user behaviour when they are navigating from one page to another.

Fix : Reset the cache control header to ‘no-cache’ and suppress the ETag from the response headers.

I’m sure you might ask this leads to increase in traffic to my cache server (Akamai or Varnish). In my case, i easily traded to Personalised Content vs More Traffic since Akamai is capable of handling the load very easily. That’s why i’m saying scalable infrastructure is also very important factor to go with this direction. 

 

Conclusion 

As i mentioned, opting for ESI is completely based on your niche use cases and the infrastructure considerations. I hope this information will be useful and for any questions please do comment below, i’m happy to have a conversation. 

Leave a Reply

avatar
  Subscribe  
Notify of