What is ESI?
Edge Side Includes or ESI is a small markup language for edge-level dynamic web content assembly. The purpose of ESI is to tackle the problem of web infrastructure scaling. It is an application of edge computing.
Static Content vs. Dynamic Content?
Let’s assume we have a dynamic website, with most pages being a mixture of static and dynamic content. The portion of the page should be uncacheable, and the rest of the page should serve from the cache. How do you ensure the cache won’t be impacted by having dynamic content on the page? A few possible options are –
Option 1: Load dynamic content Asynchronously (AJAX)
Option 2: Load dynamic content using ESI.
I would say both options are the best, but you should choose based on the type of website and infra considerations. Let’s talk about the scenarios and when to determine what.
When to choose AJAX?
- When SEO is not a constraint.
- Best for SPA-type applications.
- UGC heavy website.
- Page response time shouldn’t be a constraint. Usually slow.
- Hard to perform A/B testing when you inject personalized content asynchronously. It would help if you chose when this option is less relevant.
When to choose ESI?
- Multi Page Application.
- SEO guidelines have to be met.
- Dynamic header/footers that are updated frequently.
- To avoid the invalidation of the entire site when the header/footer keeps changing.
- Scalable and dynamic website without writing a new code.
- To minimize the number of requests per page.
- Highly scalable CDN server that supports ESI.
There could be more, but I hope the above pointers will help you decide which direction to go.
NOTE: Suppose you choose esi as an option to render dynamic content, but in a few cases, you can’t use esi because of the nature of the component or fragment. Example: Advanced Search( Personalised ) – You can’t expect the page to be reloaded whenever a user tries to search or apply any filters. In this case, AJAX is inevitable and gives you a better user experience. A hybrid option is always a possibility.
How to set up SDI in AEM?
SDI Set-up
https://experienceleague.adobe.com/docs/experience-manager-learn/foundation/development/set-up-sling-dynamic-include.html?lang=en
The above like has complete details of how to configure publishers & Apache.
My setup looks like the above, but it may not be the same for every implementation. Since all the caching layers like Dispatcher, Varnish & Akamai can include dynamic content( ESI or SSI ), it can be in any combination. Having varnish onboarded has its advantages since it also implemented(partially) ESI 1.0 specifications; we can leverage it for lower environments for non-akamai servers, and it acts like your ESI processing server to perform the testing before it goes to production.
Realtime Problems?
So far, we have discussed when to choose esi vs. ajax and how it works. Now, let’s see what real problems we get into during implementation.
#Challenge 1 – Your dispatcher cache ratio might be impacted if you don’t differentiate b/w authenticated and non-authenticated requests.
In most cases, the same components have to behave differently when you are authenticated or non-authenticated. The same part has to render different content based on the context. Since all the esi calls have the selector ‘.nocache,’ these requests will be bypassed and delegated to publishers directly. How do you ensure non-authenticated calls are cached in the dispatcher?
Let’s take an Example :
Component Name: Recommended Products
ESI URL : <esi:include src=”/content/website/us/en-us/_jcr_content/recommended_products.nocache.html” />
As I said, the above call is always delegated to the publisher irrespective of whether the user is authenticated or not because of the ‘.noache’ selector.
Fix :
Implement a simple rule in Apache to rewrite the URL coming with ‘nocache.html’ –> ‘anon.html’ when the user is not authenticated by checking the cookie in the request. Any request with a ‘noache’ selector will be interpreted by the below rule and check if the request contains ‘cookie-id'(indicates the user is authenticated); if it does not exist rewrite condition will be triggered and change the selector to ‘anon.’
RewriteCond %{HTTP_COOKIE} !cookie-id
RewriteRule /(.*).nocache.html /$1.anon.html
This will ensure the first request will reach the publisher and then serve it from the dispatcher cache for subsequent requests.
Dispatcher Deny Rule :
/0001
{
/glob "*.nocache.html*"
/type "deny"
}
Akamai/Varnish :
You need to configure a similar rule in both servers in specified syntax.
#Challenge 2 – Embedded components leads to 404.
SDI module doesn’t generate proper esi URL if you embed the component in a component or template. The reason is, embedded components won’t cause a JCR node by default unless you drag and drop or open the dialog and save. We need to create the component JCR node to generate proper URLs to fix the issue.
Fix: Write a simple groovy script to find the references and create the component nodes programmatically, or Author has to configure the component manually by opening the dialog and saving. For a few pages, it makes sense to author it manually, but when you touch/enable the exiting component referenced on 1000+ pages, it makes sense to write a groovy script and bulk update. You need to train the authors on how to rerun the script in the future.
Example :
Embedded Component :
<div data-sly-resource="${'relatedproducts' @resourceType='website/components/content/related_products'}"></div>
Generated URL: This URL leads to a 404 error.
<esi:include src="/us/en-us/_jcr_content/related_products.nocache.html/website/components/content/related_products.html" />
Expected URL :
<esi:include src="/us/en-us/_jcr_content/related_products.nocache.html" />
#Challenge 3 – Browser cache headers for *.html lead to inconsistent user behavior.
Typically, in any implementation, we cache *.html files in the client browser by specifying “Cache-Control: max-age=600”. As you know, assembling ESI content is controlled by Akamai or Varnish; each request has to reach out to one of these servers to render personalized content based on the user’s info. Because of these cache headers or ETag, the content(pages) will be cached on the client browser until it expires. This leads to inconsistent user behavior when navigating from one page to another.
Fix: Reset the cache control header to ‘no-cache’ and suppress the ETag from the response headers.
I’m sure you might ask if this leads to increased traffic to my cache server (Akamai or Varnish). In my case, I quickly traded to Personalised Content vs. More Traffic since Akamai is capable of handling the load very quickly. That’s why scalable infrastructure is also an essential factor in this direction.
#Challenge 4 – Caching 302 calls leads to an in-consistence experience.
In a few scenarios, 302 redirection is a possible option for you to navigate random secure pages once after successful login. This leads to an inconsistent experience if you enable 302 caches in Akamai or Varnish layers. A few possible options are to deny the 302 caches on pre-defined URLs or deny for all(This option has some performance impacts, please choose carefully).
Conclusion
As I mentioned, ESI is wholly based on your niche use cases and infrastructure considerations. I hope this information will be helpful and for any questions, please do comment below; I’m happy to have a conversation.
Leave a Reply