Offline Experiences with Adobe Experience Manager

Websites that work offline. How does that work? One of the newer features of the web platform is the Service Worker API. Service workers sit between a web page and the network and are capable of intercepting all the requests coming into or out from the web page. In conjunction with the Cache API you can provide offline experiences to your website. We would be aghast if we received a “system error” when opening a mobile or desktop app. However we don’t think twice when our browser gives us the offline page when trying to connect to a website. It’s time for that to change!

Progressive Enhancement

Now you might be thinking that this sounds great – but I need to support older browsers. While these API’s are already available for 85% of users, it is true that they are not available for everyone. However why degrade the experience of the majority of your users that are on newer browsers because some of your users are not up to date? Our mentality needs to change away from “the features I can use are based upon the most outdated browser that I have to support” to “I will provide the best possible experience that each user can have”. This is called progressive enhancement. If a feature is available for the user, then you make the most of it and give them the best possible experience that they can have given the platform that they are on.

Progressively enhance the experience of the user based upon what is available on their platform.

We will apply these concepts to Adobe Experience Manager, trying out one way of installing a service worker with AEM and implementing a few offline features.

Create and Install the Project

This is an easy one. Just run the following command in the directory where you want to create the project. If you call the project “aempwaexample” it will help follow along as my links throughout the post will expect that project name. Assuming that you have an instance of AEM 6.4 running, go ahead and run the install command from the projects root directory.

mvn archetype:generate -DarchetypeGroupId=com.adobe.granite.archetypes -DarchetypeArtifactId=aem-project-archetype -DarchetypeVersion=16

# It will ask you for a "groupId" and "appsFolderName".
# For these values enter "aempwaexample".
# The rest of the options should provide defaults that you can use.

# After the interactive archetype generate command completes go to the projects root directory.
# Make sure that AEM is running. I used AEM 6.4 but it should work with other versions.

mvn clean install -PautoInstallPackage

Now like most AEM projects there is a lot of boilerplate code. Whenever I mention code I will provide a project path to the file that is being modified. If you have followed these steps than this should be straight forward to find in your code base. If you are simply reading the post then you can use my aem-pwa-example github repository as a reference if it is helpful.

Create New Content

Now use the content template to create the following page structure from the sites admin. This will give us some example pages to test our offline functionality.

• /content/aempwaexample/en
  • Products
    • Product A
      • Sub Product X
    • Product B
    • Product C
  • News
    • Article A
    • Article B
    • Article C

Clean Up Existing Content

The  template that this starting content uses is going to need some work. Open up the content template in structure mode and delete everything except the navigation component and parsys. Then add a title component so that it will be clear as to what page we are on and an image component so that we can see an example of caching important assets. I also modified the layout putting the components into a two column view. After this is done your template should look something like the below example. It is okay if your’s is slightly different.

You should have some sample content at the english home page. This page has a bunch of extra content in it so go ahead and delete everything leaving a mostly blank page. Add one text component in the right column with some example text. We will use this component to test our service worker cache. At the end of this step your home page should look something like this:

Create the Service Worker

Like I have already said service workers act as the intermediary between the web page and the network. They are scoped by the browser based upon a domain and path. Additionally they have lifecycles going from “installing” to “active”. Once they are active, they can intercept the requests that the web page makes and respond to the page in any way they want. While they can be used to do many things, in this post we will use it to implement a “Network then Cache” strategy. If content is available from the network then we will update the cache and return the response to the web page. If the network is not available then we will look for a response in the cache.

Register the Service Worker

Add a service worker registration to the end of the following file:

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/structure/page/customfooterlibs.html

if ('serviceWorker' in navigator) {
  navigator.serviceWorker.register('/content/aempwaexample/sw.js')
  .then(function(reg) {
    console.debug('Registration succeeded. Scope is ' + reg.scope);
  }).catch(function(error) {
    console.debug('Registration failed with ' + error);
  });
}

Notice on line three that we are looking for a service worker JavaScript file at the ‘/content/aempwaexample/sw.js’ location. In order for the browser to except the service worker it must live on the same path as the resource that it is acting on behalf of. This means that we cannot place the service worker under the /etc path for example. Additionally the maximum scope of the service worker is limited to the path that it is on. This is why we place it at the /content/aempwaexample path and not under a lower level page. If you wanted separate service workers for separate countries then they could be placed under the country site.

This script tag and the JavaScript inside it will run in the page context of the browser like any normal JavaScript. However as we will see the JavaScript code that it registers runs in the “service worker” context. This means that it is separated from the normal execution of the web page.

Create the service worker

Now that we are referring to a service worker we should probably create it. To start we will create a cache that will get initialized with the home page. When the service worker is installed we will initialize the cache. Finally, when the web page makes requests we will intercept the request. However for the moment we won’t actually do anything with the request other than just send it off to the network.

• ui.content/src/main/content/jcr_root/content/aempwaexample/sw.js

// This cache will contain our site content. You could have other caches for
// other purposes such as application assets and images.
var contentCacheName = 'aempwaexample-content';

// Put all of your caches into an array so we can add other types of caches later.
var cacheNames = [contentCacheName];

// This will be the pages that get cached when the service worker is installed.
var filesToCache = [
  '/content/aempwaexample/en.html'
];

// During installation precache the files that we know we want cached.
self.addEventListener('install', function(e) {
  console.debug('[ServiceWorker] Install');
  e.waitUntil(
    caches.open(contentCacheName).then(function(cache) {
      console.debug('[ServiceWorker] Caching');
      return cache.addAll(filesToCache);
    })
  );
});

// When the page fetches new content intercept the request.
self.addEventListener('fetch', function(e) {
  console.debug('[ServiceWorker] Fetch', e.request.url);

  // This doesn't change any behavior, but we will update this soon.
  e.respondWith(fetch(e.request));
});

Update the filter.xml

Changes to the service worker will not make it into the repository on repeated installs until we tell the filter.xml file to replace the service worker. Update the following file as shown.

• ui.content/src/main/content/META-INF/vault/filter.xml

<?xml version="1.0" encoding="UTF-8"?>
<workspaceFilter version="1.0">
    <filter root="/conf/aempwaexample" mode="merge"/>
    <filter root="/content/aempwaexample/sw.js" mode="replace"/>
    <filter root="/content/aempwaexample" mode="merge">
      <exclude pattern=".*\sw.js" />
    </filter>
    <filter root="/content/dam/aempwaexample" mode="merge"/>
</workspaceFilter>

Now run the “mvn clean install -PautoInstallPackage” command in order to install these updates. When it is done inspect the DevTools console in Chrome of the home page in preview mode. Go to the “Service Workers” section of the “Application” tab. After refreshing the page you should see an installed service worker.

Add the content cache

So far we are only initializing our cache with the home page. Lets update the service worker fetch event listener as shown below. This will implement the “Network then Cache” strategy that we mentioned earlier. It will send the request off to the network and cache the response if there is one. If the fetch fails it will look for a cached response.

• ui.content/src/main/content/jcr_root/content/aempwaexample/sw.js

...
// When the page fetches new content intercept the request.
self.addEventListener('fetch', function(e) {
  console.debug('[ServiceWorker] Fetch', e.request.url);
  e.respondWith(
    caches.open(contentCacheName).then(function(cache) {
      return fetch(e.request)
      .then(function(response){
        // If the fetch to the network is successful cache the response and return
        // it to the page.
        console.debug('[ServiceWorker] Updating cache', e.request.url);
        cache.put(e.request.url, response.clone());
        return response;
      })
      .catch(function() {
        // If the fetch to the network fails either give the user the cached cached
        // response if it is available or fallback to the offline page.
        console.debug('[ServiceWorker] Using response from cache', e.request.url);
        return cache.match(e.request);
      });
    })
  );
});

Now install this code and then open up an example page and begin using the navigation component to navigate to some of the pages. Then open up DevTools and go to the application tab. Select the “offline” option in the “Service Workers” section to force this browser tab to disconnect from the network. Now navigate to one the pages that you have previously visited. You can now access your website even while offline! However the user is going to have difficulty knowing what content is available while offline. To fix this lets create a component to help them out.

Create the Offline Component

Now that the webpages that the user has visited are available offline we should add some functionality in this scenario to assist the user. What we will do here is use the Navigator Web API in order to detect when we are offline and show links to the pages that the user has recently visited.

To do this create a component at the below location. You can check the same component on github for reference however I will highlight all of the important code below.

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline

This component will need two pieces of html. One will be what is shown when the user is online and the other is what will be shown when the user is offline. You might want to simply show nothing if they are online however for clarity I added a simple message in this scenario. The offline section will need an empty list where we can dynamically add links based upon the content cache.

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline/offline.html

<div class="offline-container" style="display: none;">
  <p>We noticed that you are offline. However you still have access to your recently viewed pages:</p>
  <ul class="saved-links"></ul>
  <p>This is just one example use case of how you can utilize a service worker.</p>
</div>

<div class="online-container">
  <p>You are online</p>
</div>

Now we need to add some JavaScript to this component that will hide or show the right section of html based upon whether or not the user is online. In addition this JavaScript will create links within the list of all of the html responses that are saved in the cache.

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline/clientlib/js/offline.js

window.addEventListener('offline', showOfflineComponent);
window.addEventListener('online', hideOfflineComponent);

caches.open('aempwaexample-content').then(cache => {
  cache.keys().then(keys => {
    document.querySelector('.offline-container .saved-links').innerHTML = '';
    keys.forEach(key => {
      cache.match(key).then(response => {
        response.text().then(text => {
          var matches = text.match(/<h1.+?>(.+?)<\/h1>/);
          if (matches && matches.length > 1 && response.ok && response.headers.get('Content-Type').indexOf('text/html') === 0) {
            var liElment = document.createElement('li');
            var aElement = document.createElement('a');
            aElement.href = response.url;
            aElement.innerText = matches[1];
            liElment.appendChild(aElement);
            document.querySelector('.offline-container .saved-links').appendChild(liElment);
            showOfflineComponent();
          }
        });
      });
    })
  })
});

function showOfflineComponent() {
  // We only want to display the offline container if we found at least one saved link.
  if (! navigator.onLine && document.querySelectorAll('.offline-container .saved-links a').length > 0) {
    document.querySelector('.offline-container').style.display = 'block';
    document.querySelector('.online-container').style.display = 'none';
  }
}

function hideOfflineComponent() {
  document.querySelector('.offline-container').style.display = 'none';
  document.querySelector('.online-container').style.display = 'block';
}

In addition we will need to create a content xml for our client lib and a js.txt file as shown below.

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline/clientlib/.content.xml

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jcr:primaryType="cq:ClientLibraryFolder"
    categories="[aempwaexample.offline]"/>

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline/clientlib/js.txt

js/offline.js

We will need to add our component JavaScript to the client lib. Do this by “aempwaexample.offline” to the “embed” list in the following file:

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/clientlibs/clientlib-base/.content.xml

Lastly add in a content xml and a dialog for the component as shown below.

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline/_cq_dialog/.content.xml

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
    jcr:primaryType="nt:unstructured"
    jcr:title="Properties"
    sling:resourceType="cq/gui/components/authoring/dialog">
    <content
        jcr:primaryType="nt:unstructured"
        sling:resourceType="granite/ui/components/coral/foundation/fixedcolumns">
        <items jcr:primaryType="nt:unstructured">
            <column
                jcr:primaryType="nt:unstructured"
                sling:resourceType="granite/ui/components/coral/foundation/container">
                <items jcr:primaryType="nt:unstructured">
                    <text
                        jcr:primaryType="nt:unstructured"
                        sling:resourceType="granite/ui/components/coral/foundation/form/textfield"
                        fieldLabel="Text"
                        name="./text"/>
                </items>
            </column>
        </items>
    </content>
</jcr:root>

• ui.apps/src/main/content/jcr_root/apps/aempwaexample/components/content/offline/.content.xml

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
    jcr:primaryType="cq:Component"
    jcr:title="Offline"
    componentGroup="aempwaexample"/>

After installing this updated code go ahead and open the content page template and add the offline component. We can now open up one of our pages and we will see the “Your are online” message. Now use the navigation component to navigate around to some of the pages but do not go to all of them. Finally open up DevTools and go to the application tab and select the offline checkbox. This will force this Chrome tab to disconnect from the network. Notice that our offline component appears with a message telling us that some of our pages are still available even while offline.

Create an Offline Page

We now have an offline component that gives the user access to their previously visited content. But what if they visit a page that is not in the cache? In this case we will want to show the user an “offline” page. This is what the user will see when they do not have a network connection and they attempt to access a page that they do not have stored in the cache. This is something of a mix between the “offline” screen that a native mobile app might have and the 404 page more familiar to web developers. However this should not be treated as an “error” or used simply to tell the user that they are offline. Rather this should be a targeted experience in and of itself.

What functionality can we give the user even while they are offline?

A common scenario on messaging apps is to still allow the user to send messages even while offline. From the users perspective nothing has changed. When network access is regained then the messages are sent without the user needing to even know that there was a delay. This is just one example of offline functionality on the web.

Offline experiences offer an entirely new realm of what is possible on the web platform.

In our example we are going to provide a user with a message that explains that even while offline some content is still available. Then we will show them the offline component that we have previously created so that they have quick access to all the content that is available offline. To do this create a “/content/aempwaexample/en/offline.html” page from the sites admin.

Now we will need to update the service worker so that if a page is requested, and we do not have network access, and it is not saved in the content cache, then we give the user the offline page. Update the service worker fetch event listener as shown below.

• ui.content/src/main/content/jcr_root/content/aempwaexample/sw.js

// This cache will contain our site content. You could have other caches for
// other purposes such as application assets and images.
var contentCacheName = 'aempwaexample-content';

// Put all of your caches into an array so we can add other types of caches later.
var cacheNames = [contentCacheName];

// We will give the web page this offline page if we are off the network and
// the page it asks for is not available in the cache.
var offlinePage = '/content/aempwaexample/en/offline.html';

// This will be the pages that get cached when the service worker is installed.
var filesToCache = [
  '/content/aempwaexample/en.html',
  offlinePage
];

// During installation precache the files that we know we want cached.
self.addEventListener('install', function(e) {
  self.skipWaiting();
  console.debug('[ServiceWorker] Install');
  e.waitUntil(
    caches.open(contentCacheName).then(function(cache) {
      console.debug('[ServiceWorker] Caching');
      return cache.addAll(filesToCache);
    })
  );
});

// When the page fetches new content intercept the request.
self.addEventListener('fetch', function(e) {
  console.debug('[ServiceWorker] Fetch', e.request.url);
  e.respondWith(
    caches.open(contentCacheName).then(function(cache) {
      return fetch(e.request)
      .then(function(response){
        // If the fetch to the network is successful cache the response and return
        // it to the page.
        console.debug('[ServiceWorker] Updating cache', e.request.url);
        cache.put(e.request.url, response.clone());
        return response;
      })
      .catch(function() {
        // If the fetch to the network fails either give the user the cached cached
        // response if it is available or fallback to the offline page.
        console.debug('[ServiceWorker] Using response from cache', e.request.url);
        return cache.match(e.request).then(function(cachedResponse) {
          if (cachedResponse) {
            return cachedResponse;
          } else {
            console.debug('[ServiceWorker] Using offline page from cache');
            return cache.match(offlinePage);
          }
        });
      });
    })
  );
});

After installing the project turn off the offline mode in DevTools in order to get the updated service worker. Then turn the offline mode back on and navigate around the site. You will receive the offline page for any page that is not cached. The content and features on this page can be updated to target the offline user experience.

Conclusion

In this post we have only seen the tip of the iceberg of what offline experiences are possible on the modern web platform. We have seen how this can be applied to content management platforms such as Adobe Experience Manager. In a future post we will find out how to take the next step and make an AEM website into a true progressive web app that can be installed on both desktop and mobile devices.