The voluntary group of IT experts (under Česko.Digital community) helped to create a crisis information website for Czech Ministry of Health about the coronavirus outbreak in the Czech Republic koronavirus.mzcr.cz in a very short time with a great ability to manage the huge turnout of visitors. Vaclav Jirovský, Cloud Solution Architect, MSD IT is the author and driver of the technical solution described in the article below.
PR Crisis Microsite Deployed in Cloud in Few Hours
At Česko.Digital community, we helped to create a crisis information website for Czech Ministry of Health about the coronavirus outbreak in the Czech Republic. The website has since become one of the main information channels for all Czech citizens. This article describes how we modified the website with the Ministry’s IT dept. to handle expected load.
During PR crises of corporations (e.g. faulty service or product), main company websites often become overloaded. This situation typically leads to an increased amount of inbound telephone calls and overloading the call centers. Changing the main company website usually takes some time (they are built on enterprise CMS) and scaling the customer telephone lines is not so easy (and really expensive). All of the above are reasons why most of companies choose to create a crisis-dedicated microsite or move communication to external platform (e.g. Medium).
A microsite solution (looks more professional) needs a decision where to run it. I don’t see any better scenario for a website to be deployed into the cloud – we need to have website scale-able and quickly deployed. I am sure there are even better tools you could use to build the crisis website, but for our solution we had to make a compromise – sometimes because of usability but most importantly because of the time constraint – what is easiest to set up, with what we have the most experience to be able to launch MVP (Minimum Viable Product) as soon as possible.
We wanted to have a solution which would be stable and we didn’t want to make the ministry’s PR department go through a Markdown language training, and so we decided to use WordPress. This CMS brings a friendly user interface but it’s a bigger challenge to configure it to be secure and ready for high performance.
The solution described below is designed to be deployed in Microsoft Azure. It can be also easily ported to AWS (CloudFront, S3, etc.). We selected Azure because most enterprise companies have Azure implemented in their organizations (usually because Azure Active Directory or Office 365).
Solution is built on idea, standard users don’t interact with CMS, only but with static HTML files generated by CMS. Thanks to this approach, we can easily achieve the desired performance and security.
WordPress (WP) is installed on a virtual server, which is isolated and accessible only via VPN (or IP filter at least). WP installation is configured for a different URL, than the destination URL of the website. Without this WP installation doesn’t work well. When the author generates static files (output is located on the server’s filesystem), our custom built script is triggered – this script syncs these generated files to Azure Blob Storage via Shared Access Signature (this could be simplified by mounting Blob storage as directory on filesystem).
Azure CDN is placed In front of Blob Storage. This service provides caching of content to the closest physical nodes (and in fact solves all our performance needs). When the author is satisfied with the exported version of articles, the author performs CDN cache invalidation and all readers will see the updated version in a few minutes. Thanks to saving SSL private key in Key Vault, the Azure CDN can behave as our website.
The diagram also shows a „registered app in AAD“. This application is granted to perform the CDN cache invalidation and provides access for our custom WP plugin – this plugin enables to trigger the invalidation from WP user interface via the Azure REST API.
Limitation of this solution is a lack of interactive content – e.g. search, poll etc. However, these features are usually not needed for a crisis website.
Solution Details and Usage
We used plugin WP2Static for static files export. The plugin works as a crawler. It crawls all pages and articles, generates static HTML files from them and downloads all other necessary files (e.g. images, CSS files, etc.). The plugin replaces URL of WP installation with production website URL.
Publishing the new content version was initially performed manually by running a bash script (in early stages of MVP).Few hours later we developed with Vladimir Smitka custom WP plugin (for cache invalidation) and script, which detects new export and uploads it on Blob Storage. This combination enables the authors to make a deployment by themselves from WP directly.
I also recommend setting up HTTPS-only redirection on Azure CDN (+redirection from a www. subdomain) by custom rules in the Rules engine feature.
The website was pretty fast even with 11K concurrent active users. I believe Azure CDN capacity could handle a bigger amount but we haven’t needed it yet.
Despite this fact, the website was stable, according to Hlídač webů service (a monitoring service of the government websites). The graph below shows the moments when content was deployed. The website slowed down at those points.
Graph showing results on monitoring
Checklist Before You Start
General Tips for a Crisis Website
To get this solution ready also for the possibility of an Azure region outage (where your website is running), I also describe this solution in HA version. However, for the basic crisis websites the solution described above is sufficient.
The entire solution is duplicated also to another failover Azure region. An Azure Traffic Manager is placed in front of the Azure CDN. This service performs a health check and in case of failure of the primary region it automatically switches to the failover region. The Azure Site Recovery service is used for replication of VM with WP administration.
The script and the plugin requires a small modification to perform all changes on both Azure Storage and CDN.
Source Code and Settings
You can find the source code of WP plugin, the script and the necessary settings on my GitHub.
Thanks to Adam J. Sporka for text correction of the article.
Vaclav Jirovsky, Cloud Solution Architect, MSD IT