Skip to content →

Month: May 2016

Visualising your AWS infrastructure

Building out an AWS infrastructure can get pretty complicated as there are many different elements and dependencies that need to be considered. Visualising the relationships is a great way to understand all the dependencies involved, however the existing tools may create results that were not exactly expected. The visualisations that I have been using are based on my Cloud Formation scripts. These are static JSON files that AWS uses to build out the architecture. The first tool that I used to help visualise the script was Cloud Formation Viz. Its a simple command line tool that can draw out the different components and relationships.
The below diagram is of a single EC2 instance.

This was a pretty simple Cloud Formation Script, but already you can see how complicated its starting to get!!!!
The second image is of a multi EC2 instance VPC.

Now we really start to see how complex architecture is!!! In reality though you need something like this to keep track of what you are building and how it all hangs together!!!

AWS CloudFormation has a builtin designer which visualises your cloud formation scripts, however this requires you to be editing your templates in the online designer. The benefits are you get template validation, however its difficult to tie the designer into a proper devops style workflow.

The below is the AWS visualisation of the single EC2 instance VPC taken as a screenshot.

The following is of the more complex multi instance VPC

What both of these solutions has in common is that they represent the architecture as a graph diagram rather than a what we would expect of a traditional architecture diagram. Typically people expect some thing like the below. (I used Lucid Chart)

The above is much more easily recognised as an Architecture diagram, but the reality is that its an abstraction and pov of an architecture. Its a much more digestible format for communicating your architecture.

Summary

Tooling is not great in this area and depending on your audience you will get very different results from handcrafting to auto-generating. At the moment for me I will be using both methods for tracking change and communicating my designs.

Leave a Comment

Setting up Adobe AEM on Amazon Webservices AWS

One of the AWS concepts that I love is the idea of a Stack, a grouping of related resources manage as a single unit. I guess one of the early incarnations of the stack was the idea of a virtual appliance that VMWare used to market. The conceptual difference between the virtual appliance and the AWS stack is that the Stack includes everything above the hypervisor including networks, security groups, network acls, packaged into a single Virtual Private Cloud.

From an architectural perspective the Stack enables us to move away from documenting best practises and target architectures in documents and Visio diagrams and instead encode them in machine readable artefacts that can be used as templates for distribution and implementation.

In AWS the CloudFormation service takes a JSON template file as its input and constructs the AWS environment from the meta data in the script.  Azure has an identical concept called Resource Manager. Hand coding these JSON templates is a little challenging as tooling support for large JSON documents is very limited, its ideal as a machine readable data exchange format but not great as a human readable data construct Sad smile. The design, deploy, debug/test cycle is slow and the environment provisioning has many dependencies and options that are not clear when hand coding the estate.  Both AWS & Azure provide some basic integration in to IDEs.   I have tried the AWS in Eclipse on  a Mac which was better than nothing but not exactly a nice experience. Microsoft’s Visual Studio has much better integration with their Resource Manager however this only applies to Windows.   The alternative method for building out the stack is to use the AWS GUI to build out your resources and then reverse engine the JSON templates. I have used this technique multiple times to get my head around how things work, but you will end up having to clean the final scripts up and the templates contain a lot of autogenerated metadata which is not that helpful for the human eye.

AWS have a number of sample templates that can be used as a basis for building out your own stacks. I am currently working with Adobe AEM so decided to build out a simple single instance EC2 with Adobe AEM installed on it. Its easy to provision and is a self contained instance ideal for development, prototypes etc.

You can use the link below to launch the stack into your own AWS account or you can visualise the stack using the AWS visualisation tools

launch stack button  View the Stack

Next step was to build out a more realistic environment that could be used for testing. This stack deploys AEM across 3 instances including an author, publisher and dispatcher instance.

launch stack button  View the Stack

When you launch either of the above stacks you will be asked to pass in a number of parameters. This include:

AEMDownloadUrl: Each of the script requires that you have a copy of the AEM Quick Start that can be downloaded onto the instance. You can pass the URL into the cloud formation script. I have my quick start uploaded to S3 and it gets pulled down when the EC2 instance initialises.

InstanceType: It is recommended to run AEM on a pretty significant instance to ensure that it is performant, however as costs apply I have left the instance type for you to select.

Keyname: This is required so that you can SSH into your EC2 instances

SSHLocation: This is a firewall rule that enables you to lock down your SSH access to only an IP that you specify

There is still a lot of work to do on these scripts to turn them into production read stacks, however for anybody starting on the journey I hope that they are help full.

Further work and consideration needs to be taken to include:

  1. setting up private subnets & VPN access for author & publisher nodes.
  2. setting up IAM users
  3. CDN configuration
  4. multi AZ set up for HA and DR scenarios.
  5. Shared content stores on S3, MongoDB & TarMK
  6. Auditing, Logging etc
  7. Adobe AEM Configuration for content replication

The scripts are also available on GitHub so please fork the templates, make changes and let me know how you get on.

Leave a Comment