Building machine images with Packer

Alan Christie 2018-07-10

Automation IaC

You’ve probably created a machine image at some point. A base image for AWS that builds upon someone’s work by adding a particular version of Java or Python or a new utility. Did you create the image on AWS using an EC2 instance, login, run some yum or apt-get and then save it? Great, and if someone wants the source code for that image or you want to build a similar image on a different region or provider? Well, Packer is an IaC tool for automating the construction of machine images.

Packer, from HashiCorp, automates the building of machine images. It has more than 20 builders for platforms like AWS EC2, DigitalOcean and Google Cloud amongst others.

Images are defined in Templates (JSON files) and contain sections relating to variables, builders and provisioners.

The builder section varies depending on the target platform and it contains things like your API keys and and platform dependent settings. The work for constructing the image is provided in the provisioner.

It’s all part of the Infrastructure as Code (IaC) methodology - providing clean and consistent working environments, and streamlines DevOps.

Their site has plenty of documentation, but here, just as a taster, is a template that (given suitable keys like an AWS API access code and secret key and a source AMI) will build an AMI from the AWS Linux image in Ireland, install Docker, Java 8 and Nextflow: -

Once you have your template you build it with the command packer build <template-file>.

There can be more than one builder in a file. So, on AWS, if I wanted to build images for 8 different regions I’d have one variables section, one provisioners section and eight buidlers sections, one for each region. When Packer’s run on this sort of template it builds all your images independently and in parallel, registering each image for you and giving you an image reference you can use when you spin up your new machine.

But I use YAML…

Sadly JSON, in my opinion, is not really for humans. I prefer something a little more friendly so I author my images using YAML. That way I can also add inline documentation (vital) and then run a simple Python module to do the conversion before running packer.

The conversation is simply sys.stdout.write(json.dumps(yaml.load(sys.stdin), indent=2)).

And the following is the above JSON template as YAML, but with beauty of a file format that permits embedding documentation.

Now you have an image as code. That’s extremely valuable in itself.