Daniel Jaouen's Tech Blog

Wisdom garnered from many years of tech experience

How I Fully Automated OS X Provisioning With Ansible

One cool thing about Ansible is that you can use it to provision your OS X boxes using the homebrew module (which I co-wrote with Andrew Dunham). Today, I’m going to share with you how I fully automate the provisioning of new OS X boxes. (Note: this blog entry assumes the use of Ansible 1.6, which is currently still in development)

Ansible Setup

I have a private repository with a playbook that looks like this:

site.yml (site.yml) download
1
2
3
---
- include: base_osx.yml
- include: dotfiles.yml

Those two playbooks in turn look like this:

base_osx.yml (base_osx.yml) download
1
2
3
4
5
6
# file: base_osx.yml

---
- hosts: base_osx
  roles:
    - base_osx
dotfiles.yml (dotfiles.yml) download
1
2
3
4
5
6
7
8
# file: dotfiles.yml

---
- hosts: dotfiles
  sudo: yes
  sudo_user: "{{ dan.user.name }}"
  roles:
    - dotfiles

And the inventory file:

inventories/osx (osx) download
1
2
3
4
5
6
7
8
9
10
11
# file: inventory

[base_osx]
127.0.0.1

[dotfiles]
127.0.0.1

[osx:children]
base_osx
dotfiles

Note the added osx group that I use to define certain variables for use with my roles:

group_vars/osx.yml (osx.yml) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# file: group_vars/osx.yml

---
system: osx
os: osx

executables:
  brew: /usr/local/bin/brew
  vim: /Applications/MacVim.app/Contents/MacOS/Vim
  zsh: /usr/local/bin/zsh

directories:
  etc: /usr/local/etc
  var: /usr/local/var

dan:
  user:
    name: dan
    group: staff
    shell: "{{ executables.zsh }}"

  template_home: home/dan
  home: /Users/dan
  system: osx
  ruby:
    version: "{{ ruby.version }}"
  rbenv:
    global: "{{ ruby.version }}"

I also have some global variables defined in my all.yml file:

group_vars/all.yml (all.yml) download
1
2
3
4
# file: group_vars/all.yml
---
ruby:
  version: 2.0.0-p451

Roles

I use two separate roles here: base_osx and dotfiles. Note that these roles have been created for my own unique needs, so if anything, they should only be used as a base to point you in the right direction.

base_osx

base_osx adds some Homebrew taps and then installs a few Homebrew and Python packages. In particular, see: tasks/common.yml and defaults/main.yml

dotfiles

dotfiles clones down a few git and hg repositories, installs Ruby with rbenv, then rake installs my dotfiles (see https://github.com/danieljaouen/dotfiles) using a Rakefile. In particular, see tasks/dan.yml and defaults/main.yml.

Bringing it all together

I use the following shell script to both prepare base boxes for Ansible provisioning as well as to do the actual provisioning:

bootstrap.sh (bootstrap.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
#!/bin/bash

set -e

SRC_DIRECTORY="$HOME/src"
ANSIBLE_DIRECTORY="$SRC_DIRECTORY/ansible"
ANSIBLE_CONFIGURATION_DIRECTORY="$HOME/.ansible.d"

# Download and install Command Line Tools
if [[ ! -x /usr/bin/gcc ]]; then
    echo "Info   | Install   | xcode"
    xcode-select --install
fi

# Download and install Homebrew
if [[ ! -x /usr/local/bin/brew ]]; then
    echo "Info   | Install   | homebrew"
    ruby -e "$(curl -fsSL https://raw.github.com/Homebrew/homebrew/go/install)"
fi

# Modify the PATH
export PATH=/usr/local/bin:$PATH

# Download and install zsh
if [[ ! -x /usr/local/bin/zsh ]]; then
    echo "Info   | Install   | zsh"
    brew install zsh
fi

# Download and install git
if [[ ! -x /usr/local/bin/git ]]; then
    echo "Info   | Install   | git"
    brew install git
fi

# Download and install python
if [[ ! -x /usr/local/bin/python ]]; then
    echo "Info   | Install   | python"
    brew install python --framework --with-brewed-openssl
fi

# Download and install hg
if [[ ! -x /usr/local/bin/hg ]]; then
    pip install mercurial
fi

# Download and install Ansible
if [[ ! -x /usr/local/bin/ansible ]]; then
    brew install ansible
fi

# Make the code directory
mkdir -p $SRC_DIRECTORY

# Clone down ansible
if [[ ! -d $ANSIBLE_DIRECTORY ]]; then
    git clone -b devel git://github.com/danieljaouen/ansible.git $ANSIBLE_DIRECTORY
fi

# Use the forked Ansible
source $ANSIBLE_DIRECTORY/hacking/env-setup > /dev/null

# Clone down the Ansible repo
if [[ ! -d $ANSIBLE_CONFIGURATION_DIRECTORY ]]; then
    git clone git@bitbucket.org:danieljaouen/ansible-base-box.git $ANSIBLE_CONFIGURATION_DIRECTORY
    (cd $ANSIBLE_CONFIGURATION_DIRECTORY && git submodule init && git submodule update)
fi

# Provision the box
ansible-playbook --ask-sudo-pass -i $ANSIBLE_CONFIGURATION_DIRECTORY/inventories/osx $ANSIBLE_CONFIGURATION_DIRECTORY/site.yml --connection=local

# Link the casks.
~/.bin/link-casks

Note that at the moment, OS X actually comes with a gcc program that just spits out an error telling you to install Command Line Utilities, so the first block where I check for the existence of gcc is not completely correct.

How to make it better

As of this writing, the only thing that I can’t automate is App Store downloads. Hopefully Apple will fix this in the near future :)

That’s it! Thanks for reading!

A Short Primer on Erlang and OTP

After spending a bit of time experimenting with Erlang, I am conviced that it is “The Next Big Thing”. Not only is it extremely performant (just check out Cowboy, a light-weight HTTP server written in Erlang), it also avoids the kind of callback mess commonly seen in node.js applications. Erlang web apps are the best of both worlds.

How does Erlang manage to do this? The answer is OTP.

What is OTP?

OTP is a set of behaviours that form a “contract” between your code and the external world. Scalable web apps tend to have certain properties, and these properties have been codified using Erlang’s OTP library.

Let’s dig in with an example.

supervisor behaviour

One commonly seen pattern in scalable web apps is the “supervisor” pattern. We set up one process to watch running code, perhaps restarting processes that have died. Erlang’s OTP library codifies this pattern into a “behaviour”. Essentially, you create an Erlang module and specify that the module will use the supervisor behaviour. For example:

“Example Supervisor” (example_sup.erl) download
1
2
3
4
5
6
7
8
9
10
11
12
-module(example_sup).
-behaviour(supervisor).

-export([start_link/0, init/1]).

start_link() ->
    supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
    Children = [{child, {child, start_link, []},
                 permanent, 5000, worker, [cowboy_clock]}],
    {ok, {{one_for_one, 10, 10}, Children}}.

application behaviour

An application in OTP can be thought of as one thing that you can start or stop as a whole. A common pattern is to call your supervisor’s start_link function, which usually starts Erlang processes.

For example:

“Example Application” (example_app.erl) download
1
2
3
4
5
6
7
8
9
10
-module(example_app).
-behaviour(application).

-export([start/2, stop/1]).

start(_StartType, _StartArgs) ->
    example_sup:start_link().

stop(_State) ->
    ok.

More information

You can find more information about Erlang’s OTP functionality here: http://www.erlang.org/doc/design_principles/users_guide.html. Note that I’m still really new to Erlang myself, so I might have a few details wrong here or there. If you find any issues, feel free to reach out to me on Twitter: @danieljaouen.

Thanks for reading!

Intro to Ansible

Ansible is a radically simple configuration management tool comparable to Puppet or Chef. Today, I would like to share with you how I structure my Ansible roles and playbooks so as to facilitate both sane defaults and maximum configurability.

Role Directory Structure

At the root level, I have a roles directory that contains each of my roles. I tend to structure my roles like so:

.
└── apache
    ├── defaults
    │   └── main.yml
    ├── handlers
    │   └── main.yml
    ├── tasks
    │   ├── common.yml
    │   └── main.yml
    └── templates
        └── etc
            └── apache2
                └── apache2.conf.j2

What’s nice about this structure is that it contains defaults (instead of vars) that can be overridden at the group level (which I will discuss later). The use of defaults allows things to just work out of the box.

Role Variable/Default Structure

Each role has namespaced variables. The main.yml file under the defaults directory looks something like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
base_osx:
  common:
    taps:
      - homebrew/dupes
      - homebrew/science
      - homebrew/versions
      - phinze/cask

    packages:
      - ansible
      - aspell
      - bitlbee
      - brew-cask

The top-level dict key (i.e., base_osx) is the name of the role, while immediately below that (i.e., common) is the task filename. This helps to avoid variable naming conflicts.

Role Task Structure

The role task structure looks something like this:

1
2
3
4
5
6
7
8
9
# file: roles/base_osx/tasks/common.yml

---
- name: base_osx | common | brew update
  homebrew_class: update_homebrew=yes

- name: base_osx | common | brew tap base_osx_common.taps
  homebrew_tap: name=$item state=present
  with_items: base_osx.common.taps

Notice how my with_items tasks refer to namespaced variables.

Role Group Variables

One thing I do to keep things nice and decoupled is to create a separate playbook and inventory group for each role. For example:

.
├── site.yml
├── apache.yml
└── group_vars
    ├── all.yml
    └── apache.yml

And the inventory file:

[servers]
192.168.1.1

[apache:children]
servers

Thanks!

Much love to Michael DeHaan (@laserllama) for creating Ansible!

Feel free to reach out to me on Twitter (@danieljaouen) with any comments or criticism. Thanks!