Using EYAML with Puppet 4

Happy 2016 all

This weekend I finally got round to adding eyaml support to Puppet in my lab. What is on earth am I talking about?

Puppet can use a thing called Hiera as a data source, think of it as a database for configuraion. In an ideal world, your manifests will be completely generic - in fact your control repo could consist of nothing but a Puppetfile with a list of modules to install (if any one lives in that ideal world, you are better than me). Hiera in turn can have different backends for describing this data, such as:

• JSON
• YAML
• eYAML or encrypted YAML

So how do we use it? On my workstation (where I do my Puppet development), I also the eyaml gem:

gem install hiera-eyaml

It should be noted that I run RVM and run a Ruby environment and Gemset just for Puppet development (https://gogs.chriscowley.me.uk/chriscowley/puppet/src/production/.rvmrc). My Gemfile includes the hiera-eyaml gem, so I simply run bundle install.

Next, you need to create some keys:

eyaml createkeys

This creates your key pair in ./keys. Move this somewhere more sensible and configure eyaml to look there.

mkdir -pv ~/.eyaml
mv -v keys ~/.eyaml/
cat > ~/.eyaml/config.yaml < EOF
---
pkcs7_private_key: '/home/chris/.eyaml/keys/private_key.pkcs7.pem'
pkcs7_public_key: '/home/chris/.eyaml/keyspublic_key.pkcs7.pem'
EOF

Now you can test it by running eyaml encrypt -l msql_root_password -s "correcthorsebatterystaple" which will output something like:

[hiera-eyaml-core] Loaded config from /home/mmzv6833/.eyaml/config.yaml
msql_root_password: ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAPvmib/bFce7ArK/FSMHX9DVsqDo38tL/Xpc9XtWCPlqvfkfwBFPRD0qM2qbEL3JchRSmirb/yBy/20HFk7vX84PIy7IfSYEt+u2RkVUuWgSHfjnKVnJc5wul8IqHdeWIoFT5/D6dsrBP94qD6CwbIzKRRzSijuxPMbXhQQecwPBBSQAtNHWAVsw4U7lv7tVP+OoZSSnP0zqJOp2Pt6x4ivj/Wha4hPcF8KvUNKLR7ZcebHbJslJUTYqg1cMwRPMuccbXS3JvGdoFiACAPEjghbAmK6UgaZ2nTxuVGJ4B81Q6Nnsk3Ir/JVjFCKI+x+bZoVn+y1coLBPy52RE5OdPoDBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBCrcffFAXvzkNnYGpjcIVr2gCBpSG4Q9HZRDT07Yz0ijDb+3RlbLnRzlMvsP2O4phTOig==]

OR

msql_root_password: >
    ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw
    DQYJKoZIhvcNAQEBBQAEggEAPvmib/bFce7ArK/FSMHX9DVsqDo38tL/Xpc9
    XtWCPlqvfkfwBFPRD0qM2qbEL3JchRSmirb/yBy/20HFk7vX84PIy7IfSYEt
    +u2RkVUuWgSHfjnKVnJc5wul8IqHdeWIoFT5/D6dsrBP94qD6CwbIzKRRzSi
    juxPMbXhQQecwPBBSQAtNHWAVsw4U7lv7tVP+OoZSSnP0zqJOp2Pt6x4ivj/
    Wha4hPcF8KvUNKLR7ZcebHbJslJUTYqg1cMwRPMuccbXS3JvGdoFiACAPEjg
    hbAmK6UgaZ2nTxuVGJ4B81Q6Nnsk3Ir/JVjFCKI+x+bZoVn+y1coLBPy52RE
    5OdPoDBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBCrcffFAXvzkNnYGpjc
    IVr2gCBpSG4Q9HZRDT07Yz0ijDb+3RlbLnRzlMvsP2O4phTOig==]

This is all well and good for you dev environment, but utterly useless for the Puppetmaster as it has no idea about the key. Even if it did, out-of-the-box it will not look for the encrypted data anyway.

So, copy the keys to your Puppetmaster:

scp -r ~/.eyaml/keys user@puppetmaster:~/

Notice I user scp: at least the private key is extremely sensitive; never transport it in clear text, make sure you store it securely, etc, etc. You have been warned.

Now put them where your Puppetserver can get to them and install the Gem.

sudo mkdir -pv /etc/puppetlabs/puppet/secure
sudo mv -v keys /etc/puppetlabs/puppet/secure/
sudo chown -Rv puppet:puppet /etc/puppetlabs/puppet/secure
sudo chmod -Rv 550 /etc/puppetlabs/puppet/secure
sudo puppetserver gem install hiera-eyaml

Now you need to modify your hiera config file (/etc/puppetlabs/code/hiera.yaml) to look something like:

---
:backends:
  - eyaml
  - yaml

:hierarchy:
  - %{::clientcert}
  - %{::environment}
  - "virtual_%{::is_virtual}"
  - common

:yaml:
  :datadir: '/etc/puppetlabs/code/hieradata'
:eyaml:
  :datadir: '/etc/puppetlabs/code/hieradata'

  :pkcs7_private_key: /etc/puppetlabs/secure/keys/private_key.pkcs7.pem
  :pkcs7_public_key:  /etc/puppetlabs/secure/keys/public_key.pkcs7.pem

A couple of things here:

• I do not keep my hiera data in my repo, I like to have it completely separate.
  • It is truly a database for my Puppet code
  • I can use the same data across environments
• I keep my encrypted data separate too, as is default, in .eyaml files.

That is the Puppetmaster ready to use, we just need to actually put some encrypted data in there for it to collect. Think back to Puppet’s excellent Hiera docs

Now the host db01 will have potentially 2 files specific to it: db01.example.com.yaml and db01.example.com.eyaml. So to install Mysql on that (naturally using the Puppetlabs module) db01.example.com.yaml will contain:

---
classes:
  - mysql::server
mysql::server::remove_default_accounts: true

then in db01.example.com.eyaml put:

---
mysql::server::root_password: >
    ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw
    DQYJKoZIhvcNAQEBBQAEggEAhWgxLsgvtUzALxqE23nrcgy8xR+UbV5b45Vo
    joRLq4QLDhLKuwAsoaQ3MbYfrbJ5RQ2PTFlwB+Cp7X2uLQ0YYfisABT/dwaK
    9iYZoXkvsSvt8iqZkVNP9HZLf/X1EkLfljbsEx7vigMyWu8ApDt5aGCxqGA6
    NTGZkeOoUhfRM9KuzRvkIQB0eutuIx420EgKI0gdCVPv1Y51UdEMl7rClwz3
    4ATlPmL0F2NVNifZC+KdWGei+PYSYM394JvS0ZBxuNWLowlmR2SgbzSCpWZn
    mB1jolaG7nXv7Y1OnvZraA3EIUcwKiILlsC1vlXuVc6xdKBvhb70j6p30SzB
    6eF2IzBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBCH5GYGDeLdZniZdkCt
    Xe7bgCAaY8TVUw4NaHc2ARbCSAsZSH91UPaDMAWaC6wrYorLEw==]

When the agent on db01 connects, the Puppetmaster will work its way down through the hierachy populating each variable as it goes. As the eyaml backend is defined first in the hierachy, then it will take priority (for simple paramters, hashes and arrays are more complex). This means that if you were to leave an old value for mysql::server::root_password in the plain yaml file, it will be ignored.

I hope this is helpful for someone, what it means is that you can share all the code for your infrastructure with the world. I will certainly be doing this, I just need to refacter things a little to put my defined types into Hiera as I use them to create databases. This is not complex and will be the subject of a future post.