Viper - install, step 6/6 - configuration notes

All configuration is performed by Puppet configuration management system, http://www.reductivelabs.com/.

During installation, 'puppet' host is added to /etc/hosts with the appropriate IP. That allows puppetd daemon to start after reboot, find its configuration server and start with the work.

Puppetd contacts Puppetmaster on the server as usual, nothing to say about that step.

Puppetmaster needs to compile configuration for a host. It does that with the help of, of course, Puppet manifest files and modules, and information about the host. Information about the host is produced as follows:
  1. Puppet collects Facter facts (as usual)
  2. Puppet runs external_nodes script, scripts/node_data, to come up with the list of classes a host belongs to. In the same step, it also produces a list of parameters, basically variables that will be accessible via $name in Puppet manifests and name in ERB templates.
Then Puppet proceeds to do its thing, independent of Viper.

What we're interested in here is the scripts/node_data script. It produces all data based on information returned from LDAP queries. Initially, we were using Puppet's support for ldap nodes directly, but that approach did not offer flexibility needed.

To find node information, Puppet starts with the certificate name, which most often coincides with FQDN.

So, node_data is called with host's FQDN as argument, then it does a couple LDAP queries to extract and process all information needed (class and parameters list), and dumps the result in YAML format, as expected by Puppet.

Here's how it produces the information:

  • Classes list: looks at puppetClass attribute for the host

  • Parameters (variables):
    1. All attributes under the host entry (same like Puppet's LDAP driver would do)
    2. Looks under Debconf tree, finds all questions with "flag: puppet" and for each found, uses Debconf key name as variable name, and value as value (non-alphanumeric chars are converted to _).
Let's see an example invocation and output:


$ ./scripts/node_data h2.c1.com

Output

---
classes:
  - test
  - ntp::client
parameters:
  clientName:
    - c1.com
  cn:
    - h2
  dhcpHWAddress:
    - ethernet 00:11:6b:34:ae:8d
  dhcpOption:
    - host-name "h2"
    - routers 10.0.1.1
    - domain-name-servers 192.168.1.254
    - nis-domain "c1.com"
    - domain-name "c1.com"
    - subnet-mask 255.255.255.0
    - broadcast-address 10.0.1.255
  dhcpStatements:
    - fixed-address 10.0.1.8
  domainName:
    - c1.com
  hostName:
    - h2
  ipBroadcastNumber:
    - 10.0.1.255
  ipHostNumber:
    - 10.0.1.8
  ipNetmaskNumber:
    - 255.255.255.0
  ipNetworkNumber:
    - 10.0.1.0
  macAddress:
    - 00:11:6b:34:ae:8d
  ntp_servers:
    - 10.0.1.9
    - 10.0.1.10
  objectClass:
    - top
    - device
    - dhcpHost
    - ipHost
    - ieee802Device
    - puppetClient
  popularity_contest_participate:
    - true
  puppetclass:
    - test
    - ntp::client

Description

As we can see on the left, the script output is in YAML format, which has the property of being very natural to read.

We see that the host belongs to classes 'test' and 'ntp::client'.

The parameters list contains parameters, produced according to notes above, and ALL parameters are lists, converted to Ruby arrays when Puppet loads them from YAML.

It is an extraordinary quality of Puppet that it supports all data structures, including arrays, and does not try to play silly with various string expansions, splitting etc.

Anyway, the parameters and their "arraic" property comes to play in ERB templates. Since ALL parameters are array, it is very straightforward to remember how to access them. Also, if in the future you change single-value parameters to multi-value (or vice versa), the scripts will still work (maybe not exactly as you expect, but they won't break configuration nor be completely off).

Example: in the output on the left, we see parameter ntp_servers. That one comes from a following Debconf question (in LDIF format):

dn: cn=ntp/servers,ou=hosts,ou=defaults
objectClass: top
objectClass: debConfDbEntry
cn: ntp/servers
owners: ntp
flags: puppet
template: ntp/servers
value: find $ ... 2 0 500 3600 (&(objectClass=puppetClient)(puppetClass=ntp::server*)) 0 ipHostNumber \0 \0

So, ntp/servers is converted to ntp_servers, and the value: is produced by running a Viper overlay called "find", which expands value to two NTP servers.

Then, ntp_servers is available as an array in ERB templates, so in case of a ntp.conf template, we could write simply:

<% ntp_servers.each do |s| -%>

server <%= s %>

<% end -%>

To produce:

server 10.0.1.9
server 10.0.1.10