12 min read

In this article by Thomas Uphill, the author of Mastering Puppet Second Edition, we will look at custom types. Puppet separates the implementation of a type into the type definition and any one of the many providers for that type. For instance, the package type in Puppet has multiple providers depending on the platform in use (apt, yum, rpm, gem, and others). Early on in Puppet development there were only a few core types defined. Since then, the core types have expanded to the point where anything that I feel should be a type is already defined by core Puppet. The LVM module creates a type for defining logical volumes, and the concat module creates types for defining file fragments. The firewall module creates a type for defining firewall rules. Each of these types represents something on the system with the following properties:

  • Unique
  • Searchable
  • Atomic
  • Destroyable
  • Creatable

(For more resources related to this topic, see here.)

When creating a new type, you have to make sure your new type has these properties. The resource defined by the type has to be unique, which is why the file type uses the path to a file as the naming variable (namevar). A system may have files with the same name (not unique), but it cannot have more than one file with an identical path. As an example, the ldap configuration file for openldap is /etc/openldap/ldap.conf, the ldap configuration file for the name services library is /etc/ldap.conf. If you used filename, then they would both be the same resource. Resources must be unique. By atomic, I mean it is indivisible; it cannot be made of smaller components. For instance, the firewall module creates a type for single iptables rules. Creating a type for the tables (INPUT, OUTPUT, FORWARD) within iptables wouldn’t be atomic—each table is made up of multiple smaller parts, the rules. Your type has to be searchable so that Puppet can determine the state of the thing you are modifying. A mechanism has to exist to know what the current state is of the thing in question. The last two properties are equally important. Puppet must be able to remove the thing, destroy it, and likewise, Puppet must be able to create the thing anew.

Given these criteria, there are several modules that define new types, with some examples including types that manage:

  • Git repositories
  • Apache virtual hosts
  • LDAP entries
  • Network routes
  • Gem modules
  • Perl CPAN modules
  • Databases
  • Drupal multisites

Creating a new type

As an example, we will create a gem type for managing Ruby gems installed for a user. Ruby gems are packages for Ruby that are installed on the system and can be queried like packages.

Installing gems with Puppet can already be done using the gem, pe_gem, or pe_puppetserver_gem providers for the package type.

To create a custom type requires some knowledge of Ruby. In this example, we assume the reader is fairly literate in Ruby. We start by defining our type in the lib/puppet/type directory of our module. We’ll do this in our example module, modules/example/lib/puppet/type/gem.rb.

The file will contain the newtype method and a single property for our type, version as shown in the following code:

Puppet::Type.newtype(:gem) do
  ensurable
  newparam(:name, :namevar => true) do
    desc 'The name of the gem'
  end
  newproperty(:version) do
    desc 'version of the gem'
    validate do |value|
      fail("Invalid gem version #{value}") unless value =~ /^[0-9]
        +[0-9A-Za-z.-]+$/
    end
  end
end

The ensurable keyword creates the ensure property for our new type, allowing the type to be either present or absent. The only thing we require of the version is that it starts with a number and only contain numbers, letters, periods, or dashes.

A more thorough regular expression here could save you time later, such as checking that the version ends with a number or letter.

Now we need to start making our provider. The name of the provider is the name of the command used to manipulate the type. For packages, the providers are named things like yum, apt, and dpkg. In our case we’ll be using the gem command to manage gems, which makes our path seem a little redundant. Our provider will live at modules/example/lib/puppet/provider/gem/gem.rb.

We’ll start our provider with a description of the provider and the commands it will use as shown in the following code:

Puppet::Type.type(:gem).provide :gem do
  desc "Manages gems using gem"

Then we’ll define a method to list all the gems installed on the system as shown in the following code, which defines the self.instances method:

def self.instances
  gems = []
  command = 'gem list -l'
    begin
      stdin, stdout, stderr = Open3.popen3(command)
      for line in stdout.readlines
        (name,version) = line.split(' ')
        gem = {}
        gem[:provider] = self.name
        gem[:name] = name
        gem[:ensure] = :present
        gem[:version] = version.tr('()','')
        gems << new(gem)
      end
    rescue
      raise Puppet::Error, "Failed to list gems using '#
        {command}'"
    end
    gems
  end

This method runs gem list -l and then parses the output looking for lines such as gemname (version). The output from the gem command is written to the variable stdout. We then use readlines on stdout to create an array which we iterate over with a for loop. Within the for loop we split the lines of output based on a space character into the gem name and version. The version will be wrapped in parenthesis at this point, we use the tr (translate) method to remove the parentheses. We create a local hash of these values and then append the hash to the gems hash. The gems hash is returned and then Puppet knows all about the gems installed on the system.

Puppet needs two more methods at this point, a method to determine if a gem exists (is installed), and if it does exist, which version is installed. We already populated the ensure parameter, so as to use that to define our exists method as follows:

def exists?
  @property_hash[:ensure] == :present
end

To determine the version of an installed gem, we can use the property_hash variable as follows:

def version
  @property_hash[:version] || :absent
end

To test this, add the module to a node and pluginsync the module over to the node as follows:

[root@client ~]# puppet plugin download
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/provider/gem]/
  ensure: created
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/provider/gem/
  gem.rb]/ensure: defined content as 
  '{md5}4379c3d0bd6c696fc9f9593a984926d3'
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/provider/gem/
  gem.rb.orig]/ensure: defined content as 
  '{md5}c6024c240262f4097c0361ca53c7bab0'
Notice: /File[/opt/puppetlabs/puppet/cache/lib/puppet/type/gem.rb]/
  ensure: defined content as '{md5}48749efcd33ce06b401d5c008d10166c'
Downloaded these plugins: /opt/puppetlabs/puppet/cache/lib/puppet/provider/gem, /opt/puppetlabs/puppet/cache/lib/puppet/provider/gem/gem.rb, /opt/puppetlabs/puppet/cache/lib/puppet/provider/gem/gem.rb.orig, /opt/puppetlabs/puppet/cache/lib/puppet/type/gem.rb

This will install our type/gem.rb and provider/gem/gem.rb files into /opt/puppetlabs/puppet/cache/lib/puppet on the node. After that, we are free to run puppet resource on our new type to list the available gems as shown in the following code:

[root@client ~]# puppet resource gem
gem { 'bigdecimal':
  ensure  => 'present',
  version => '1.2.0',
}
gem { 'bropages':
  ensure  => 'present',
  version => '0.1.0',
}
gem{ 'commander':
  ensure  => 'present',
  version => '4.1.5',
}
gem { 'highline':
  ensure  => 'present',
  version => '1.6.20',
}
…

Now, if we want to manage gems, we’ll need to create and destroy them, and we’ll need to provide methods for those operations. If we try at this point, Puppet will fail, as we can see from the following output:

[root@client ~]# puppet resource gem bropages
pugem { 'bropages':
  ensure => 'present',
  version => '0.1.0',
}
[root@client ~]# puppet resource gem bropages ensure=absent
gem { 'bropages':
  ensure => 'absent',
}
[root@client ~]# puppet resource gem bropages ensure=absent
gem { 'bropages':
  ensure => 'absent',
}

When we run puppet resource, there is no destroy method, so puppet returns that it the gem was removed but doesn’t actually do anything. To get Puppet to actually remove the gem, we’ll need a method to destroy (remove) gems, gem uninstall should do the trick, as shown in the following code:

def destroy
  g = @resource[:version] ? [@resource[:name], '--version', 
    @resource[:version]] : @resource[:name]
  command = "gem uninstall #{g} -q -x"
    begin
      system command
    rescue
      raise Puppet::Error, "Failed to remove #{@resource[:name]} 
        '#{command}'"
    end
    @property_hash.clear
  end

Using the ternary operator, we either run gem uninstall name -q -x if no version is defined, or gem uninstall name –version version -q -x if a version is defined. We finish by calling @property_hash.clear to remove the gem from the property_hash since the gem is now removed.

Now we need to let Puppet know about the state of the bropages gem using
our instances method we defined earlier, we’ll need to write a new method to prefetch all the available gems. This is done with self.prefetch, as shown in
the following code: 

def self.prefetch(resources)
    gems = instances
    resources.keys.each do |name|
      if provider = gems.find{ |gem| gem.name == name }
        resources[name].provider = provider
      end
    end
  end

We can see this in action using puppet resource as shown in the following output:

[root@client ~]# puppet resource gem bropages ensure=absent
Removing bro
Successfully uninstalled bropages-0.1.0
Notice: /Gem[bropages]/ensure: removed
gem { 'bropages':
  ensure => 'absent',
}

Almost there, now we want to add bropages back, we’ll need a create method, as shown in the following code: 

 def create
    g = @resource[:version] ? [@resource[:name], '--version', 
      @resource[:version]] : @resource[:name]
    command = "gem install #{g} -q"
    begin
      system command
      @property_hash[:ensure] = :present
    rescue
      raise Puppet::Error, "Failed to install #{@resource[:name]} 
        '#{command}'"
    end
  end

Now when we run puppet resource to create the gem, we see the installation, as shown in the following output:

[root@client ~]# puppet resource gem bropages ensure=present
Successfully installed bropages-0.1.0
Parsing documentation for bropages-0.1.0
Installing ri documentation for bropages-0.1.0
1 gem installed
Notice: /Gem[bropages]/ensure: created
gem { 'bropages':
  ensure => 'present',
}

Nearly done now, we need to handle versions. If we want to install a specific version of the gem, we’ll need to define methods to deal with versions.

 def version=(value)
    command = "gem install #{@resource[:name]} --version 
      #{@resource[:version]}"
    begin
      system command
      @property_hash[:version] = value
    rescue
      raise Puppet::Error, "Failed to install gem 
        #{resource[:name]} using #{command}"
    end
  end

Now, we can tell Puppet to install a specific version of the gem and have the correct results as shown in the following output:

[root@client ~]# puppet resource gem bropages version='0.0.9'
Fetching: highline-1.7.8.gem (100%)
Successfully installed highline-1.7.8
Fetching: bropages-0.0.9.gem (100%)
Successfully installed bropages-0.0.9
Parsing documentation for highline-1.7.8
Installing ri documentation for highline-1.7.8
Parsing documentation for bropages-0.0.9
Installing ri documentation for bropages-0.0.9
2 gems installed
Notice: /Gem[bropages]/version: version changed '0.1.0' to '0.0.9'
gem { 'bropages':
  ensure  => 'present',
  version => '0.0.9',
}

This is where our choice of gem as an example breaks down as gem provides for multiple versions of a gem to be installed. Our gem provider, however, works well enough for use at this point. We can specify the gem type in our manifests and have gems installed or removed from the node. This type and provider is only an example; the gem provider for the package type provides the same features in a standard way. When considering creating a new type and provider, search the puppet forge for existing modules first.

Summary

When the defined types are not enough, you can extend Puppet with custom types and providers written in Ruby. The details of writing providers are best learned by reading the already written providers and referring to the documentation on the Puppet Labs website.

Resources for Article:

Further resources on this subject:


LEAVE A REPLY

Please enter your comment!
Please enter your name here