A little learning is a dangerous thing, but a lot of ignorance is just as bad.
—Bob Edwards
In this article by John Arundel, the author of Puppet 4.10 Beginner’s Guide – Second Edition, we’ll go into details of packages, files, and services to see how to exploit their power to the full. Along the way, we’ll talk about the following topics:
(For more resources related to this topic, see here.)
Puppet can manage files on the server using the file resource, and the following example sets the contents of a file[TJ1] to a particular string using the content attribute (file_hello.pp):
file { ‘/tmp/hello.txt’:
content =>“hello, worldn”,
}While it’s useful to be able to set the contents of a file to a short text string, most files we’re likely to want to manage, will be too large to include directly in our Puppet manifests. Ideally, we would put a copy of the file in the Puppet repo, and have Puppet simply copy it to the desired place in the filesystem. The source attribute (file_source.pp)does exactly that:
file { ‘/etc/motd’:
source =>‘/vagrant/examples/files/motd.txt’,
}
To try this example with your Vagrant box, run the following commands:
sudo puppet apply /vagrant/examples/file_source.pp
cat /etc/motd
The best software in the world only sucks. The worst software is significantly worse than that.
-Luke Kanies
To run such examples, just apply them using sudo puppet apply as shown in the preceding example.
Why do we have to run sudo puppet apply instead of just puppet apply?
Puppet has the permissions of the user who runs it, so if Puppet needs to modify a file owned by root, it must be run with the root’s permissions (which is what sudo does). You will usually run Puppet as root because it needs those permissions to do things such as installing packagesand modifying config files owned by root.
The value of the source attribute can be a path to a file on the server, as here, or an HTTP URL, as shown in the following example (file_http.pp):
file { ‘/tmp/README.md’:
source =>‘https://raw.githubusercontent.com/puppetlabs/puppet/master/README.md’,
}
Although this is a handy feature, bear in mind that every time you add an external dependency like this to your Puppet manifest, you’re adding a potential point of failure. Wherever you can, use a local copy of such a file instead of having Puppet fetch it remotely every time. This particularly applies to software which needs to be built from a tarball downloaded from a website. If possible, download the tarball and serve it from a local web server or file server. If this isn’t practical, using a caching proxy server can help save time and bandwidth when you’re building a large number of machines.
On Unix-like systems, files are associated with an owner, a group, and a set of permissions to read, write, or execute the file. Since we normally run Puppet with the permissions of the root user (via sudo), the files Puppet manages will be owned by that user:
ls -l /etc/motd
-rw-r--r-- 1 root root 109 Aug 31 04:03 /etc/motd
Often, this is just fine, but if we need the file to belong to another user (for example, if that user needs to be able to write to the file), we can express this by setting the owner attribute (file_owner.pp):
file { ‘/etc/owned_by_vagrant’:
ensure => present,
owner =>‘vagrant’,
}
Run the following command:
ls -l /etc/owned_by_vagrant
-rw-r--r-- 1 vagrant root 0 Aug 31 04:48 /etc/owned_by_vagrant
You can see that Puppet has created the file and its owner attribute has been set to vagrant. You can also set the group ownership of the file using the group attribute (file_group.pp):
file { ‘/etc/owned_by_vagrant’:
ensure => present,
owner =>‘vagrant’,
group =>‘vagrant’,
}
Run the following command:
ls -l /etc/owned_by_vagrant
-rw-r--r-- 1 vagrant vagrant 0 Aug 31 04:48 /etc/owned_by_vagrant
This time, we didn’t specify either a content or source attribute for the file, but simply ensure => present. In this case, Puppet will create a file of zero size (useful, for example, if you want to make sure the file exists and is writeable, but doesn’t need to have any contents yet).
Files on Unix-like systems have an associated mode, which determines access permissions for the file. It governs read, write, and execute permissions for the file’s owner, any user in the file’s group, and other users. Puppet supports setting permissions on files using the mode attribute. This takes an octal value, with each digit representing the permissions for owner, group, and other, in that order. In the following example, we use the mode attribute to set a mode of 0644 (read and write for owner, read-only for group, read-only for other) on a file (file_mode.pp):
file { ‘/etc/owned_by_vagrant’:
ensure => present,
owner =>‘vagrant’,
mode =>‘0644’,
}
This will be quite familiar to experienced system administrators, as the octal values for file permissions are exactly the same as those understood by the Unixchmod command. For more information, run theman chmod command.
Creating or managing permissions on a directory is a common task, and Puppet uses the file resource to do this too. If the value of the ensure attribute is directory, the file will be a directory (file_directory.pp):
file { ‘/etc/config_dir’:
ensure => directory,
}
As with regular files, you can use the owner, group, and mode attributes to control access to directories.
Puppet can copy a single file to the server, but what about a whole directory of files, possibly including subdirectories (known as a file tree)? The recurse attribute will take care of this (file_tree.pp):
file { ‘/etc/config_dir’:
source =>‘/vagrant/examples/files/config_dir’,
recurse => true,
}
Run the following command:
ls /etc/config_dir/
1 2 3
When recurse attribute is true, Puppet will copy all the files and directories (and their subdirectories) in the source directory (/vagrant/examples/files/config_dir in this example) to the target directory (/etc/config_dir).
If the target directory already exists and has files in it, Puppet will not interfere with them, but you can change this behavior using the purge attribute.[JR4] If this is true, Puppet will delete any files and directories in the target directory which are not present in the source directory. Use this attribute with care!
Another common requirement for managing files is to create or modify a symbolic link (known as a symlink for short). You can have Puppet do this by setting ensure => link on the file resource, and specifying the target attribute (file_symlink.pp):
file { ‘/etc/this_is_a_link’:
ensure => link,
target =>‘/etc/motd’,
}
Run the following command:
ls -l /etc/this_is_a_link
lrwxrwxrwx 1 root root 9 Aug 31 05:05 /etc/this_is_a_link -> /etc/motdTo install a package usethe package resource, and this is all you need to do with most packages. However, the package resource has a few extra features which may be useful.
The ensure attribute normally takes the installedvalue in order to install a package, but if you specify absent instead, Puppet will remove the package if it happens to be installed. Otherwise, it will take no action. The following example will remove the apparmor package if it’s installed (package_remove.pp):
package { ‘apparmor’:
ensure => absent,
}If there are multiple versions of a package available to the system’s package manager, specifying ensure => installed will cause Puppet to install the default version (usually the latest). But if you need a specific version, you can specify that version string as the value of ensure, and Puppet will install that version (package_version.pp):
package { ‘openssl’:
ensure =>‘1.0.2g-1ubuntu4.2’,
}
It’s a good idea to specify an exact version whenever you manage packages with Puppet, so that all servers will get the same version of a given package. Otherwise, if you use ensure => installed, they will just get whatever version was current at the time they were built, leading to a situation where different machines have different package versions.
When a newer version of the package is released, and you decide it’s time to upgrade to it, you can update the version string specified in the Puppet manifest and Puppet will upgrade the package everywhere.
On the other hand, if you specify ensure => latest for a package, Puppet will make sure that the latest available version is installed every time it runs. When a new version of the package becomes available, it will be installed automatically on the next Puppet run.
This is not generally what you want when using a package repository that’s not under your control (for example, the main Ubuntu repository). It means that packages will be upgraded at unexpected times, which may break your application (or at least result in unplanned downtime). A better strategy is to tell Puppet to install a specific version that you know works, and test upgrades in a controlled environment before rolling them out to production.
If you maintain your own package repository, and control the release of new packages to it, ensure => latest can be a useful feature: Puppet will update a package as soon as you push a new version to the repo. If you are relying on upstream repositories, such as the Ubuntu repositories, it’s better to tell Puppet to install a specific version and upgrade that as necessary.[JR10]
Although the package resource is most often used to install packages using the normal system package manager (in the case of Ubuntu, that’s APT), it can install other kinds of packages as well. Library packages for the Ruby programming language are known as gems. Puppet can install Ruby gems for you using the provider => gem attribute (package_gem.pp):
package { ‘ruby’:
ensure => installed,
}
package { ‘bundler’:
ensure => installed,
provider => gem,
}
In the preceding code, bundler is a Ruby gem, and therefore we have to specify provider => gem for this package so that Puppet doesn’t think it’s a standard system package and try to install it via APT. Since the gem provider is not available unless Ruby is installed, we install the ruby package first, and then the bundler gem.
Puppet itself is written at least partly in Ruby, and makes use of several Ruby gems. To avoid any conflicts with the version of Ruby and gems, which the server might need for other applications, Puppet packages its own version of Ruby and associated gems under the /etc/puppetlabs directory. This means you can install (or remove) whichever version of Ruby you like, and Puppet will not be affected.
However, if you need to install a gem to extend Puppet’s capabilities in some way, then doing it with a package resource and provider => gem won’t work. That is, the gem will be installed, but only in the system Ruby context, and it won’t be visible to Puppet.
Fortunately, the puppet_gem provider is available for exactly this purpose. When you use this provider, the gem will be installed in Puppet’s context (and, naturally, won’t be visible in the system context). The following example demonstrates how to use this provider (package_puppet_gem.pp):
package { ‘hiera-eyaml’:
ensure => installed,
provider => puppet_gem,
}
To see the gems installed in Puppet’s context, use Puppet’s own version of the gem command, with the following path:
/opt/puppetlabs/puppet/bin/gem list
Although services are implemented in a number of varied and complicated ways at the operating system level, Puppet does a good job of abstracting away most of this with the service resource and exposing just the two attributes of services which really matter , whether they’re running (ensure) and whether they start at boot time (enable).
However, you’ll occasionally encounter services that don’t play well with Puppet, for a variety of reasons. Sometimes, Puppet is unable to detect that the service is already running, and keeps trying to start it. At other times, Puppet may not be able to properly restart the service when a dependent resource changes. There are a few useful attributes for service resources that can help resolve these problems.
When a service resource has theensure => running attribute, Puppet needs to be able to check whether the service is, in fact, running. The way it does this depends on the underlying operating system, but on Ubuntu 16+, for example, it runs systemctl is-active SERVICE. If the service is packaged to work with systemd, that should be just fine, but in many cases, particularly with older software, it may not respond properly.
If you find that Puppet keeps attempting to start the service on every Puppet run, even though the service is running, it may be that Puppet’s default service status detection isn’t working. In this case, you can specify the hasstatus => false attribute for the service (service_hasstatus.pp):
service { ‘ntp’:
ensure =>running,
enable => true,
hasstatus => false,
}
When hasstatus is false, Puppet knows not to try to check the service status using the default system service management command, and instead will look in the process table for a running process thatmatches the name of the service. If it finds one, it will infer that the service is running and take no further action.
Sometimes when using hasstatus => false, the service name as defined in Puppet doesn’t actually appear in the process table because the command that provides the service has a different name. If this is the case, you can tell Puppet exactly what to look for using the pattern attribute (service_pattern.pp):
service { ‘ntp’:
ensure =>running,
enable => true,
hasstatus => false,
pattern =>‘ntpd’,
}
If hasstatus is false and pattern is specified, Puppet will search for the value of pattern in the process table to determine whether or not the service is running. To find the pattern you need, you can use the ps command to see the list of running processes:
ps axWhen a service is notified (for example, if a file resource uses the notify attribute to tell the service its config file has changed), Puppet’s default behavior is to stop the service, then start it again. This usually works, but many services implement a restart command in their management scripts. If this is available, it’s usually a good idea to use itas it may be faster or safer than stopping and starting the service. Some services take a while to shut down properly when stopped, for example, and Puppet may not wait long enough before trying to restart them, so that you end up with the service not running at all.
If you specify hasrestart => true for a service, then Puppet will try to send a restart command to it, using whatever service management command is appropriate (systemctl, for example). The following example shows the use of hasrestart (service_hasrestart.pp):
service { ‘ntp’:
ensure =>running,
enable => true,
hasrestart => true,
}
To further complicate things, the default system service restart command may not work, or you may need to take certain special actions when the service is restarted (disabling monitoring notifications, for example). You can specify any restart command you like for the service using the restart attribute (service_custom_restart.pp):
service { ‘ntp’:
ensure => running,
enable => true,
restart =>‘/bin/echo Restarting >>/tmp/debug.log && systemctl restart ntp’,
}
In this example, the restart command writes a message to a log file before restarting the service in the usual way, but it could, of course, do anything you need it to.
In the extremely rare event that the service cannot be stopped or started using the default service management command, Puppet also provides the stop and start attributes so that you can specify custom commands to stop and start the service, in just the same way as with the restart attribute. If you need to use either of these, though, it’s probably safe to say that you’re having a bad day.
In this article, we explored Puppet’s file resource in detail, covering file sources, ownership, permissions, directories, symbolic links, and file trees. You learned how to manage packages by installing specific versions, or the latest version, and how to uninstall packages. We also covered Ruby gems, both in the system context and Puppet’s internal context.
We looked at service resources, including the has status, pattern, has restart, restart, stop, and start attributes.
Further resources on this subject:
I remember deciding to pursue my first IT certification, the CompTIA A+. I had signed…
Key takeaways The transformer architecture has proved to be revolutionary in outperforming the classical RNN…
Once we learn how to deploy an Ubuntu server, how to manage users, and how…
Key-takeaways: Clean code isn’t just a nice thing to have or a luxury in software projects; it's a necessity. If we…
While developing a web application, or setting dynamic pages and meta tags we need to deal with…
Software architecture is one of the most discussed topics in the software industry today, and…