In this article by Mayank Joshi, the author of Mastering Chef, we’ll learn how to go about building custom Knife plugins and we’ll also see how we can write custom handlers that can help us extend the functionality provided by a chef-client run to report any issues with a chef-client run.

(For more resources related to this topic, see here.)

Custom Knife plugins

Knife is one of the most widely used tools in the Chef ecosystem. Be it managing your clients, nodes, cookbooks, environments, roles, users, or handling stuff such as provisioning machines in Cloud environments such as Amazon AWS, Microsoft Azure, and so on, there is a way to go about doing all of these things through Knife. However, Knife, as provided during installation of Chef, isn’t capable of performing all these tasks on its own. It comes with a basic set of functionalities, which helps provide an interface between the local Chef repository, workstation and the Chef server.

The following are the functionalities, which is provided, by default, by the Knife executable:

• Management of nodes
• Management of clients and users
• Management of cookbooks, roles, and environments
• Installation of chef-client on the nodes through bootstrapping
• Searching for data that is indexed on the Chef server.

However, apart from these functions, there are plenty more functions that can be performed using Knife; all this is possible through the use of plugins. Knife plugins are a set of one (or more) subcommands that can be added to Knife to support an additional functionality that is not built into the base set of Knife subcommands. Most of the Knife plugins are initially built by users such as you, and over a period of time, they are incorporated into the official Chef code base. A Knife plugin is usually installed into the ~/.chef/plugins/knife directory, from where it can be executed just like any other Knife subcommand. It can also be loaded from the .chef/plugins/knife directory in the Chef repository or if it’s installed through RubyGems, it can be loaded from the path where the executable is installed.

Ideally, a plugin should be kept in the ~/.chef/plugins/knife directory so that it’s reusable across projects, and also in the .chef/plugins/knife directory of the Chef repository so that its code can be shared with other team members. For distribution purpose, it should ideally be distributed as a Ruby gem.

The skeleton of a Knife plugin

A Knife plugin is structured somewhat like this:

require 'chef/knife'
 
module ModuleName
class ClassName < Chef::Knife
 
   deps do
     require 'chef/dependencies'
   end
 
   banner "knife subcommand argument VALUE (options)"
 
   option :name_of_option
     :short => "-l value",
     :long => "--long-option-name value",
     :description => "The description of the option",
     :proc => Proc.new { code_to_be_executed },
     :boolean => true | false,
     :default => default_value
 
   def run
     #Code
   end
end
end

Let’s look at this skeleton, one line at a time:

• require: This is used to require other Knife plugins required by a new plugin.
• module ModuleName: This defines the namespace in which the plugin will live. Every Knife plugin lives in its own namespace.
• class ClassName < Chef::Knife: This declares that a plugin is a subclass of Knife.
• deps do: This defines a list of dependencies.
• banner: This is used to display a message when a user enters Knife subcommand –help.
• option :name_of_option: This defines all the different command line options available for this new subcommand.
• def run: This is the place in which we specify the Ruby code that needs to be executed.

Here are the command-line options:

• :short defines the short option name
• :long defines the long option name
• :description defines a description that is displayed when a user enters knife subclassName –help
• :boolean defines whether an option is true or false; if the :short and :long names define value, then this attribute should not be used
• :proc defines the code that determines the value for this option
• :default defines a default value

The following example shows a part of a Knife plugin named knife-windows:

require 'chef/knife'
require 'chef/knife/winrm_base'base'
 
class Chef
class Knife
   class Winrm < Knife
 
     include Chef::Knife::WinrmBase
 
     deps do
       require 'readline'
       require 'chef/search/query'
       require 'em-winrm'
     end
 
     attr_writer :password
 
     banner "knife winrm QUERY COMMAND (options)"
 
     option :attribute,
       :short => "-a ATTR",
       :long => "--attribute ATTR",
       :description => "The attribute to use for opening the connection - default is fqdn",
       :default => "fqdn"
 
     ... # more options
 
     def session
       session_opts = {}
       session_opts[:logger] = Chef::Log.logger if Chef::Log.level == :debug
       @session ||= begin
         s = EventMachine::WinRM::Session.new(session_opts)
         s.on_output do |host, data|
           print_data(host, data)
         end
         s.on_error do |host, err|
           print_data(host, err, :red)
         end
         s.on_command_complete do |host|
            host = host == :all ? 'All Servers' : host
           Chef::Log.debug("command complete on #{host}")
         end
         s
       end
 
     end
 
     ... # more def blocks
 
   end
end
end

Namespace

As we saw with skeleton, the Knife plugin should have its own namespace and the namespace is declared using the module method as follows:

require 'chef/knife'
#Any other require, if needed
 
module NameSpace
class SubclassName < Chef::Knife

Here, the plugin is available under the namespace called NameSpace. One should keep in mind that Knife loads the subcommand irrespective of the namespace to which it belongs.

Class name

The class name declares a plugin as a subclass of both Knife and Chef. For example:

class SubclassName < Chef::Knife

The capitalization of the name is very important. The capitalization pattern can be used to define the word grouping that makes the best sense for the use of a plugin.

For example, if we want our plugin subcommand to work as follows:

knife bootstrap hdfs

We should have our class name as: BootstrapHdfs.

If, say, we used a class name such as BootStrapHdfs, then our subcommand would be as follows:

knife boot strap hdfs

It’s important to remember that a plugin can override an existing Knife subcommand. For example, we already know about commands such as knife cookbook upload. If you want to override the current functionality of this command, all you need to do is create a new plugin with the following name:

class CookbookUpload < Chef::Knife

Banner

Whenever a user enters the knife –help command, he/she is presented with a list of available subcommands. For example:

knife --help
Usage: knife sub-command (options)
   -s, --server-url URL             Chef Server URL
Available subcommands: (for details, knife SUB-COMMAND --help)
 
** BACKUP COMMANDS **
knife backup export [COMPONENT [COMPONENT ...]] [-D DIR] (options)
knife backup restore [COMPONENT [COMPONENT ...]] [-D DIR] (options)
 
** BOOTSTRAP COMMANDS **
knife bootstrap FQDN (options)
....

Let us say we are creating a new plugin and we would want Knife to be able to list it when a user enters the knife –help command. To accomplish this, we would need to make use of banner.

For example, let’s say we’ve a plugin called BootstrapHdfs with the following code:

module NameSpace
class BootstrapHdfs < Chef::Knife
   ...
   banner "knife bootstrap hdfs (options)"
   ...
end
end

Now, when a user enters the knife –help command, he’ll see the following output:

** BOOTSTRAPHDFS COMMANDS **
knife bootstrap hdfs (options)

Dependencies

Reusability is one of the key paradigms in development and the same is true for Knife plugins. If you want a functionality of one Knife plugin to be available in another, you can use the deps method to ensure that all the necessary files are available. The deps method acts like a lazy loader, and it ensures that dependencies are loaded only when a plugin that requires them is executed.

This is one of the reasons for using deps over require, as the overhead of the loading classes is reduced, thereby resulting in code with a lower memory footprint; hence, faster execution.

One can use the following syntax to specify dependencies:

deps do
require 'chef/knife/name_of_command'
require 'chef/search/query'
#Other requires to fullfill dependencies
end

Requirements

One can acquire the functionality available in other Knife plugins using the require method. This method can also be used to require the functionality available in other external libraries. This method can be used right at the beginning of the plugin script, however, it’s always wise to use it inside deps, or else the libraries will be loaded even when they are not being put to use.

The syntax to use require is fairly simple, as follows:

require 'path_from_where_to_load_library'

Let’s say we want to use some functionalities provided by the bootstrap plugin. In order to accomplish this, we will first need to require the plugin:

require 'chef/knife/bootstrap'

Next, we’ll need to create an object of that plugin:

obj = Chef::Knife::Bootstrap.new

Once we’ve the object with us, we can use it to pass arguments or options to that object. This is accomplished by changing the object’s config and the name_arg variables. For example:

obj.config[:use_sudo] = true

Finally, we can run the plugin using the run method as follows:

obj.run

Options

Almost every other Knife plugin accepts some command line option or other. These options can be added to a Knife subcommand using the option method. An option can have a Boolean value, string value, or we can even write a piece of code to determine the value of an option.

Let’s see each of them in action once:

An option with a Boolean value (true/false):

option :true_or_false,
:short => "-t",
:long => "—true-or-false",
:description => "True/False?",
:boolean => true | false,
:default => true

Here is an option with a string value:

option :some_string_value,
:short => "-s VALUE",
:long => "—some-string-value VALUE",
:description => "String value",
:default => "xyz"

An option where a code is used to determine the option’s value:

option :tag,
:short => "-T T=V[,T=V,...]",
:long => "—tags Tag=Value[,Tag=Value,...]",
:description => "A list of tags",
:proc => Proc.new { |tags| tag.split(',') }

Here the proc attribute will convert a list of comma-separated values into an array.

All the options that are sent to the Knife subcommand through a command line are available in form of a hash, which can be accessed using the config method.

For example, say we had an option:

option :option1
:short => "-s VALUE",
:long => "—some-string-value VALUE",
:description => "Some string value for option1",
:default => "option1"

Now, while issuing the Knife subcommand, say a user entered something like this:

$ knife subcommand –option1 "option1_value"

We can access this value for option1 in our Knife plugin run method using config[:option1]

When a user enters the knife –help command, the description attributes are displayed as part of help. For example:

**EXAMPLE COMMANDS**
knife example
 -s, --some-type-of-string-value    This is not a random string value.
 -t, --true-or-false                 Is this value true? Or is this value false?
 -T, --tags                         A list of tags associated with the virtual machine.

Arguments

A Knife plugin can also accept the command-line arguments that aren’t specified using the option flag, for example, knife node show NODE. These arguments are added using the name_args method:

require 'chef/knife'
module MyPlugin
class ShowMsg << Chef::Knife
   banner 'knife show msg MESSAGE'
   def run
     unless name_args.size == 1
     puts "You need to supply a string as an argument."
       show_usage
       exit 1
     end
     msg = name_args.join(" ")
     puts msg
   end
end
end

Let’s see this in action:

knife show msg
You need to supply a string as an argument.
USAGE: knife show msg MESSAGE
   -s, --server-url URL             Chef Server URL
       --chef-zero-host HOST       Host to start chef-zero on
...

Here, we didn’t pass any argument to the subcommand and, rightfully, Knife sent back a message saying You need to supply a string as an argument.

Now, let’s pass a string as an argument to the subcommand and see how it behaves:

knife show msg "duh duh"
duh duh

Under the hood what’s happening is that name_args is an array, which is getting populated by the arguments that we have passed in the command line. In the last example, the name_args array would’ve contained two entries (“duh”,”duh”). We use the join method of the Array class to create a string out of these two entities and, finally, print the string.

The run method

Every Knife plugin will have a run method, which will contain the code that will be executed when the user executes the subcommand. This code contains the Ruby statements that are executed upon invocation of the subcommand. This code can access the options values using the config[:option_hash_symbol_name] method.

Search inside a custom Knife plugin

Search is perhaps one of the most powerful and most used functionalities provided by Chef. By incorporating a search functionality in our custom Knife plugin, we can accomplish a lot of tasks, which would otherwise take a lot of efforts to accomplish. For example, say we have classified our infrastructure into multiple environments and we want a plugin that can allow us to upload a particular file or folder to all the instances in a particular environment on an ad hoc basis, without invoking a full chef-client run. This kind of stuff is very much doable by incorporating a search functionality into the plugin and using it to find the right set of nodes in which you want to perform a certain operation. We’ll look at one such plugin in the next section.

To be able to use Chef’s search functionality, all you need to do is to require the Chef’s query class and use an object of the Chef::Search::Query class to execute a query against the Chef server. For example:

require 'chef/search/query'
query_object = Chef::Search::Query.new
query = 'chef_environment:production'
query_object.search('node',query) do |node|
puts "Node name = #{node.name}"
end

Since the name of a node is generally FQDN, you can use the values returned in node.name to connect to remote machines and use any library such as net-scp to allow users to upload their files/folders to a remote machine. We’ll try to accomplish this task when we write our custom plugin at the end of this article.

We can also use this information to edit nodes. For example, say we had a set of machines acting as web servers. Initially, all these machines were running Apache as a web server. However, as the requirements changed, we wanted to switch over to Nginx. We can run the following piece of code to accomplish this task:

require 'chef/search/query'
 
query_object = Chef::Search::Query.new
query = 'run_list:*recipe\[apache2\]*'
query_object.search('node',query) do |node|
ui.msg "Changing run_list to recipe[nginx] for #{node.name}"
node.run_list("recipe[nginx]")
node.save
ui.msg "New run_list: #{node.run_list}"
end

knife.rb settings

Some of the settings defined by a Knife plugin can be configured so that they can be set inside the knife.rb script. There are two ways to go about doing this:

• By using the :proc attribute of the option method and code that references Chef::Config[:knife][:setting_name]
• By specifying the configuration setting directly within the def Ruby blocks using either Chef::Config[:knife][:setting_name] or config[:setting_name]

An option that is defined in this way can be configured in knife.rb by using the following syntax:

knife [:setting_name]

This approach is especially useful when a particular setting is used a lot. The precedence order for the Knife option is:

1. The value passed via a command line.
2. The value saved in knife.rb
3. The default value.

The following example shows how the Knife bootstrap command uses a value in knife.rb using the :proc attribute:

option :ssh_port
:short => '-p PORT',
:long => '—ssh-port PORT',
:description => 'The ssh port',
:proc => Proc.new { |key| Chef::Config[:knife][:ssh_port] = key }

Here Chef::Config[:knife][:ssh_port] tells Knife to check the knife.rb file for a knife[:ssh_port] setting.

The following example shows how the Knife bootstrap command calls the knife ssh subcommand for the actual SSH part of running a bootstrap operation:

def knife_ssh
ssh = Chef::Knife::Ssh.new
ssh.ui = ui
ssh.name_args = [ server_name, ssh_command ]
ssh.config[:ssh_user] = Chef::Config[:knife][:ssh_user] || config[:ssh_user]
ssh.config[:ssh_password] = config[:ssh_password]
ssh.config[:ssh_port] = Chef::Config[:knife][:ssh_port] || config[:ssh_port]
ssh.config[:ssh_gateway] = Chef::Config[:knife][:ssh_gateway] || config[:ssh_gateway]
ssh.config[:identity_file] = Chef::Config[:knife][:identity_file] || config[:identity_file]
ssh.config[:manual] = true
ssh.config[:host_key_verify] = Chef::Config[:knife][:host_key_verify] || config[:host_key_verify]
ssh.config[:on_error] = :raise
ssh
end

Let’s take a look at the preceding code:

• ssh = Chef::Knife::Ssh.new creates a new instance of the Ssh subclass named ssh
• A series of settings in Knife ssh are associated with a Knife bootstrap using the ssh.config[:setting_name] syntax
• Chef::Config[:knife][:setting_name] tells Knife to check the knife.rb file for various settings
• It also raises an exception if any aspect of the SSH operation fails

User interactions

The ui object provides a set of methods that can be used to define user interactions and to help ensure a consistent user experience across all different Knife plugins. One should make use of these methods, rather than handling user interactions manually.

Method	Description
ui.ask(*args, &block)	The ask method calls the corresponding ask method of the HighLine library. More details about the HighLine library can be found at http://www.rubydoc.info/gems/highline/1.7.2.
ui.ask_question(question, opts={})	This is used to ask a user a question. If :default => default_value is passed as a second argument, default_value will be used if the user does not provide any answer.
ui.color (string, *colors)	This method is used to specify a color. For example: server = connections.server.create(server_def)   puts “#{ui.color(“Instance ID”, :cyan)}: #{server.id}”   puts “#{ui.color(“Flavor”, :cyan)}: #{server.flavor_id}”   puts “#{ui.color(“Image”, :cyan)}: #{server.image_id}”   …   puts “#{ui.color(“SSH Key”, :cyan)}: #{server.key_name}” print “n#{ui.color(“Waiting for server”, :magenta)}”
ui.color?()	This indicates that the colored output should be used. This is only possible if an output is sent across to a terminal.
ui.confirm(question,append_instructions=true)	This is used to ask (Y/N) questions. If a user responds back with N, the command immediately exits with the status code 3.
ui.edit_data(data,parse_output=true)	This is used to edit data. This will result in firing up of an editor.
ui.edit_object(class,name)	This method provides a convenient way to download an object, edit it, and save it back to the Chef server. It takes two arguments, namely, the class of object to edit and the name of object to edit.
ui.error	This is used to present an error to a user.
ui.fatal	This is used to present a fatal error to a user.
ui.highline	This is used to provide direct access to a highline object provided by many ui methods.
ui.info	This is used to present information to a user.
ui.interchange	This is used to determine whether the output is in a data interchange format such as JSON or YAML.
ui.list(*args)	This method is a way to quickly and easily lay out lists. This method is actually a wrapper to the list method provided by the HighLine library. More details about the HighLine library can be found at http://www.rubydoc.info/gems/highline/1.7.2.
ui.msg(message)	This is used to present a message to a user.
ui.output(data)	This is used to present a data structure to a user. This makes use of a generic default presenter.
ui.pretty_print(data)	This is used to enable the pretty_print output for JSON data.
ui.use_presenter(presenter_class)	This is used to specify a custom output presenter.
ui.warn(message)	This is used to present a warning to a user.

For example, to show a fatal error in a plugin in the same way that it would be shown in Knife, do something similar to the following:

unless name_args.size == 1
   ui.fatal "Fatal error !!!"
   show_usage
   exit 1
end

Exception handling

In most cases, the exception handling available within Knife is enough to ensure that the exception handling for a plugin is consistent across all the different plugins. However, if the required one can handle exceptions in the same way as any other Ruby program, one can make use of the begin-end block, along with rescue clauses, to tell Ruby which exceptions we want to handle.

For example:

def raise_and_rescue
begin
   puts 'Before raise'
   raise 'An error has happened.'
   puts 'After raise'
rescue
   puts 'Rescued'
end
puts 'After begin block'
end
 
raise_and_rescue

If we were to execute this code, we’d get the following output:

ruby test.rb
Before raise
Rescued
After begin block

A simple Knife plugin

With the knowledge about how Knife’s plugin system works, let’s go about writing our very own custom Knife plugin, which can be quite useful for some users. Before we jump into the code, let’s understand the purpose that this plugin is supposed to serve. Let’s say we’ve a setup where our infrastructure is distributed across different environments and we’ve also set up a bunch of roles, which are used while we try to bootstrap the machines using Chef.

So, there are two ways in which a user can identify machines:

• By environments
• By roles

Actually, any valid Chef search query that returns a node list can be the criteria to identify machines. However, we are limiting ourselves to these two criteria for now.

Often, there are situations where a user might want to upload a file or folder to all the machines in a particular environment, or to all the machines belonging to a particular role. This plugin will help users accomplish this task with lots of ease. The plugin will accept three arguments. The first one will be a key-value pair with the key being chef_environment or a role, the second argument will be a path to the file or folder that is required to be uploaded, and the third argument will be the path on a remote machine where the files/folders will be uploaded to. The plugin will use Chef’s search functionality to find the FQDN of machines, and eventually make use of the net-scp library to transfer the file/folder to the machines.

Our plugin will be called knife-scp and we would like to use it as follows:

knife scp chef_environment:production /path_of_file_or_folder_locally /path_on_remote_machine

Here is the code that can help us accomplish this feat:

require 'chef/knife'
 
module CustomPlugins
class Scp < Chef::Knife
   banner "knife scp SEARCH_QUERY PATH_OF_LOCAL_FILE_OR_FOLDER PATH_ON_REMOTE_MACHINE"
 
   option :knife_config_path,
     :short => "-c PATH_OF_knife.rb",
     :long => "--config PATH_OF_knife.rb",
     :description => "Specify path of knife.rb",
     :default => "~/.chef/knife.rb"
 
   deps do
     require 'chef/search/query'
     require 'net/scp'
     require 'parallel'
   end
 
   def run
     if name_args.length != 3
       ui.msg "Missing arguments! Unable to execute the command successfully."
       show_usage
       exit 1
     end
                 Chef::Config.from_file(File.expand_path("#{config[:knife_config_path]}"))
     query = name_args[0]
     local_path = name_args[1]
     remote_path = name_args[2]
     query_object = Chef::Search::Query.new
     fqdn_list = Array.new
     query_object.search('node',query) do |node|
       fqdn_list << node.name
     end
     if fqdn_list.length < 1
       ui.msg "No valid servers found to copy the files to"
     end
     unless File.exist?(local_path)
       ui.msg "#{local_path} doesn't exist on local machine"
       exit 1
     end
 
     Parallel.each((1..fqdn_list.length).to_a, :in_processes => fqdn_list.length) do |i|
       puts "Copying #{local_path} to #{Chef::Config[:knife][:ssh_user]}@#{fqdn_list[i-1]}:#{remote_path} "
       Net::SCP.upload!(fqdn_list[i-1],"#{Chef::Config[:knife][:ssh_user]}","#{local_path}","#{remote_path}",:ssh => { :keys => ["#{Chef::Config[:knife][:identity_file]}"] }, :recursive => true)
     end
   end
end
end

This plugin uses the following additional gems:

• The parallel gem to execute statements in parallel. More information about this gem can be found at https://github.com/grosser/parallel.
• The net-scp gem to do the actual transfer. This gem is a pure Ruby implementation of the SCP protocol. More information about the gem can be found at https://github.com/net-ssh/net-scp.

Both these gems and the Chef search library are required in the deps block to define the dependencies.

This plugin accepts three command line arguments and uses knife.rb to get information about which user to connect over SSH and also uses knife.rb to fetch information about the SSH key file to use. All these command line arguments are stored in the name_args array.

A Chef search is then used to find a list of servers that match the query, and eventually a parallel gem is used to parallely SCP the file from a local machine to a list of servers returned by a Chef query.

As you can see, we’ve tried to handle a few error situations, however, there is still a possibility of this plugin throwing away errors as the Net::SCP.upload function can error out at times.

Let’s see our plugin in action:

Case1: The file that is supposed to be uploaded doesn’t exist locally. We expect the script to error out with an appropriate message:

knife scp 'chef_environment:ft' /Users/mayank/test.py /tmp
/Users/mayank/test.py doesn't exist on local machine

Case2: The /Users/mayank/test folder is:

knife scp 'chef_environment:ft' /Users/mayank/test /tmp
Copying /Users/mayank/test to [email protected]:/tmp
Copying /Users/mayank/test to [email protected]:/tmp

Case3: A config other than /etc/chef/knife.rb is specified:

knife scp -c /Users/mayank/.chef/knife.rb 'chef_environment:ft' /Users/mayank/test /tmp
Copying /Users/mayank/test to [email protected]:/tmp
Copying /Users/mayank/test to [email protected]:/tmp

Distributing plugins using gems

As you must have noticed, until now we’ve been creating our plugins under ~/.chef/plugins/knife. Though this is sufficient for plugins that are meant to be used locally, it’s just not good enough to be distributed to a community. The most ideal way of distributing a Knife plugin is by packaging your plugin as a gem and distributing it via a gem repository such as rubygems.org. Even if publishing your gem to a remote gem repository sounds like a far-fetched idea, at least allowing people to install your plugin by building a gem locally and installing it via gem install. This is a far better way than people downloading your code from an SCM repository and copying it over to either ~/.chef/plugins/knife or any other folder they’ve configured for the purpose of searching for custom Knife plugins. With distributing your plugin using gems, you ensure that the plugin is installed in a consistent way and you can also ensure that all the required libraries are preinstalled before a plugin is ready to be consumed by users.

All the details required to create a gem are contained in a file known as Gemspec, which resides at the root of your project’s directory and is typically named the <project_name>.gemspec. Gemspec file that consists of the structure, dependencies, and metadata required to build your gem.

The following is an example of a .gemspec file:

Gem::Specification.new do |s|
s.name = 'knife-scp'
s.version = '1.0.0'
s.date = '2014-10-23'
s.summary = 'The knife-scp knife plugin'
s.authors = ["maxcoder"]
s.email = '[email protected]"
s.files = ["lib/chef/knife/knife-scp.rb"]
s.homepage = "https://github.com/maxc0d3r/knife-plugins"
s.add_runtime_dependency "parallel","~> 1.2", ">= 1.2.0"
s.add_runtime_dependency "net-scp","~> 1.2", ">= 1.2.0"
end

The s.files variable contains the list of files that will be deployed by a gem install command. Knife can load the files from gem_path/lib/chef/knife/<file_name>.rb, and hence we’ve kept the knife-scp.rb script in that location.

The s.add_runtime_dependency dependency is used to ensure that the required gems are installed whenever a user tries to install our gem.

Once the file is there, we can just run a gem build to build our gem file as follows:

 knife-scp git:(master) x gem build knife-scp.gemspec
WARNING: licenses is empty, but is recommended. Use a license abbreviation from:
http://opensource.org/licenses/alphabetical
WARNING: See http://guides.rubygems.org/specification-reference/ for help
 Successfully built RubyGem
 Name: knife-scp
 Version: 1.0.0
 File: knife-scp-1.0.0.gem

The gem file is created and now, we can just use gem install knife-scp-1.0.0.gem to install our gem. This will also take care of the installation of any dependencies such as parallel, net-scp gems, and so on.

You can find a source code for this plugin at the following location:

https://github.com/maxc0d3r/knife-plugins.

Once the gem has been installed, the user can run it as mentioned earlier.

For the purpose of distribution of this gem, it can either be pushed using a local gem repository, or it can be published to https://rubygems.org/. To publish it to https://rubygems.org/, create an account there.

Run the following command to log in using a gem:

gem push

This will ask for your email address and password.

Next, push your gem using the following command:

gem push your_gem_name.gem

That’s it! Now you should be able to access your gem at the following location:

http://www.rubygems.org/gems/your_gem_name.

As you might have noticed, we’ve not written any tests so far to check the plugin. It’s always a good idea to write test cases before submitting your plugin to the community. It’s useful both to the developer and consumers of the code, as both know that the plugin is going to work as expected. Gems support adding test files into the package itself so that tests can be executed when a gem is downloaded. RSpec is a popular choice to test a framework, however, it really doesn’t matter which tool you use to test your code. The point is that you need to test and ship.

Some popular Knife plugins, built by a community, and their uses, are as follows:

knife-elb:

This plugin allows the automation of the process of addition and deletion of nodes from Elastic Load Balancers on AWS.

knife-inspect:

This plugin allows you to see the difference between what’s on a Chef server versus what’s on a local Chef repository.

knife-community:

This plugin helps to deploy Chef cookbooks to Chef Supermarket.

knife-block:

This plugin allows you to configure and manage multiple Knife configuration files against multiple Chef servers.

knife-tagbulk:

This plugin allows bulk tag operations (creation or deletion) using standard Chef search queries. More information about the plugin can be found at: https://github.com/priestjim/knife-tagbulk.

You can find a lot of other useful community-written plugins at: https://docs.chef.io/community_plugin_knife.html.

Custom Chef handlers

A Chef handler is used to identify different situations that might occur during a chef-client run, and eventually it instructs the chef-client on what it should do to handle these situations. There are three types of handlers in Chef:

• The exception handler: This is used to identify situations that have caused a chef-client run to fail. This can be used to send out alerts over an email or dashboard.
• The report handler: This is used to report back when a chef-client run has successfully completed. This can report details about the run, such as the number of resources updated, time taken for a chef-client run to complete, and so on.
• The start handler: This is used to run events at the beginning of a chef-client run.

Writing custom Chef handlers is nothing more than just inheriting your class from Chef::Handler and overriding the report method.

Let’s say we want to send out an email every time a chef-client run breaks. Chef provides a failed? method to check the status of a chef-client run. The following is a very simple piece of code that will help us accomplish this:

require 'net/smtp'
module CustomHandler
class Emailer < Chef::Handler
   def send_email(to,opts={})
     opts[:server] ||= 'localhost'
     opts[:from] ||='[email protected]'
     opts[:subject] ||='Error'
     opts[:body] ||= 'There was an error running chef-client'
 
     msg = <<EOF
     From: <#{opts[:from]}>
     To: #{to}
     Subject: #{opts[:subject]}
 
     #{opts[:body]}
     EOF
 
     Net::SMTP.start(opts[:server]) do |smtp|
       smtp.send_message msg, opts[:from], to
     end
   end
 
   def report
     name = node.name
     subject = "Chef run failure on #{name}"
     body = [run_status.formatted_exception]
     body += ::Array(backtrace).join("n")
     if failed?
       send_email(
         "[email protected]",
         :subject => subject,
         :body => body
       )
     end
   end
end
end

If you don’t have the required libraries already installed on your machine, you’ll need to make use of chef_gem to install them first before you actually make use of this code.

With your handler code ready, you can make use of the chef_handler cookbook to install this custom handler. To do so, create a new cookbook, email-handler, and copy the file emailer.rb created earlier to the file’s source. Once done, add the following recipe code:

include_recipe 'chef_handler'
 
handler_path = node['chef_handler']['handler_path']
handler = ::File.join handler_path, 'emailer'
 
cookbook_file "#{handler}.rb" do
source "emailer.rb"
end
 
chef_handler "CustomHandler::Emailer" do
source handler
   action :enable
end

Now, just include this handler into your base role, or at the start of run_list and during the next chef-client run, if anything breaks, an email will be sent across to [email protected].

You can configure many different kinds of handlers like the ones that push notifications over to IRC, Twitter, and so on, or you may even write them for scenarios where you don’t want to leave a component of a system in a state that is undesirable. For example, say you were in a middle of a chef-client run that adds/deletes collections from Solr. Now, you might not want to leave the Solr setup in a messed-up state if something were to go wrong with the provisioning process. In order to ensure that a system is in the right state, you can write your own custom handlers, which can be used to handle such situations and revert the changes done until now by the chef-client run.

Summary

In this article, we learned about how custom Knife plugins can be used. We also learned how we can write our own custom Knife plugin and distribute it by packaging it as a gem. Finally, we learned about custom Chef handlers and how they can be used effectively to communicate information and statistics about a chef-client run to users/admins, or handle any issues with a chef-client run.

Resources for Article:

---

Further resources on this subject:

• An Overview of Automation and Advent of Chef [article]
• Testing Your Recipes and Getting Started with ChefSpec [article]
• Chef Infrastructure [article]

---