In this article by Walter Bentley, the author of the book OpenStack Administration with Ansible 2 – Second Edition. This article will serve as a high-level overview of Ansible 2.0 and components that make up this open source configuration management tool. We will cover the definition of the Ansible components and their typical use. Also, we will discuss how to define variables for the roles and defining/setting facts about the hosts for the playbooks. Next, we will transition into how to set up your Ansible environment and the ways you can define the host inventory used to run your playbooks against. We will then cover some of the new components introduced in Ansible 2.0 named Blocks and Strategies. It will also review the cloud integrations natively part of the Ansible framework. Finally, the article will finish up with a working example of a playbook that will confirm the required host connectivity needed to use Ansible. The following topics are covered:
- Ansible 2.0 overview
- What are playbooks, roles, and modules?
- Setting up the environment
- Variables and facts
- Defining the inventory
- Blocks and Strategies
- Cloud integrations
(For more resources related to this topic, see here.)
Ansible 2.0 overview
Ansible in its simplest form has been described as a python-based open source IT automation tool that can be used to configuremanage systems, deploy software (or almost anything), and provide orchestration to a process. These are just a few of the many possible use cases for Ansible. In my previous life as a production support infrastructure engineer, I wish such a tool would have existed. Would have surely had much more sleep and a lot less gray hairs.
One thing that always stood out to me in regard to Ansible is that the developer’s first and foremost goal was to create a tool that offers simplicity and maximum ease of use. In the world filled with complicated and intricate software, keeping it simple goes a long way for most IT professionals.
Staying with the goal of keeping things simple, Ansible handles configuration/management of hosts solely through Secure Shell (SSH). Absolutely no daemon or agent is required. The server or workstation where you run the playbooks from only needs python and a few other packages, most likely already present, installed. Honestly, it does not get simpler than that.
The automation code used with Ansible is organized into something named playbooks and roles, of which is written in YAML markup format. Ansible follows the YAML formatting and structure within the playbooks/roles. Being familiar with YAML formatting helps in creating your playbooks/roles. If you are not familiar do not worry, as it is very easy to pick up (it is all about the spaces and dashes).
The playbooks and roles are in a noncomplied format making the code very simple to read if familiar with standard UnixLinux commands. There is also a suggested directory structure in order to create playbooks. This also is one of my favorite features of Ansible. Enabling the ability to review and/or use playbooks written by anyone else with little to no direction needed.
It is strongly suggested that you review the Ansible playbook best practices before getting started: http://docs.ansible.com/playbooks_best_practices.html. I also find the overall Ansible website very intuitive and filled with great examples at http://docs.ansible.com.
My favorite excerpt from the Ansible playbook best practices is under the Content Organization section. Having a clear understanding of how to organize your automation code proved very helpful to me. The suggested directory layout for playbooks is as follows:
group_vars/
group1 # here we assign variables to particular groups
group2 # ""
host_vars/
hostname1 # if systems need specific variables, put them here
hostname2 # ""
library/ # if any custom modules, put them here (optional)
filter_plugins/ # if any custom filter plugins, put them here (optional)
site.yml # master playbook
webservers.yml # playbook for webserver tier
dbservers.yml # playbook for dbserver tier
roles/
common/ # this hierarchy represents a "role"
tasks/ #
main.yml # <-- tasks file can include smaller files if warranted
handlers/ #
main.yml # <-- handlers file
templates/ # <-- files for use with the template resource
ntp.conf.j2 # <------- templates end in .j2
files/ #
bar.txt # <-- files for use with the copy resource
foo.sh # <-- script files for use with the script resource
vars/ #
main.yml # <-- variables associated with this role
defaults/ #
main.yml # <-- default lower priority variables for this role
meta/ #
main.yml # <-- role dependencies
It is now time to dig deeper into reviewing what playbooks, roles, and modules consist of. This is where we will break down each of these component’s distinct purposes.
What are playbooks, roles, and modules?
The automation code you will create to be run by Ansible is broken down in hierarchical layers. Envision a pyramid with its multiple levels of elevation. We will start at the top and discuss playbooks first.
Playbooks
Imagine that a playbook is the very topmost triangle of the pyramid. A playbook takes on the role of executing all of the lower level code contained in a role. It can also be seen as a wrapper to the roles created. We will cover the roles in the next section.
Playbooks also contain other high-level runtime parameters, such as the host(s) to run the playbook against, the root user to use, and/or if the playbook needs to be run as a sudo user. These are just a few of the many playbook parameters you can add. Below is an example of what the syntax of a playbook looks like:
---
# Sample playbooks structure/syntax.
- hosts: dbservers
remote_user: root
become: true
roles:
- mysql-install
In the preceding example, you will note that the playbook begins with —. This is required as the heading (line 1) for each playbook and role.
Also, please note the spacing structure at the beginning of each line. The easiest way to remember it is each main command starts with a dash (–). Then, every subcommand starts with two spaces and repeats the lower in the code hierarchy you go. As we walk through more examples, it will start to make more sense.
Let’s step through the preceding example and break down the sections. The first step in the playbook was to define what hosts to run the playbook against; in this case, it was dbservers (which can be a single host or list of hosts). The next area sets the user to run the playbook as locally, remotely, and it enables executing the playbook as sudo. The last section of the syntax lists the roles to be executed.
The earlier example is similar to the formatting of the other playbooks. This format incorporates defining roles, which allows for scaling out playbooks and reusability (you will find the most advanced playbooks structured this way). With Ansible’s high level of flexibility, you can also create playbooks in a simpler consolidated format. An example of such kind is as follows:
---
# Sample simple playbooks structure/syntax
- name: Install MySQL Playbook
hosts: dbservers
remote_user: root
become: true
tasks:
- name: Install MySQL
apt: name={{item}} state=present
with_items:
- libselinux-python
- mysql
- mysql-server
- MySQL-python
- name: Copying my.cnf configuration file
template: src=cust_my.cnf dest=/etc/my.cnf mode=0755
- name: Prep MySQL db
command: chdir=/usr/bin mysql_install_db
- name: Enable MySQL to be started at boot
service: name=mysqld enabled=yes state=restarted
- name: Prep MySQL db
command: chdir=/usr/bin mysqladmin -u root password 'passwd'
Now that we have reviewed what playbooks are, we will move on to reviewing roles and their benefits.
Roles
Moving down to the next level of the Ansible pyramid, we will discuss roles. The most effective way to describe roles is the breaking up a playbook into multiple smaller files. So, instead of having one long playbook with multiple tasks defined, all handling separately related steps, you can break the playbook into individual specific roles. This format keeps your playbooks simple and leads to the ability to reuse roles between playbooks.
The best advice I personally received concerning creating roles is to keep them simple. Try to create a role to do a specific function, such as just installing a software package. You can then create a second role to just do configurations. In this format, you can reuse the initial installation role over and over without needing to make code changes for the next project.
The typical syntax of a role can be found here and would be placed into a file named main.yml within the roles/<name of role>/tasks directory:
---
- name: Install MySQL
apt: name={{item}} state=present
with_items:
- libselinux-python
- mysql
- mysql-server
- MySQL-python
- name: Copying my.cnf configuration file
template: src=cust_my.cnf dest=/etc/my.cnf mode=0755
- name: Prep MySQL db
command: chdir=/usr/bin mysql_install_db
- name: Enable MySQL to be started at boot
service: name=mysqld enabled=yes state=restarted
- name: Prep MySQL db
command: chdir=/usr/bin mysqladmin -u root password 'passwd'
The complete structure of a role is identified in the directory layout found in the Ansible Overview section of this article. We will review additional functions of roles as we step through the working examples. With having covered playbooks and roles, we are prepared to cover the last topic in this session, which are modules.
Modules
Another key feature of Ansible is that it comes with predefined code that can control system functions, named modules. The modules are executed directly against the remote host(s) or via playbooks. The execution of a module generally requires you to pass a set number of arguments. The Ansible website (http://docs.ansible.com/modules_by_category.html) does a great job of documenting every available module and the possible arguments to pass to that module.
The documentation for each module can also be accessed via the command line by executing the command ansible-doc <module name>.
The use of modules will always be the recommended approach within Ansible as they are written to avoid making the requested change to the host unless the change needs to be made. This is very useful when re-executing a playbook against a host more than once. The modules are smart enough to know not to re-execute any steps that have already completed successfully, unless some argument or command is changed.
Another thing worth noting is with every new release of Ansible additional modules are introduced. Personally, there was an exciting addition to Ansible 2.0, and these are the updated and extended set of modules set to ease the management of your OpenStack cloud.
Referring back to the previous role example shared earlier, you will note the use of various modules. The modules used are highlighted here again to provide further clarity:
---
- name: Install MySQL
apt: name={{item}} state=present
with_items:
- libselinux-python
- mysql
- mysql-server
- MySQL-python
- name: Copying my.cnf configuration file
template: src=cust_my.cnf dest=/etc/my.cnf mode=0755
- name: Prep MySQL db
command: chdir=/usr/bin mysql_install_db
- name: Enable MySQL to be started at boot
service: name=mysqld enabled=yes state=restarted
...
Another feature worth mentioning is that you are able to not only use the current modules, but you can also write your very own modules. Although the core of Ansible is written in python, your modules can be written in almost any language. Underneath it, all the modules technically return JSON format data, thus allowing for the language flexibility.
In this section, we were able to cover the top two sections of our Ansible pyramid, playbooks, and roles. We also reviewed the use of modules, that is, the built-in power behind Ansible. Next, we transition into another key features of Ansible—variable substitution and gathering host facts.
Setting up the environment
Before you can start experimenting with Ansible, you must install it first. There was no need in duplicating all the great documentation to accomplish this already created on http://docs.ansible.com/ . I would encourage you to go to the following URL and choose an install method of your choice: http://docs.ansible.com/ansible/intro_installation.html.
If you are installing Ansible on Mac OS, I found using Homebrew was much simpler and consistent. More details on using Homebrew can be found at http://brew.sh.
The command to install Ansible with Homebrew is brew install ansible.
Upgrading to Ansible 2.0
It is very important to note that in order to use the new features part of Ansible version 2.0, you must update the version running on your OSA deployment node. The version currently running on the deployment node is either 1.9.4 or 1.9.5. The method that seemed to work well every time is outlined here. This part is a bit experimental, so please make a note of any warnings or errors incurred.
From the deployment node, execute the following commands:
$ pip uninstall -y ansible
$ sed -i 's/^export ANSIBLE_GIT_RELEASE.*/export ANSIBLE_GIT_RELEASE=${ANSIBLE_GIT_RELEASE:-"v2.1.1.0-1"}/' /opt/openstack-ansible/scripts/bootstrap-ansible.sh
$ cd /opt/openstack-ansible
$ ./scripts/bootstrap-ansible.sh
New OpenStack client authenticate
Alongside of the introduction of the new python-openstackclient, CLI was the unveiling of the os-client-config library. This library offers an additional way to provide/configure authentication credentials for your cloud. The new OpenStack modules part of Ansible 2.0 leverages this new library through a package named shade. Through the use of os-client-config and shade, you can now manage multiple cloud credentials within a single file named clouds.yml. When deploying OSA, I discovered that shade will search for this file in the $HOME/.config/openstack/ directory wherever the playbook/role and CLI command is executed. A working example of the clouds.yml file is shown as follows:
# Ansible managed: /etc/ansible/roles/openstack_openrc/templates/clouds.yaml.j2 modified on 2016-06-16 14:00:03 by root on 082108-allinone02
clouds:
default:
auth:
auth_url: http://172.29.238.2:5000/v3
project_name: admin
tenant_name: admin
username: admin
password: passwd
user_domain_name: Default
project_domain_name: Default
region_name: RegionOne
interface: internal
identity_api_version: "3"
Using this new authentication method drastically simplifies creating automation code to work on an OpenStack environment. Instead of passing a series of authentication parameters in line with the command, you can just pass a single parameter, –os-cloud=default. The Ansible OpenStack modules can also use this new authentication method. More details about os-client-config can be found at: http://docs.openstack.org/developer/os-client-config.
Installing shade is required to use the Ansible OpenStack modules in version 2.0. Shade will be required to be installed directly on the deployment node and the Utility container (if you decide to use this option). If you encounter problems installing shade, try the command—pip install shade—isolated.
Variables and facts
Anyone who has ever attempted to create some sort of automation code, whether be via bash or Perl scripts, knows that being able to define variables is an essential component. Although Ansible does not compare with other programming languages mentioned, it does contain some core programming language features such as variable substitution.
Variables
To start, let’s first define the meaning of variables and use in the event this is a new concept.
Variable (computer science), a symbolic name associated with a value and whose associated value may be changed
Using variable allows you to set a symbolic placeholder in your automation code that you can substitute values for on each execution. Ansible accommodates defining variables within your playbooks and roles in various ways. When dealing with OpenStack and/or cloud technologies in general being able to adjust your execution parameters on the fly is critical.
We will step through a few ways how you can set variable placeholders in your playbooks, how to define variable values, and how you can register the result of a task as a variable.
Setting variable placeholders
In the event you wanted to set a variable placeholder within your playbooks, you would add the following syntax like this:
- name: Copying my.cnf configuration file
template: src=cust_my.cnf dest={{ CONFIG_LOC }} mode=0755
In the preceding example, the variable CONFIG_LOC was added in the place of the configuration file location (/etc/my.cnf) designated in the earlier example. When setting the placeholder, the variable name must be encased within {{ }} as shown in the example.
Defining variable values
Now that you have added the variable to your playbook, you must define the variable value. This can be done easily by passing command-line values as follows:
$ ansible-playbook base.yml --extra-vars "CONFIG_LOC=/etc/my.cnf"Or you can define the values directly in your playbook, within each role or include them inside of global playbook variable files. Here are the examples of the three options.
Define a variable value directly in your playbook by adding the vars section:
---
# Sample simple playbooks structure/syntax
- name: Install MySQL Playbook
hosts: dbservers
...
vars:
CONFIG_LOC: /etc/my.cnf
...
Define a variable value within each role by creating a variable file named main.yml within the vars/ directory of the role with the following contents:
---
CONFIG_LOC: /etc/my.cnf
To define the variable value inside of the global playbook, you would first create a host-specific variable file within the group_vars/ directory in the root of the playbook directory with the exact same contents as mentioned earlier. In this case, the variable file must be named to match the host or host group name defined within the hosts file.
As in the earlier example, the host group name is dbservers; in turn, a file named dbservers would be created within the group_vars/ directory.
Registering variables
The situation at times arises when you want to capture the output of a task. Within the process of capturing the result you are in essence registering a dynamic variable. This type of variable is slightly different from the standard variables we have covered so far.
Here is an example of registering the result of a task to a variable:
- name: Check Keystone process
shell: ps -ef | grep keystone
register: keystone_check
The registered variable value data structure can be stored in a few formats. It will always follow a base JSON format, but the value can be stored under different attributes. Personally, I have found it difficult at times to blindly determine the format, The tip given here will save you hours of troubleshooting.
To review and have the data structure of a registered variable returned when running a playbook, you can use the debug module, such as adding this to the previous example: – debug: var=keystone_check.
Facts
When Ansible runs a playbook, one of the first things it does on your behalf is gather facts about the host before executing tasks or roles. The information gathered about the host will range from the base information such as operating system and IP addresses to the detailed information such as the hardware type/resources. The details capture on then stored into a variable named facts.
You can find a complete list of available facts on the Ansible website at: http://docs.ansible.com/playbooks_variables.html#information-discovered-from-systems-facts.
You have the option to disable the facts gather process by adding the following to your playbook: gather_facts: false. Facts about a host are captured by default unless the feature is disabled.
A quick way of viewing all facts associated with a host, you can manually execute the following via a command line:
$ ansible dbservers –m setupThere is plenty more you can do with facts, and I would encourage you to take some time reviewing them in the Ansible documentation. Next, we will learn more about the base of our pyramid, the host inventory. Without an inventory of hosts to run the playbooks against, you would be creating the automation code for nothing.
So to close out this artticle, we will dig deeper into how Ansible handles host inventory whether it be in a static and/or dynamic format.
Defining the inventory
The process of defining a collection of hosts to Ansible is named the inventory. A host can be defined using its fully qualified domain name (FQDN), local hostname, and/or its IP address. Since Ansible uses SSH to connect to the hosts, you can provide any alias for the host that the machine where Ansible is installed can understand.
Ansible expects the inventory file to be in an INI-like format and named hosts. By default, the inventory file is usually located in the /etc/ansible directory and will look as follows:
athena.example.com
[ocean]
aegaeon.example.com
ceto.example.com
[air]
aeolus.example.com
zeus.example.com
apollo.example.com
Personally I have found the default inventory file to be located in different places depending on the operating system Ansible is installed on. With that point, I prefer to use the –i command-line option when executing a playbook. This allows me to designate the specific hosts file location. A working example would look like this: ansible-playbook -i hosts base.yml.
In the preceding example, there is a single host and a group of hosts defined. The hosts are grouped together into a group by defining a group name enclosed in [ ] inside the inventory file. Two groups are defined in the earlier-mentioned example—ocean and air.
In the event where you do not have any hosts within your inventory file (such as in the case of running a playbook locally only), you can add the following entry to define localhost like this:
[localhost]
localhost ansible_connection=local
The option exists to define variable for hosts and a group inside of your inventory file. More information on how to do this and additional inventory details can be found on the Ansible website at http://docs.ansible.com/intro_inventory.html.
Dynamic inventory
It seemed appropriate since we are automating functions on a cloud platform to review yet another great feature of Ansible, which is the ability to dynamically capture an inventory of hosts/instances. One of the primary principles of cloud is to be able to create instances on demand directly via an API, GUI, CLI, and/or through automation code, like Ansible. That basic principle will make relying on a static inventory file pretty much a useless choice. This is why, you will need to rely heavily on dynamic inventory.
A dynamic inventory script can be created to pull information from your cloud at runtime and then, in turn, use that information for the playbooks execution. Ansible provides the functionality to detect if an inventory file is set as an executable and if so will execute the script to pull current time inventory data.
Since creating an Ansible dynamic inventory script is considered more of an advanced activity, I am going to direct you to the Ansible website, (http://docs.ansible.com/intro_dynamic_inventory.html), as they have a few working examples of dynamic inventory scripts there.
Fortunately, in our case, we will be reviewing an OpenStack cloud built using openstack-ansible (OSA) repository. OSA comes with a prebuilt dynamic inventory script that will work for your OpenStack cloud. That script is named dynamic_inventory.py and can be found within the playbooks/inventory directory located in the root OSA deployment folder.
First, execute the dynamic inventory script manually to become familiar with the data structure and group names defined (this example assumes that you are in the root OSA deployment directory):
$ cd playbooks/inventory
$ ./dynamic_inventory.py
This will print to the screen an output similar to this:
...
},
"compute_all": {
"hosts": [
"compute1_rsyslog_container-19482f86",
"compute1",
"compute2_rsyslog_container-dee00ea5",
"compute2"
]
},
"utility_container": {
"hosts": [
"infra1_utility_container-c5589031"
]
},
"nova_spice_console": {
"hosts": [
"infra1_nova_spice_console_container-dd12200f"
],
"children": []
},
...
Next, with this information, you now know that if you wanted to run a playbook against the utility container, all you would have to do is execute the playbook like this:
$ ansible-playbook -i inventory/dynamic_inventory.py playbooks/base.yml –l utility_containerBlocks & Strategies
In this section, we will cover two new features added to version 2.0 of Ansible. Both features add additional functionality to how tasks are grouped or executed within a playbook. So far, they seem to be really nice features when creating more complex automation code. We will now briefly review each of the two new features.
Blocks
The Block feature can simply be explained as a way of logically grouping tasks together with the option of also applying customized error handling. It gives the option to group a set of tasks together establishing specific conditions and privileges. An example of applying the block functionality to an earlier example can be found here:
---
# Sample simple playbooks structure/syntax
- name: Install MySQL Playbook
hosts: dbservers
tasks:
- block:
- apt: name={{item}} state=present
with_items:
- libselinux-python
- mysql
- mysql-server
- MySQL-python
- template: src=cust_my.cnf dest=/etc/my.cnf mode=0755
- command: chdir=/usr/bin mysql_install_db
- service: name=mysqld enabled=yes state=restarted
- command: chdir=/usr/bin mysqladmin -u root password 'passwd'
when: ansible_distribution == 'Ubuntu'
remote_user: root
become: true
Additional details on how to implement Blocks and any associated error handling can be found at http://docs.ansible.com/ansible/playbooks_blocks.html.
Strategies
The strategy feature allows you to add control on how a play is executed by the hosts. Currently, the default behavior is described as being the linear strategy, where all hosts will execute each task before any host moves on to the next task. As of today, the two other strategy types that exist are free and debug. Since Strategies are implemented as a new type of plugin to Ansible more can be easily added by contributing code. Additional details on Strategies can be found at http://docs.ansible.com/ansible/playbooks_strategies.html.
A simple example of implementing a strategy within a playbook is as follows:
---
# Sample simple playbooks structure/syntax
- name: Install MySQL Playbook
hosts: dbservers
strategy: free
tasks:
...
The new debug strategy is extremely helpful when you need to step through your playbook/role to find something like a missing variable, determine what variable value to supply or figure out why it may be sporadically failing. These are just a few of the possible use cases. Definitely I encourage you to give this feature a try. Here is the URL to more details on the playbook debugger: http://docs.ansible.com/ansible/playbooks_debugger.html.
Cloud integrations
Since cloud automation is the main and most important theme of this article, it only makes sense that we highlight the many different cloud integrations Ansible 2.0 offers right out of the box. Again, this was one of the reasons why I immediately fell in love with Ansible. Yes, the other automation tools also have hooks into many of the cloud providers, but I found at times they did not work or were not mature enough to leverage. Ansible has gone above and beyond to not fall into that trap. Not saying Ansible has all the bases covered, it does feel like most are and that is what matters most to me.
If you have not checked out the cloud modules available for Ansible, take a moment now and take a look at http://docs.ansible.com/ansible/list_of_cloud_modules.html. From time to time check back as I am confident, you will be surprised to find more have been added. I am very proud of my Ansible family for keeping on top of these and making it much easier to write automation code against our clouds.
Specific to OpenStack, a bunch of new modules have been added to the Ansible library as of version 2.0. The extensive list can be found at http://docs.ansible.com/ansible/list_of_cloud_modules.html#openstack. You will note that the biggest changes, from the first version of this book to this one, will be focused on using as many of the new OpenStack modules when possible.
Summary
Let’s pause here on exploring the dynamic inventory script capabilities and continue to build upon it as we dissect the working examples.
We will create our very first OpenStack administration playbook together. We will start off with a fairly simple task of creating users and tenants. This will also include reviewing a few automation considerations you will need to keep in mind when creating automation code for OpenStack. Ready? Ok, let’s get started!
Resources for Article:
Further resources on this subject:
- AIO setup of OpenStack – preparing the infrastructure code environment [article]
- RDO Installation [article]
- Creating Multiple Users/Tenants [article]