OpenStack Heat orchestration makes it easy to build templates for application deployment and auto-scaling. The built-in resource types offer access to many of the existing OpenStack services. However, you may need to integrate with an internal CMDB or service registry, or configure some other services outside of OpenStack, as you launch your application. In this post I will explain how you can add your own custom Heat resources to extend Heat orchestration to meet your needs.
As example code, I’ll use the Heat resources we developed at Infoblox, which can be found at http://github.com/infobloxopen/heat-infoblox. In our use case, we have an existing management interface for our DNS services, called the grid. In order to scale up the DNS service, we need to orchestrate the addition of members to our grid by making RESTful APIs calls to the grid master. We built custom Heat resource types to set up the grid to properly configure the new member to serve DNS.
These custom resources perform the following operations:
- Tell the grid master about the new member that will be joining.
- Configure the networking for one or more interfaces on the member.
- Configure the licenses for each member.
- Enable the DNS service on the new member.
- Configure the “name server group” for the member – that is, configure which zones the member will serve.
With these resources, we can scale up the DNS service for particular sets of domains with a simple Heat CLI command, or even auto-scale based upon the load seen on the instances. We use two different resource types for this, with the Infoblox::Grid::Member handling 1-4, and Infoblox::Grid::NameServerGroupMember handling 5.
So, what do we need to do to build a Heat resource? First, we need to understand the main features of a resource. From a developer standpoint, each resource consists of a property schema, an attribute schema, a set of lifecycle methods, and a resource identifier. It is important to think about whatever actions you need to take in terms of a resource that can be created, updated, or deleted. That is, the way Heat works is to manage resources; sometimes configuration doesn’t fit neatly into that concept, but you’ll need to find a way to define resources that make sense even so.
Properties are the inputs to the resource creation and update processes, and are specified by the user in the template when utilizing the resource. Attributes, on the other hand, are the run-time data values associated with an existing resource. For example, the Infoblox::Grid::Member resource type, defined in the heat_infoblox/resources/grid_member.py file, has properties such as name and port, but its attributes include the user data to inject during Nova boot. That user data is actually generated on the fly by the resource when it is needed.
The lifecycle methods are called by Heat to create, update, delete, or validate the resource. This is where all the critical logic resides. The resource identifier is generated by the create method, and is used as the input for the delete method or other methods that operate on an existing resource. Thus, it is critical that the resource id value provides a unique reference to the resource.
When building a new resource type, the first thing to do is understand what the critical properties are that the user will need to set. These are defined in the properties_schema (this snippet is from Infoblox::Grid::Member code in the stable/juno branch of the heat-infoblox project; there are some small differences in more recent versions of Heat):
properties_schema = {
NAME: properties.Schema(
properties.Schema.STRING,
_('Member name.'),
),
MODEL: properties.Schema(
properties.Schema.STRING,
_('Infoblox model name.'),
constraints=[
constraints.AllowedValues(ALLOWED_MODELS)
]
),
LICENSES: properties.Schema(
properties.Schema.LIST,
_('List of licenses to pre-provision.'),
schema=properties.Schema(
properties.Schema.STRING
),
constraints=[
constraints.AllowedValues(ALLOWED_LICENSES_PRE_PROVISION)
]
),
…etc…
Each property in turn has its own schema that describes its type, any constraints on the input values, whether the property is required or optional, and a default value if appropriate. In many cases, the property itself may be another dictionary with many different additional options that each in turn have their own schema. Each property or sub-property should also include a clear description. These descriptions are shown to the user in the newer versions of Horizon, and are critical to making the resource type useful.
Next, you’ll need to understand what attributes are needed, if any. Attributes aren’t always necessary, but may be needed if the new resource is to be consumed as input to subsequent resources. For example, the Infoblox::Grid::Member resource has a user_data attribute, which is fed into the OS::Nova::Server user_data property when spinning up the Nova instance for the new member. Like properties, attributes are specified with a schema:
attributes_schema = {
USER_DATA: attributes.Schema(
_('User data for the Nova boot process.')),
NAME_ATTR: attributes.Schema(
_('The member name.'))
}
In this case, however, the schema is simpler. Since it is essentially just documenting the outputs for use by template authors, there is no need to specify constraints, defaults, or even data types. Like the properties example, the code snippet above is from the Juno version of Heat-Infoblox. The newer version allows you to specify a type, though it is still not required.
Finally, you need to specify the lifecycle methods. The handle_create and handle_delete methods are critical and must be implemented. There are a number of other handler methods that can be optionally implemented: handle_update, handle_suspend, and handle_resume are the most commonly implemented. If one of these operations happens asynchronously (such as launching a Nova instance), then you can utilize the handle_<action>_complete method, which will be repeatedly called in a loop until it returns True, after the handle_<action> method is called.
Let’s take a closer look at the handle_create method defined by Infoblox::Grid::Member. Here is the complete code of this method:
def handle_create(self):
mgmt = self._make_port_network_settings(self.MGMT_PORT)
lan1 = self._make_port_network_settings(self.LAN1_PORT)
lan2 = self._make_port_network_settings(self.LAN2_PORT)
name = self.properties[self.NAME]
nat = self.properties[self.NAT_IP]
self.infoblox().create_member(name=name, mgmt=mgmt, lan1=lan1,
lan2=lan2, nat_ip=nat)
self.infoblox().pre_provision_member(
name,
hwmodel=self.properties[self.MODEL], hwtype='IB-VNIOS',
licenses=self.properties[self.LICENSES])
dns = self.properties[self.DNS_SETTINGS]
if dns:
self.infoblox().configure_member_dns(
name,
enable_dns=dns['enable']
)
self.resource_id_set(name)
Breaking this down, we see the first thing it does is convert the properties into a format understood by the Infoblox RESTful API:
mgmt = self._make_port_network_settings(self.MGMT_PORT)
lan1 = self._make_port_network_settings(self.LAN1_PORT)
lan2 = self._make_port_network_settings(self.LAN2_PORT)
The _make_port_network_settings here will actually call out to the Neutron API to gather details about the port, and return a JSON structure that represents the configuration of those ports.
def _make_port_network_settings(self, port_name):
if self.properties[port_name] is None:
return None
port = self.client('neutron').show_port(
self.properties[port_name])['port']
if port is None:
return None
ipv4 = None
ipv6 = None
for ip in port['fixed_ips']:
if ':' in ip['ip_address'] and ipv6 is None:
ipv6 = self._make_ipv6_settings(ip)
else:
if ipv4 is None:
ipv4 = self._make_network_settings(ip)
return {'ipv4': ipv4, 'ipv6': ipv6}
After that, it calls the methods that interface with the Infoblox API, passing in the properly formatted data that was created based upon the resource properties:
name = self.properties[self.NAME]
nat = self.properties[self.NAT_IP]
self.infoblox().create_member(name=name, mgmt=mgmt, lan1=lan1,
lan2=lan2, nat_ip=nat)
self.infoblox().pre_provision_member(
name,
hwmodel=self.properties[self.MODEL], hwtype='IB-VNIOS',
licenses=self.properties[self.LICENSES])
dns = self.properties[self.DNS_SETTINGS]
if dns:
self.infoblox().configure_member_dns(
name,
enable_dns=dns['enable']
)
Finally, it must set the resource_id value for the resource. This must be unique to the type of resource, so that the handle_delete method will know the appropriate resource to act upon. In our case, the name is sufficient, so we use that:
self.resource_id_set(name)Once a resource is created, the template may want to access the attributes we defined for that resource. To make these accessible, you just need to override the _resolve_attribute method, which takes the name of the attribute to resolve.
def _resolve_attribute(self, name):
member_name = self.resource_id
member = self.infoblox().get_member(
member_name,
return_fields=['host_name', 'vip_setting', 'ipv6_setting'])[0]
token = self._get_member_tokens(member)
LOG.debug("MEMBER for %s = %s" % (name, member))
if name == self.USER_DATA:
return self._make_user_data(member, token)
if name == self.NAME_ATTR:
return member['host_name']
return None
This is called an instance method, so the resource_id is available in the object itself. In our case, we call the Infoblox RESTful API to query for the details about the member referenced in the resource_id, then use that data to generate the attribute requested.
That is really all there is to a Heat resource. The short version is: define the resource ID, attributes, and properties then use the properties in the RESTful API calls in the handle_* and _resolve_attribute methods, to manage your custom resource.
From 4th to 10th April save 50% on some of our top cloud eBooks. From OpenStack to AWS, we’ve got a range of titles to help you unlock cloud’s transformative potential! Find them all here.
About the author
John Belamaric is a software and systems architect with nearly 20 years of software design and development experience. His current focus is on cloud network automation. He is a key architect of the Infoblox Cloud products, concentrating on OpenStack integration and development. He brings to this his experience as the lead architect for the Infoblox Network Automation product line, along with a wealth of networking, network management, software, and product design knowledge. He is a contributor to both the OpenStack Neutron and Designate projects. He lives in Bethesda, Maryland with his wife Robin and two children, Owen and Audrey.
