In my previous two posts, I explored how to use Capistrano to deploy multiple applications in different environments and servers. This, however, is only one part of our deployment procedures. It just takes care of the applications themselves, but we still rely on the server being properly set up so that our Capistrano recipes work. In these two posts I’ll explain how to use Chef to manage servers, and how to integrate it with Capistrano and perform all of your deployment procedures from a single project.
Introducing the sample deployment project
After I wrote the previous two posts, I realized I was not fully happy with a few issues of our company’s deployment strategy:
- Duplicate settings: This was the main issue that was puzzling me. I didn’t like the fact that we had to duplicate some settings like the application’s binding port in both Chef and Capistrano projects.
- Too many required files (45 to support 3 servers, 5 environments, and 3 applications): While the files were really small, I felt that this situation could be further improved by the use of some conventions.
So, I decided to work in a proof-of-concept project that would integrate both Chef and Capistrano and fix these issues. After a weekend working (almost) full time on it, I came up with a sample project so that you can fork it and adapt it to your deployment scenario. The main goal of this project hasn’t changed from my previous article. We want to be able to support new environments and servers very quickly by simply adding some settings to the project.
Go ahead and clone it. Follow the instructions on the README and it should deploy the Rails Devise sample application into a VirtualBox Virtual Machine (VM) using Vagrant. The following sections will explain how it works and the reasons behind its design.
The overall idea
While it’s possible to accomplish all of your deployment tasks with either Chef or Capistrano alone, I feel that they are more suitable for different tasks. There are many existing recipes that you can take advantage of for both projects, but they usually don’t overlap much.
There are Chef community cookbooks available to help you install nginx, apache2, java, databases, and much more.
You probably want to use Chef to perform administrative tasks like managing services, server backup, installing software, and so on.
Capistrano, on the other hand, will help you by deploying the applications itself after the server is ready to go, and after running your Chef recipes. This includes creating releases of your application, which allows you to easily rollback to a previous working version, for example.
You’ll find existing Capistrano recipes to help you with several application-related tasks like running Bundler, switching between Ruby versions with either rbenv, rvm or chruby, running Rails migrations and assets precompilation, and so on.
Capistrano recipes are well integrated with the Capistrano deploy flow. For instance, the capistrano-puma recipe will automatically generate a settings file if it is missing and start puma after the remaining deployment tasks have finished by including this in its recipes:
after 'deploy:check', 'puma:check'
after 'deploy:finished', 'puma:smart_restart'
Another difference between sysadmin and deployment tasks is that usually the former will require superuser privileges while the latter is recommended to be accomplished by a regular user. This way, you can feel safer when deploying Capistrano recipes, since you know it won’t affect the server itself, except for the applications managed by that user account. And deploying an application is way more common than installing and configuring programs or changing the proxy’s settings.
Some of the settings required by Chef and Capistrano recipes overlap. One example is a Chef recipe that generates an nginx settings file that will proxy requests to a Rails application listening on a local port. In this scenario, the binding address used by the Capistrano puma recipe needs to coincide with the port declared in the proxy settings for the nginx configuration file.
Managing deployment settings
Capistrano and Chef provide different built-in ways of managing their settings. Capistrano will use a Domain Specific Language (DSL) like set/fetch, while Chef will read the attributes following a well described precedence.
I strongly advise you to keep with those approaches for settings that are specific for each project. To enable you to remove any duplication by overlapping deployment settings, I introduced another configuration declaration framework for the shared settings using the configatron gem, by taking advantage of the fact that both Chef and Capistrano are written in Ruby.
Take a look at the settings directory in the sample project:
settings/
├── applications
│ └── rails-devise.rb
├── common.rb
├── environments
│ ├── development.rb
│ └── production.rb
└── servers
└── vagrant.rb
The settings are split in common, along with those specific for each application, environment, and servers. As you would expect, the Rails Devise application deployed to the production environment in the vagrant server will read the settings from common.rb, servers/vagrant.rb, environments/production.rb, and applications/rails-devise.rb. If some of your settings apply to the Rails Devise running on a given server or environment (or both), it’s possible to override the specific settings in other files like rails-devise_production.rb, vagrant_production.rb, or vagrant_production_rails-devise.rb. Here’s the definition of load_app_settings in common_helpers/settings_loader.rb:
def load_app_settings(app_name, app_server, app_env)
cfg.app_name = app_name
cfg.app_server = app_server
cfg.app_env = app_env
[
'common',
"servers/#{app_server}",
"environments/#{app_env}",
"applications/#{app_name}",
"#{app_server}_#{app_env}", "#{app_server}_#{app_name}",
"#{app_name}_#{app_env}",
"#{app_server}_#{app_env}_#{app_name}",
].each{|s| load_settings s }
cfg.lock!
end
Feel free to change the load path order. The latest settings take precedence over the first ones. So if the binding port is usually 3000 for production but 4000 for your ec2 server, you can add a cfg.my_app.binding_port = 3000 to environments/production.rb and override it on ec2_production.rb.
Once those settings are loaded, they are locked and can’t be changed by the deployment recipes.
As a final note, the settings can also be set using a hash notation, which can be useful if you’re using a dynamic setting attribute. Here’s an example: cfg[:my_app][“binding_#{‘port’}”] = 3000. This is not really useful in this case, but it illustrates the setting capabilities.
Calculated settings
Two types of calculated settings are supported on this project: delayed and dynamic. Delayed are lazily evaluated the first time they are requested, while dynamic are always evaluated. They are useful for providing default values for some settings that could be overridden by other settings files. I prefer to use delayed attributes for those that are meant to be overridden and dynamic ones for those that are meant to be calculated, even though delayed ones would be suitable for both cases. Here’s the common.rb from the sample project to illustrate the idea:
require 'set'
cfg.chef_runlist = Set.new
cfg.deploy_user = 'deploy'
cfg.deployment_repo_url = '[email protected]:rosenfeld/capistrano-chef-deployment.git'
cfg.deployment_repo_host = 'github.com'
cfg.deployment_repo_symlink = false
cfg.nginx.default = false
# Delayed attributes: they are set to the block values unless explicitly set to other value
cfg.database_name = delayed_attr{ "app1_#{cfg.app_env}" }
cfg.nginx.subdomain = delayed_attr{ cfg.app_env }
# Dynamic/calculated attributes: those are always evaluated by the block
# Those attributes are not meant to be overrideable
cfg.nginx.host = dyn_attr{ "#{cfg.nginx.subdomain}.mydomain.com" }
cfg.nginx.host in this instance is not meant to be overridden by any other settings file and follows the company’s policy. But it would be okay to override the production database name to app1 instead of using the default app1_production.
This is just a guideline, but it should give you a good idea of some ways that Chef and Capistrano can be used together.
Conclusion
I hope you found this post as useful as I did. Being able to fully deploy the whole application stack from a single repository saves us a lot of time and simplifies our deployment a lot, and in the next post, Part 2, I will walk you through that deployment.
About The Author
Rodrigo Rosenfeld Rosas lives in Vitória-ES, Brazil, with his lovely wife and daughter. He graduated in Electrical Engineering with a Master’s degree in Robotics and Real-time Systems. For the past 5 years Rodrigo has focused on building and maintaining single page web applications. He is the author of some gems including active_record_migrations, rails-web-console, the JS specs runner oojspec, sequel-devise, and the Linux X11 utility ktrayshortcut. Rodrigo was hired by e-Core (Porto Alegre – RS, Brazil) to work from home, building and maintaining software for Matterhorn Transactions Inc. with a team of great developers. Matterhorn’s main product, the Market Tracker, is used by LexisNexis clients.
