33 min read

This article, by Mani Tadayon, author of the book, RSpec Essentials, discusses support code to set tests up and clean up after them. Initialization, configuration, cleanup, and other support code related to RSpec specs are important in real-world RSpec usage. We will learn how to cleanly organize support code in real-world applications by learning about the following topics:

  • Configuring RSpec with spec_helper.rb
  • Initialization and configuration of resources
  • Preventing tests from accessing the Internet with WebMock
  • Maintaining clean test state
  • Custom helper code
  • Loading support code on demand with tags

(For more resources related to this topic, see here.)

Configuring RSpec with spec_helper.rb

The RSpec specs that we’ve seen so far have functioned as standalone units. Specs in the real world, however, almost never work without supporting code to prepare the test environment before tests are run and ensure it is cleaned up afterwards. In fact, the first line of nearly every real-world RSpec spec file loads a file that takes care of initialization, configuration, and cleanup:

require 'spec_helper'

By convention, the entry point for all support code for specs is in a file called spec_helper.rb. Another convention is that specs are located in a folder called spec in the root folder of the project. The spec_helper.rb file is located in the root of this spec folder.

Now that we know where it goes, what do we actually put in spec_helper.rb? Let’s start with an example:

# spec/spec_helper.rb

require 'rspec'

 

RSpec.configure do |config|

  config.order            = 'random'

  config.profile_examples = 3

   end

To see what these two options do, let’s create a couple of dummy spec files that include our spec_helper.rb. Here’s the first spec file:

# spec/first_spec.rb

require 'spec_helper'

 

describe 'first spec' do

  it 'sleeps for 1 second' do

    sleep 1

  end

 

  it 'sleeps for 2 seconds' do

    sleep 2

  end 

 

  it 'sleeps for 3 seconds' do

    sleep 3

  end 

end

And here’s our second spec file:

# spec/second_spec.rb

require 'spec_helper'

 

describe 'second spec' do

  it 'sleeps for 4 second' do

    sleep 4

  end

 

  it 'sleeps for 5 seconds' do

    sleep 5

  end 

 

  it 'sleeps for 6 seconds' do

    sleep 6

  end 

end

Now let’s run our two spec files and see what happens:

We note that we used –format documentation when running RSpec so that we see the order in which the tests were run (the default format just outputs a green dot for each passing test). From the output, we can see that the tests were run in a random order. We can also see the three slowest specs.

Although this was a toy example, I would recommend using both of these configuration options for RSpec. Running examples in a random order is very important, as it is the only reliable way of detecting bad tests which sometimes pass and sometimes fail based on the order the in which overall test suite is run. Also, keeping tests running fast is very important for maintaining a productive development flow, and seeing which tests are slow on every test run is the most effective way of encouraging developers to make the slow tests fast, or remove them from the test run.

We’ll return to both test order and test speed later. For now, let us just note that RSpec configuration is very important to keeping our specs reliable and fast.

Initialization and configuration of resources

Real-world applications rely on resources, such as databases, and external services, such as HTTP APIs. These must be initialized and configured for the application to work properly. When writing tests, dealing with these resources and services can be a challenge because of two opposing fundamental interests.

First, we would like the test environment to match as closely as possible the production environment so that tests that interact with resources and services are realistic. For example, we may use a powerful database system in production that runs on many servers to provide the best performance. Should we spend money and effort to create and maintain a second production-grade database environment just for testing purposes?

Second, we would like the test environment to be simple and relatively easy to understand, so that we understand what we are actually testing. We would also like to keep our code modular so that components can be tested in isolation, or in simpler environments that are easier to create, maintain, and understand. If we think of the example of the system that relies on a database cluster in production, we may ask ourselves whether we are better off using a single-server setup for our test database. We could even go so far as to use an entirely different database for our tests, such as the file-based SQLite.

As always, there are no easy answers to such trade-offs. The important thing is to understand the costs and benefits, and adjust where we are on the continuum between production faithfulness and test simplicity as our system evolves, along with the goals it serves. For example, for a small hobbyist application or a project with a limited budget, we may choose to completely favor test simplicity. As the same code grows to become a successful fan site or a big-budget project, we may have a much lower tolerance for failure, and have both the motivation and resources to shift towards production faithfulness for our test environment.

Some rules of thumb to keep in mind:

  • Unit tests are better places for test simplicity
  • Integration tests are better places for production faithfulness
  • Try to cleverly increase production faithfulness in unit tests
  • Try to cleverly increase test simplicity in integration tests
  • In between unit and integration tests, be clear what is and isn’t faithful to the production environment

A case study of test simplicity with an external service

Let’s put these ideas into practice. I haven’t changed the application code, except to rename the module OldWeatherQuery. The test code is also slightly changed to require a spec_helper file and to use a subject block to define an alias for the module name, which makes it easier to rename the code without having to change many lines of test code.

So let’s look at our three files now. First, here’s the application code:

# old_weather_query.rb

 

require 'net/http'

require 'json'

require 'timeout'

 

module OldWeatherQuery

  extend self

 

  class NetworkError < StandardError

  end

 

  def forecast(place, use_cache=true)

    add_to_history(place)

 

    if use_cache

      cache[place] ||= begin

        @api_request_count += 1

        JSON.parse( http(place) )

      end

    else

      JSON.parse( http(place) )

    end

  rescue JSON::ParserError

    raise NetworkError.new("Bad response")

  end

 

  def api_request_count

    @api_request_count ||= 0

  end

 

  def history

    (@history || []).dup

  end

 

  def clear!

    @history           = []

    @cache             = {}

    @api_request_count = 0

  end

 

  private

 

  def add_to_history(s)

    @history ||= []

    @history << s

  end

 

  def cache

    @cache ||= {}

  end

 

  BASE_URI = 'http://api.openweathermap.org/data/2.5/weather?q='

  def http(place)

    uri = URI(BASE_URI + place)

 

    Net::HTTP.get(uri)

  rescue Timeout::Error

    raise NetworkError.new("Request timed out")

  rescue URI::InvalidURIError

    raise NetworkError.new("Bad place name: #{place}")

  rescue SocketError

    raise NetworkError.new("Could not reach #{uri.to_s}")

  end

end

Next is the spec file:

# spec/old_weather_query_spec.rb

 

require_relative 'spec_helper'

require_relative '../old_weather_query'

 

describe OldWeatherQuery do

  subject(:weather_query) { described_class }

 

  describe 'caching' do

    let(:json_response) do

      '{"weather" : { "description" : "Sky is Clear"}}'

    end

 

    around(:example) do |example|

      actual = weather_query.send(:cache)

      expect(actual).to eq({})

 

      example.run

 

      weather_query.clear!

    end

 

    it "stores results in local cache" do

      weather_query.forecast('Malibu,US')

 

      actual = weather_query.send(:cache)

      expect(actual.keys).to eq(['Malibu,US'])

      expect(actual['Malibu,US']).to be_a(Hash)

    end

 

    it "uses cached result in subsequent queries" do

      weather_query.forecast('Malibu,US')

      weather_query.forecast('Malibu,US')

      weather_query.forecast('Malibu,US')

    end

  end

 

  describe 'query history' do

    before do

      expect(weather_query.history).to eq([])

      allow(weather_query).to receive(:http).and_return("{}")

    end

    after do

      weather_query.clear!

    end

 

    it "stores every place requested" do

      places = %w(

        Malibu,US

        Beijing,CN

        Delhi,IN

        Malibu,US

        Malibu,US

        Beijing,CN

      )

 

      places.each {|s| weather_query.forecast(s) }

 

      expect(weather_query.history).to eq(places)

    end

 

    it "does not allow history to be modified" do

      expect {

        weather_query.history = ['Malibu,CN']

      }.to raise_error

 

      weather_query.history << 'Malibu,CN'

      expect(weather_query.history).to eq([])

    end

  end

 

  describe 'number of API requests' do

    before do

      expect(weather_query.api_request_count).to eq(0)

      allow(weather_query).to receive(:http).and_return("{}")

    end

 

    after do

      weather_query.clear!

    end

 

    it "stores every place requested" do

      places = %w(

        Malibu,US

        Beijing,CN

        Delhi,IN

        Malibu,US

        Malibu,US

        Beijing,CN

      )

 

      places.each {|s| weather_query.forecast(s) }

 

      expect(weather_query.api_request_count).to eq(3)

    end

 

    it "does not allow count to be modified" do

      expect {

        weather_query.api_request_count = 100

      }.to raise_error

 

      expect {

        weather_query.api_request_count += 10

      }.to raise_error

 

      expect(weather_query.api_request_count).to eq(0)

    end

  end

end

And last but not least, our spec_helper file, which has also changed only slightly: we only configure RSpec to show one slow spec (to keep test results uncluttered) and use color in the output to distinguish passes and failures more easily:

# spec/spec_helper.rb

 

require 'rspec'

 

RSpec.configure do |config|

  config.order            = 'random'

  config.profile_examples = 1

  config.color            = true

end

When we run these specs, something unexpected happens. Most of the time the specs pass, but sometimes they fail. If we keep running the specs with the same command, we’ll see the tests pass and fail apparently at random. These are flaky tests, and we have exposed them because of the random order configuration we chose. If our tests run in a certain order, they fail. The problem could be simply in our tests. For example, we could have forgotten to clear state before or after a test. However, there could also be a problem with our code. In any case, we need to get to the bottom of the situation:

We first notice that at the end of the failing test run, RSpec tells us “Randomized with seed 318”. We can use this information to run the tests in the order that caused the failure and start to debug and diagnose the problem. We do this by passing the –seed parameter with the value 318, as follows:

$ rspec spec/old_weather_query_spec.rb --seed 318

The problem has to do with the way that we increment @api_request_count without ensuring it has been initialized. Looking at our code, we notice that the only place we initialize @api_request_count is in OldWeatherQuery.api_request_count and OldWeatherQuery.clear!. If we don’t call either of these methods first, then OldWeatherQuery.forecast, the main method in this module, will always fail. Our tests sometimes pass because our setup code calls one of these methods first when tests are run in a certain order, but that is not at all how our code would likely be used in production. So basically, our code is completely broken, but our specs pass (sometimes). Based on this, we can create a simple spec that will always fail:

describe 'api_request is not initialized' do

  it "does not raise an error" do

    weather_query.forecast('Malibu,US')

  end   

end

At least now our tests fail deterministically. But this is not the end of our troubles with these specs. If we run our tests many times with the seed value of 318, we will start seeing a second failing test case that is even more random than the first. This is an OldWeatherQuery::NetworkError and it indicates that our tests are actually making HTTP requests to the Internet! Let’s do an experiment to confirm this. We’ll turn off our Wi-Fi access, unplug our Ethernet cables, and run our specs. When we run our tests without any Internet access, we will see three errors in total. One of them is the error with the uninitialized @api_request_count instance variable, and two of them are instances of OldWeatherQuery::NetworkError, which confirms that we are indeed making real HTTP requests in our code.

What’s so bad about making requests to the Internet? After all, the test failures are indeed very random and we had to purposely shut off our Internet access to replicate the errors. Flaky tests are actually the least of our problems. First, we could be performing destructive actions that affect real systems, accounts, and people! Imagine if we were testing an e-commerce application that charged customer credit cards by using a third-party payment API via HTTP. If our tests actually hit our payment provider’s API endpoint over HTTP, we would get a lot of declined transactions (assuming we are not storing and using real credit card numbers), which could lead to our account being suspended due to suspicions of fraud, putting our e-commerce application out of service. Also, if we were running a continuous integration (CI) server such as Jenkins, which did not have access to the public Internet, we would get failures in our CI builds due to failing tests that attempted to access the Internet.

There are a few approaches to solving this problem. In our tests, we attempted to mock our HTTP requests, but obviously failed to do so effectively. A second approach is to allow actual HTTP requests but to configure a special server for testing purposes. Let’s focus on figuring out why our HTTP mocks were not successful. In a small set of tests like in this example, it is not hard to hunt down the places where we are sending actual HTTP requests. In larger code bases with a lot of test support code, it may be harder. Also, it would be nice to prevent access to the Internet altogether so we notice these issues as soon as we run the offending tests.

Fortunately, Ruby has many excellent tools for testing, and there is one that addresses our needs exactly: WebMock (https://github.com/bblimke/webmock). We simply install the gem and add a couple of lines to our spec helper file to disable all network connections in our tests:

require 'rspec'

 

# require the webmock gem

require 'webmock/rspec'

 

RSpec.configure do |config|

  # this is done by default, but let's make it clear

  WebMock.disable_net_connect!

 

  Config.order            = 'random'

  config.profile_examples = 1

  config.color            = true

end

When we run our tests again, we’ll see one or more instances of WebMock::NetConnectNotAllowedError, along with a backtrace to lead us to the point in our tests where the HTTP request was made:

If we examine our test code, we’ll notice that we mock the OldWeatherQuery.http method in a few places. However, we forgot to set up the mock in the first describe block for caching where we defined a json_response object, but never mocked the OldWeatherQuery.http method to return json_response. We can solve the problem by mocking OldWeatherQuery.http throughout the entire test file. We’ll also take this opportunity to clean up the initialization of @api_request_count in our code. Here’s what we have now:

# new_weather_query.rb

 

require 'net/http'

require 'json'

require 'timeout'

 

module NewWeatherQuery

  extend self

 

  class NetworkError < StandardError

  end

 

  def forecast(place, use_cache=true)

    add_to_history(place)

    if use_cache

      cache[place] ||= begin

        increment_api_request_count

        JSON.parse( http(place) )

      end

    else

      JSON.parse( http(place) )

    end

  rescue JSON::ParserError => e

    raise NetworkError.new("Bad response: #{e.inspect}")

  end

 

  def increment_api_request_count

    @api_request_count ||= 0

    @api_request_count += 1

  end

 

  def api_request_count

    @api_request_count ||= 0

  end

 

  def history

    (@history || []).dup

  end

 

  def clear!

    @history           = []

    @cache             = {}

    @api_request_count = 0

  end

 

  private

 

  def add_to_history(s)

    @history ||= []

    @history << s

  end

 

  def cache

    @cache ||= {}

  end

 

  BASE_URI = 'http://api.openweathermap.org/data/2.5/weather?q='

  def http(place)

    uri = URI(BASE_URI + place)

 

    Net::HTTP.get(uri)

  rescue Timeout::Error

    raise NetworkError.new("Request timed out")

  rescue URI::InvalidURIError

    raise NetworkError.new("Bad place name: #{place}")

  rescue SocketError

    raise NetworkError.new("Could not reach #{uri.to_s}")

  end

end

And here is the spec file to go with it:

# spec/new_weather_query_spec.rb

 

require_relative 'spec_helper'

require_relative '../new_weather_query'

 

describe NewWeatherQuery do

  subject(:weather_query) { described_class }

 

  after { weather_query.clear! }

 

  let(:json_response) { '{}' }

  before do

    allow(weather_query).to receive(:http).and_return(json_response)     

  end

 

  describe 'api_request is initialized' do

    it "does not raise an error" do

      weather_query.forecast('Malibu,US')

    end   

  end

 

describe 'caching' do

    let(:json_response) do

      '{"weather" : { "description" : "Sky is Clear"}}'

    end

 

    around(:example) do |example|

      actual = weather_query.send(:cache)

      expect(actual).to eq({})

     

      example.run

    end

 

    it "stores results in local cache" do

      weather_query.forecast('Malibu,US')

 

      actual = weather_query.send(:cache)

      expect(actual.keys).to eq(['Malibu,US'])

      expect(actual['Malibu,US']).to be_a(Hash)

    end

 

    it "uses cached result in subsequent queries" do

      weather_query.forecast('Malibu,US')

      weather_query.forecast('Malibu,US')

      weather_query.forecast('Malibu,US')

    end

  end

 

  describe 'query history' do

    before do

      expect(weather_query.history).to eq([])

    end

 

    it "stores every place requested" do

      places = %w(

        Malibu,US

        Beijing,CN

        Delhi,IN

        Malibu,US

        Malibu,US

        Beijing,CN

      )

 

      places.each {|s| weather_query.forecast(s) }

 

      expect(weather_query.history).to eq(places)

    end

 

    it "does not allow history to be modified" do

      expect {

        weather_query.history = ['Malibu,CN']

      }.to raise_error

 

      weather_query.history << 'Malibu,CN'

      expect(weather_query.history).to eq([])

    end

  end

 

  describe 'number of API requests' do

    before do

      expect(weather_query.api_request_count).to eq(0)

    end

 

    it "stores every place requested" do

      places = %w(

        Malibu,US

        Beijing,CN

        Delhi,IN

        Malibu,US

        Malibu,US

        Beijing,CN

      )

 

      places.each {|s| weather_query.forecast(s) }

 

      expect(weather_query.api_request_count).to eq(3)

    end

 

    it "does not allow count to be modified" do

      expect {

        weather_query.api_request_count = 100

      }.to raise_error

 

      expect {

        weather_query.api_request_count += 10

      }.to raise_error

 

      expect(weather_query.api_request_count).to eq(0)

    end

  end

end

Now we’ve fixed a major bug with our code that slipped through our specs and used to pass randomly. We’ve made it so that our tests always pass, regardless of the order in which they are run, and without needing to access the Internet. Our test code and application code has also become clearer as we’ve reduced duplication in a few places.

A case study of production faithfulness with a test resource instance

We’re not done with our WeatherQuery example just yet. Let’s take a look at how we would add a simple database to store our cached values. There are some serious limitations to the way we are caching with instance variables, which persist only within the scope of a single Ruby process. As soon as we stop or restart our app, the entire cache will be lost. In a production app, we would likely have many processes running the same code in order to serve traffic effectively. With our current approach, each process would have a separate cache, which would be very inefficient. We could easily save many HTTP requests if we were able to share the cache between processes and across restarts. Economizing on these requests is not simply a matter of improved response time. We also need to consider that we cannot make unlimited requests to external services. For commercial services, we would pay for the number of requests we make. For free services, we are likely to get throttled if we exceed some threshold. Therefore, an effective caching scheme that reduces the number of HTTP requests we make to our external services is of vital importance to the function of a real-world app. Finally, our cache is very simplistic and has no expiration mechanism short of clearing all entries. For a cache to be effective, we need to be able to store entries for individual locations for some period of time within which we don’t expect the weather forecast to change much. This will keep the cache small and up to date.

We’ll use Redis (http://redis.io) as our database since it is very fast, simple, and easy to set up. You can find instructions on the Redis website on how to install it, which is an easy process on any platform. Once you have Redis installed, you simply need to start the server locally, which you can do with the redis-server command. We’ll also need to install the Redis Ruby client as a gem (https://github.com/redis/redis-rb).

Let’s start with a separate configuration file to set up our Redis client for our tests:

# spec/config/redis.rb

 

require 'rspec'

require 'redis'

 

ENV['WQ_REDIS_URL'] ||= 'redis://localhost:6379/15'

 

RSpec.configure do |config|

  if ! ENV['WQ_REDIS_URL'].is_a?(String)

    raise "WQ_REDIS_URL environment variable not set"

  end

  ::REDIS_CLIENT = Redis.new( :url => ENV['WQ_REDIS_URL'] )

 

  config.after(:example) do    

    ::REDIS_CLIENT.flushdb

  end

end

Note that we place this file in a new config folder under our main spec folder. The idea is to configure each resource separately in its own file to keep everything isolated and easy to understand. This will make maintenance easy and prevent problems with configuration management down the road.

We don’t do much in this file, but we do establish some important conventions. There is a single environment variable, which takes care of the Redis connection URL. By using an environment variable, we make it easy to change configuration and also allow flexibility in how these configurations are stored. Our code doesn’t care if the Redis connection URL is stored in a simple .env file with key-value pairs or loaded from a configuration database. We can also easily override this value manually simply by setting it when we run RSpec, like so:

$ WQ_REDIS_URL=redis://1.2.3.4:4321/0 rspec spec

Note that we also set a sensible default value, which is to run on the default Redis port of 6379 on our local machine, on database number 15, which is less likely to be used for local development. This prevents our tests from relying on our development database, or from polluting or destroying it. It is also worth mentioning that we prefix our environment variable with WQ (short for weather query). Small details like this are very important for keeping our code easy to understand and to prevent dangerous clashes. We could imagine the kinds of confusion and clashes that could be caused if we relied on REDIS_URL and we had multiple apps running on the same server, all relying on Redis. It would be very easy to break many applications if we changed the value of REDIS_URL for a single app to point to a different instance of Redis.

We set a global constant, ::REDIS_CLIENT, to point to a Redis client. We will use this in our code to connect to Redis. Note that in real-world code, we would likely have a global namespace for the entire app and we would define globals such as REDIS_CLIENT under that namespace rather than in the global Ruby namespace.

Finally, we configure RSpec to call the flushdb command after every example tagged with :redis to empty the database and keep state clean across tests. In our code, all tests interact with Redis, so this tag seems pointless. However, it is very likely that we would add code that had nothing to do with Redis, and using tags helps us to constrain the scope of our configuration hooks only to where they are needed. This will also prevent confusion about multiple hooks running for the same example. In general, we want to prevent global hooks where possible and make configuration hooks explicitly triggered where possible.

So what does our spec look like now? Actually, it is almost exactly the same. Only a few lines have changed to work with the new Redis cache. See if you can spot them!

# spec/redis_weather_query_spec.rb

 

require_relative 'spec_helper'

require_relative '../redis_weather_query'

 

describe RedisWeatherQuery, redis: true do

  subject(:weather_query) { described_class }

 

  after { weather_query.clear! }

 

  let(:json_response) { '{}' }

  before do

    allow(weather_query).to receive(:http).and_return(json_response)     

  end

 

  describe 'api_request is initialized' do

    it "does not raise an error" do

      weather_query.forecast('Malibu,US')

    end   

  end

    

     describe 'caching' do

    let(:json_response) do

      '{"weather" : { "description" : "Sky is Clear"}}'

    end

 

    around(:example) do |example|

      actual = weather_query.send(:cache).all

      expect(actual).to eq({})

     

      example.run

    end

    it "stores results in local cache" do

      weather_query.forecast('Malibu,US')

 

      actual = weather_query.send(:cache).all

      expect(actual.keys).to eq(['Malibu,US'])

      expect(actual['Malibu,US']).to be_a(Hash)

    end

 

    it "uses cached result in subsequent queries" do

      weather_query.forecast('Malibu,US')

      weather_query.forecast('Malibu,US')

      weather_query.forecast('Malibu,US')

    end

  end

 

  describe 'query history' do

    before do

      expect(weather_query.history).to eq([])

    end

 

    it "stores every place requested" do

      places = %w(

        Malibu,US

        Beijing,CN

        Delhi,IN

        Malibu,US

        Malibu,US

        Beijing,CN

      )

 

      places.each {|s| weather_query.forecast(s) }

 

      expect(weather_query.history).to eq(places)

    end

 

    it "does not allow history to be modified" do

      expect {

        weather_query.history = ['Malibu,CN']

      }.to raise_error

 

      weather_query.history << 'Malibu,CN'

      expect(weather_query.history).to eq([])

    end

  end

  describe 'number of API requests' do

    before do

      expect(weather_query.api_request_count).to eq(0)

    end

 

    it "stores every place requested" do

      places = %w(

        Malibu,US

        Beijing,CN

        Delhi,IN

        Malibu,US

        Malibu,US

        Beijing,CN

      )

 

      places.each {|s| weather_query.forecast(s) }

 

      expect(weather_query.api_request_count).to eq(3)

    end

 

    it "does not allow count to be modified" do

      expect {

        weather_query.api_request_count = 100

      }.to raise_error

 

      expect {

        weather_query.api_request_count += 10

      }.to raise_error

 

      expect(weather_query.api_request_count).to eq(0)

    end

  end

end

So what about the actual WeatherQuery code? It changes very little as well:

# redis_weather_query.rb

 

require 'net/http'

require 'json'

require 'timeout'

 

# require the new cache module

require_relative 'redis_weather_cache'

module RedisWeatherQuery

  extend self

 

  class NetworkError < StandardError

  end

 

  # ... same as before ...

 

  def clear!

    @history           = []

    @api_request_count = 0

     

    # no more clearing of cache here

  end

 

  private

 

  # ... same as before ...

   

  # the new cache module has a Hash-like interface

  def cache

    RedisWeatherCache

  end

 

  # ... same as before ...

   

end

We can see that we’ve preserved pretty much the same code and specs as before. Almost all of the new functionality is accomplished in a new module that caches with Redis. Here is what it looks like:

# redis_weather_cache.rb

 

require 'redis'

 

module RedisWeatherCache

  extend self

 

  CACHE_KEY             = 'weather_query:cache'

  EXPIRY_ZSET_KEY       = 'weather_query:expiry_tracker'

  EXPIRE_FORECAST_AFTER = 300 # 5 minutes  

 

  def redis_client

    if ! defined?(::REDIS_CLIENT)

      raise("No REDIS_CLIENT defined!")

    end

   

    ::REDIS_CLIENT

  end

 

  def []=(location, forecast)

    redis_client.hset(CACHE_KEY, location, JSON.generate(forecast))

    redis_client.zadd(EXPIRY_ZSET_KEY, Time.now.to_i, location)

  end

 

  def [](location)

    remove_expired_entries

   

    raw_value = redis_client.hget(CACHE_KEY, location)

   

    if raw_value

      JSON.parse(raw_value)

    else

      nil

    end

  end

 

  def all

    redis_client.hgetall(CACHE_KEY).inject({}) do |memo, (location, forecast_json)|

      memo[location] = JSON.parse(forecast_json)

      memo

    end

  end

 

  def clear!

    redis_client.del(CACHE_KEY)

  end

 

  def remove_expired_entries

    # expired locations have a score, i.e. creation timestamp, less than a certain threshold

    expired_locations = redis_client.zrangebyscore(EXPIRY_ZSET_KEY, 0, Time.now.to_i - EXPIRE_FORECAST_AFTER)

 

    if ! expired_locations.empty?

      # remove the cache entry

      redis_client.hdel(CACHE_KEY, expired_locations)           

      # also clear the expiry entry

      redis_client.zrem(EXPIRY_ZSET_KEY, expired_locations) 

    end

  end

end

We’ll avoid a detailed explanation of this code. We simply note that we accomplish all of the design goals we discussed at the beginning of the section: a persistent cache with expiration of individual values. We’ve accomplished this using some simple Redis functionality along with ZSET or sorted set functionality, which is a bit more complex, and which we needed because Redis does not allow individual entries in a Hash to be deleted. We can see that by using method names such as RedisWeatherCache.[] and RedisWeatherCache.[]=, we’ve maintained a Hash-like interface, which made it easy to use this cache instead of the simple in-memory Ruby Hash we had in our previous iteration. Our tests all pass and are still pretty simple, thanks to the modularity of this new cache code, the modular configuration file, and the previous fixes we made to our specs to remove Internet and run-order dependencies.

Summary

In this article, we delved into setting up and cleaning up state for real-world specs that interact with external services and local resources by extending our WeatherQuery example to address a big bug, isolate our specs from the Internet, and cleanly configure a Redis database to serve as a better cache.

Resources for Article:


Further resources on this subject:


LEAVE A REPLY

Please enter your comment!
Please enter your name here