Observability as code, secrets as a service, and chaos katas: ThoughtWorks outlines key engineering techniques to trial and assess

5 min read

ThoughtWorks has just published vol. 19 of its essential radar report. As always, it’s a vital insight into what’s beginning to emerge in the technology field.

In the techniques quadrant of its radar, there were some really interesting new entries. Let’s take a look at some of them now, so you can better plan and evaluate your roadmap and skill set for 2019.

8 of the best new techniques you should be trialling (according to ThoughtWorks)

1% canary: a way to build better feedback loops

This sounds like a weird one, but the concept is simple. It’s essentially about building a quick feedback loop to a tiny segment of customers – say, 1%. This can allow engineering teams to learn things quickly and make changes on other aspects of the project as it evolves.

Bounded buy: a smarter way to buy out-of-the-box software solutions

Bounded buy mitigates the scope creep that can cause headaches for businesses dealing with out-of-the-box software. It means those responsible for purchasing software focus only on solutions that are modular, with each ‘piece’ directly connecting into a particular department’s needs or workflow.

Crypto shredding: securing sensitive data

Crypto shredding is a method of securing data that might otherwise be easily replicated or copied. Essentially, it overwrites sensitive data with encryption keys which can easily be removed or deleted. It adds an extra layer of control over a large data set – a technique that could be particularly useful in a field like healthcare.

Four key metrics – focus on what’s most important to build a high performance team

Building a high performance team, can be challenging. Accelerate, the team behind the State of DevOps report, highlighted key drivers that engineers and team leaders should focus on: lead time, deployment frequency, mean time to restore (MTTR), and change fail percentage. According to ThoughtWorks “each metric creates a virtuous cycle and focuses the teams on continuous improvement.”

Observability as code – breaking through the limits of traditional monitoring tools

Observability has emerged as a bit of a buzzword over the last 12 months. But in the context of microservices, and increased complexity in software architecture, it is nevertheless important. However, the means through which you ‘do’ observability – a range of monitoring tools and dashboards – can be limiting in terms of making adjustments and replicating dashboards. This is why treating observability as code is going to become increasingly more important.

It makes sense – if infrastructure as code is the dominant way we think about building software, why shouldn’t it be the way we monitor it too?

Run cost as architecture fitness function

There’s a wide assumption that serverless can save you money. This is true when you’re starting out, or want to do something quickly, but it’s less true as you scale up. If you’re using serverless functions repeatedly, you’re likely to be paying a lot – more than if you has a slightly less fashionable cloud or on premise server.

To combat this complacency, you should instead watch how much services cost against the benefit delivered by them. Seems obvious, but easy to miss if you’ve just got excited about going serverless.

Secrets as a service

Without wishing to dampen what sounds incredibly cool, secrets as a service are ultimately just elaborate password managers. They can help organizations more easily decouple credentials, API keys from their source code, a move which should ensure improved security – and simplicity. By using credential rotation, organizations can be much better prepared at tackling and mitigating any security issues.

AWS has ‘Secrets Manager‘ while HashiCorp’s Vault offers similar functionality.

Security chaos engineering

In the last edition of Radar, security chaos engineering was in the assess phase – which means ThoughtWorks thinks it’s worth looking at, but maybe too early to deploy. With volume 19, security chaos engineering has moved into trial.

Clearly, while chaos engineering more broadly has seen slower adoption, it would seem that over the last 12 months the security field has taken chaos engineering to heart.

2 new software engineering techniques to assess

Chaos katas

If chaos engineering is finding it hard to gain mainstream adoption, perhaps chaos katas is the way forward. This is essentially a technique that helps engineers deploy chaos practices in their respective domains using the training approach known as kata – a Japanese word that simply refers to a set of choreographed movements.

In this context, the ‘katas’ are a set of code patterns that implement failures in a structured way, which engineers can then identify and explore. This is essentially a bottom up way of doing chaos engineering that also gives engineers a deeper insight into their software infrastructure.

Infrastructure configuration scanner

The question of who should manage your infrastructure is still a tricky one, with plenty of conflicting perspectives. However, from a productivity and agility perspective, putting the infrastructure in the hands of engineers makes a lot of sense.

Of course, this could feel like an extra burden – but with an infrastructure configuration scanner, like Scout2 or Watchmen, engineers can ensure that everything is configured correctly.

Software engineering techniques need to maintain simplicity as complexity increases

There’s clearly a diverse range of techniques on the ThoughtWorks Radar. Ultimately, however, the picture that emerges is one where efficiency and observability are key. A crucial part of software engineering will managing increased complexity and developing new tools and processes to instil some degree of simplicity and clarity.

Was there anything ThoughtWorks missed?