What’s that smell?

More and more frequently, I see user stories that look like this:

As a User
Given I have accessed the login page
AND I enter my username
AND I enter my password
If my account is valid
AND my password is at least 12-40 characters long and is encrypted AES192
WHEN I press the login button
AND the login credentials have not been rejected more than 3 times
AND the remember me checkbox is ticked
Then I want the website to remember my details
AND log me in
AND create a session cookie
SO that I can log in securely

In this real example, the user story value statement “As a <user>...I want....so that...” is smashed together with BDD’s “given-when-then” scenario, with a smattering of acceptance criteria and other details chucked into a mess of a sentence where the focus and meaning has almost disappeared. Try saying it out loud!

There is a ‘smell’ here - the smell of blindly following a process, by one author, who doesn’t understand what the value statement is for. This is in direct opposition to “individual and interactions over processes and tools”

These statements are supposed to give clarity and meaning to anyone involved in the developing the software product, and yet are some of the most indecipherable nonsense around. And worse, they have generated a sub-culture of dodging responsibility as a substitute for thinking - this has become ‘the Agile way’ on how requirements are written, and its dangerous.

The most glaring issues originate with confusion around BDD. ‘Behaviour Driven Development’ has become so misunderstood that its has been twisted into ‘Acceptance criteria-driven test scripting’, very far from what the BDD authors were reaching for.

Dan North originally developed BDD as a method for “The kinds of interaction between the people doing the work to deliver software”. “BDD is actually about the sequence of interactions between the various people on my team. The stories, scenarios and the code itself is a byproduct of that” Dan North, OOPLSA 2007.

What went wrong?

Problems started when this new interactive approach to product definition came with a shiny new set of tools - JBehave, Cucumber, Concordion etc. This gave eager Developers and Testers something to grab onto, to champion their expertise - so of course we must use these tools! Add into the mix that BDD sounds like TDD (another misunderstood practice) and further confusion abounds. Everyone has set sail in the BDD ship without knowing how to navigate. This has lead to the badly confused scenarios above, a drive to automate testing rather than discuss features and a shutdown of exploratory thinking of product strategy.

As Mark Winteringham points out in The Deadly sins of BDD Scenarios, when scenarios are written in imperative language “we are focused on the granular steps, actions only” “When we use declarative language, their abstract nature allows us to think about different questions...beyond merely carrying out those steps. An excellent tool for collaboration.”

But the real criticisms of the current use of BDD are:

Smells like waterfall. Here we go again with the big, up-front design in detail.
No Conversations. Scenarios are meant to be discussed, negotiated, agreed, possibly in a 3 Amigos meeting. Instead, I just write my requirements in this format - job done.
Lots of detail = lots of work. Including lots of detail in the scenario means that has to be reflected in the automation scripts, which cause them to multiply when other details are included. So detailed scenarios lead to detailed tests. This fine-grain level, plus the proliferation of edge-cases means you team will be writing those test scripts - in addition to unit test scripts.
Difficult to maintain. The explosion of detailed scenarios needs to be managed. Details are likely to change, so you better have a good way of finding those details across your scripts, their after-effects and their dependencies.
Slow test times. When there are a multitude of scripts testing the details my mimicking the user experience, these tests take lots of cpu cycles and time, both in execution and in management (set up, tear down, test data, coordination etc). This can drag on the movement to continuous integration.

So how do we use BDD, and when?

But first why? And the answer is Cynefin. The Cynefin Framework (ref 4) tells us how to operate in a complex system - where features and requirements must be explored and experimented on in order to find better solutions. BDD is a way to express examples of what a system should do when it is difficult to express requirements in the normal way. For example, when the domain you are in has an unfamiliar, confusing or even unknown terminology- Dan North’s example (ref 1) is the futures markets, but many exist in the medical, Banking, Engineering, and pretty much all other sectors. Where ‘normal’ requirements analysis becomes uncertain or impenetrable, using high-level examples is a great way to gain insight into what is needed. Think about it - do we really need a ‘scenario’ to tell us what login is?

In Summary, We need BDD when working in the complex space, something that Liz Keogh explains effortlessly in (Ref 3, part 2)

So what are BDD scenarios?

One of Dan North’s principles is "In Agile, we assume we got it wrong", so we need to find possible solutions as early as possible using deliberate discovery skills, for example during a conversation of ‘The 3 Amigos’. We can help guide these conversations by describing the problem as scenarios, then look for uncertainty, and try things out.

What is a Scenario? “An example of the entire system in action from the point of view of someone who is using it” (ref 1)

Approach to Scenarios “Having a conversation is more valuable than documenting it, which is more valuable than automating it. (ref 3)”

Scenarios are there to guide the conversation from what the user is trying to achieve to how the system should behave.They should remain at a high-level, written in a declarative manner using the domain language. They are there to bring clarity - not detail - and also invite alternatives and negotiation for their implementation. They should also be specific in their description of the example. Following these guidelines, the above story would render better as:

User Story: As a security manager I want customers to login securely so that our data and transactions are safeguarded from hackers.

Scenario: Logon with valid account
GIVEN I type ‘dewart’ into the user ID field
AND I type ‘Password1’ into the password field
WHEN I click the login button
THEN the web session is secured to our security standards

Acceptance criteria:
Validated that the password is at least 12-40 characters long
Validated that the password is encrypted AES192 while stored
Validated that the login credentials have not been rejected more than 3 times
Validated that if the ‘remember me’ checkbox is ticked, the website remembers my details

And yes, this can be automated into a test - but that is not (so much) the point - as the Texan’s say, “thats just gravy”

So If I could sum up when to use BDD:

GIVEN we want to explore and guide the possible behaviour of a complex system
WHEN we discuss the possible actions a system should take in different scenarios
THEN write the Scenarios using declarative statements and discuss

Call to action: What has been your experience with BDD scenarios - do you find horror stories like mine? Or am I just unlucky? When do you find them best to use? Let us know in the comments below:

References:

Dan North: Interview with Dan North on Behaviour-Driven Development” 2007
Mark Winteringham: “The Deadly sins of BDD Scenarios”, YouTube 2017
Liz Keogh: “10 years of doing Behaviour Driven Development all wrong ” Parts 1 & 2, YouTube
Dave Snowdon: Introduction to the Cynefin Framework, Cognitive Edge, http://cognitive-edge.com/videos/cynefin-framework-introduction/