BDD - Specifications by Example

According to Gojko Adzic, the author of ‘Specification by Example’, Specification by Example is a set of process patterns that facilitate change in software products to ensure that the right product is delivered efficiently.”

Specification by Example is a collaborative approach to define the requirements and business-oriented functional tests for software products based on capturing and illustrating requirements using realistic examples instead of abstract statements.

Specification by Example – Overview

The objective of Specification by Example is to focus on development and delivery of prioritized, verifiable, business requirements. While the concept of Specification by Example in itself is relatively new, it is simply a rephrasing of existing practices.

It supports a very specific, concise vocabulary known as ubiquitous language that −

• Enables executable requirements.

• Is used by everyone in the team.

• Is created by a cross-functional team.

• Captures everyone's understanding.

Specification by Example can be used as a direct input into building Automated tests that reflect the business domain. Thus, the focus of Specification by Example is on building the right product and building the product right.

Purpose of Specification by Example

The primary aim of Specification by Example is to build the right product. It focuses on shared understanding, thus establishing a single source of truth. It enables automation of acceptance criteria so that focus is on defect prevention rather than defect detection. It also promotes test early to find the defects early.

Use of SbE

Specification by Example is used to illustrate the expected system behavior that describes business value. The illustration is by means of concrete and real life examples. These examples are used to create executable requirements that are −

• Testable without translation.

• Captured in live documentation.

Following are the reasons why we use examples to describe particular specifications −

• They are easier to understand.

• They are harder to misinterpret.

Advantages of SbE

The advantages of using Specification by Example are −

• Increased quality

• Reduced waste

• Reduced risk of production defects

• Focused effort

• Changes can be made more safely

• Improved business involvement

Applications of SbE

Specification by Example find applications in −

• Either complex business or complex organization.

• Does not work well for purely technical problems.

• Does not work well for UI focused software products.

• Can be applied to legacy systems as well.

SbE and Acceptance Testing

The advantages of Specification by Example in terms of Acceptance testing are −

• One single illustration is used for both, detailed requirements and testing

• Progress of the project is in terms of Acceptance tests −

  • Each test is to test a behavior.

  • A test is either passing for a behavior or it is not.

  • A passing test represents that the particular behavior is completed.

  • If a project that requires 100 behaviors to be completed has 60 behaviors completed, then it is 60% finished.

• Testers switch from defect fixing to defect prevention, and they contribute to the design of the solution.

• Automation allows instant understanding of the impact of a requirement change on the solution.

Specification by Example – What it means for different Roles

The objective of Specification by Example is to promote collaboration of everyone in the team, including the customer throughout the project to deliver business value. Everyone for better understandability uses same Vocabulary.

SbE – A Set of Process Patterns

As we have seen in the beginning of this chapter, Specification by Example is defined as a set of process patterns that facilitate change in software products to ensure that the right product is delivered efficiently.

The process patterns are −

• Collaborative specification

• Illustrating specifications using examples

• Refining the specification

• Automating examples

• Validating frequently

• Living documentation

Collaborative Specification

The objectives of collaborative specification are to −

• Get the various roles in a team to have a common understanding and a shared vocabulary.

• Get everyone involved in the project so that they can contribute their different perspectives about a feature.

• Ensure shared communication and ownership of the features.

These objectives are met in a specification workshop also known as the Three Amigos meeting. The Three Amigos are BA, QA and the developer. Though there are other roles in the project, these three would be responsible and accountable from definition to the delivery of the features.

During the meeting −

• The Business Analyst (BA) presents the requirements and tests for a new feature.

• The three Amigos (BA, Developer, and QA) discuss the new feature and review the specifications.

• The QA and developer also identify the missing requirements.

• The three Amigos

  • Utilize a shared model using a ubiquitous language.

  • Use domain vocabulary (A glossary is maintained if required).

  • Look for differences and conflicts.

• Do not jump to implementation details at this point.

• Reach a consensus about whether a feature was specified sufficiently.

• A shared sense of requirements and test ownership facilitates quality specifications

• The requirements are presented as scenarios, which provide explicit, unambiguous requirements. A scenario is an example of the system’s behavior from the users’ perspective.

Illustrating Specification using Examples

Scenarios are specified using the Given-When-Then structure to create a testable specification −

Given <some precondition>

And <additional preconditions> Optional

When <an action/trigger occurs>

Then <some post condition>

And <additional post conditions> Optional

This specification is an example of a behavior of the system. It also represents an Acceptance criterion of the system.

The team discusses the examples and the feedback is incorporated until there is agreement that the examples cover the feature's expected behavior. This ensures good test coverage.

Refining the Specification

To refine a specification,

• Be precise in writing the examples. If an example turns to be complex, split it into simpler examples.

• Focus on business perspective and avoid technical details.

• Consider both positive and negative conditions.

• Adhere to the domain specific vocabulary.

• Discuss the examples with the customer.

  • Choose conversations to accomplish this.

  • Consider only those examples that the customer is interested in. This enables production of the required code only and avoids covering every possible combination, that may not be required

• To ensure that the scenario passes, all the test cases for that scenario must pass. Hence, enhance the specifications to make them testable. The test cases can include various ranges and data values (boundary and corner cases) as well as different business rules resulting in changes in data.

• Specify additional business rules such as complex calculations, data manipulation / transformation, etc.

• Include non-functional scenarios (e.g. performance, load, usability, etc.) as Specification by Example

Automating Examples

The automation layer needs to be kept very simple – just wiring of the specification to the system under test. You can use a tool for the same.

Perform testing automation using Domain Specific Language (DSL) and show a clear connection between inputs and outputs. Focus on specification, and not script. Ensure that the tests are precise, easy to understand and testable.

Validating Frequently

Include example validation in your development pipeline with every change (addition/modification). There are many techniques and tools that can (and should) be adopted to help ensure the quality of a product. They revolve around three key principles- Test Early, Test Well and Test Often.

Execute the tests frequently so that you can identify the weak links. The examples representing the behaviors help track the progress and a behavior is said to be complete only after the corresponding test passes.

Living Documentation

Keep the specifications as simple and short as possible. Organize the specifications and evolve them as work progresses. Make the documentation accessible for all in the team.

Specification by Example Process Steps

The illustration shows the process steps in Specification by Example.

Anti-patterns

Anti-patterns are certain patterns in software development that is considered a bad programming practice. As opposed to design patterns, which are common approaches to common problems, which have been formalized and are generally considered a good development practice, anti-patterns are the opposite and are undesirable

Anti-patterns give rise to various problems.

Solution to the Problems - Quality

Quality can be ensured by keeping a watch on the anti-patterns. To minimize the problems created by anti-patterns, you should −

• Get together to specify using examples.

• Clean up and improve the examples.

• Write a code, which satisfies the examples

• Automate the examples and deploy.

• Repeat the approach for every user story.

To solve the problems due to anti-patterns means adherence to −

• Collaboration.

• Focusing on what.

• Focusing on Business.

• Be prepared.

Let us understand what each of the above mean.

Collaboration

In collaboration −

• Business people, developers and testers give input from their own perspectives.

• Automated examples prove that the team has built the correct thing.

• The process is more valuable than the tests themselves.

Focusing on what

You must focus on the question - ‘what.’ While focusing on ‘what’ −

• Do not try to cover all the possible cases.

• Do not forget to use different kind of tests.

• Keep examples as simple as possible.

• Examples should be easily understandable by the users of the system.

• Tools should not play an important part in the workshops.

Focusing on Business

To focus on the business −

• Keep specification at business intent.

• Include business in creating and reviewing specs.

• Hide all the details in the automation layer.

Be prepared

Be prepared for the following −

• Benefits are not immediately apparent, even while the team practices are changed.

• Introducing SbE is challenging.

• Requires time and investments.

• Automated testing does not come free.

Tools

Use of tools is not mandatory for Specification by Example, though in practice several tools are available. There are cases that are successful following Specification by Example even without using a tool.

The following tools support Specification by Example −

• Cucumber

• SpecFlow

• Fitnesse

• Jbehave

• Concordion

• Behat

• Jasmine

• Relish

• Speclog