Agile Testing & BDD eXchange 2016 (Part 1)

A few months ago I had the opportunity to go to the Agile Testing & BDD eXchange 2016 conference ( I’ve finally had the chance to write up the notes I took (if any of the speakers are reading this and feel that I’m ‘stealing their lunch’ or infringing on their rights, please let me know and I’ll remove the offending content.)

The notes below reflect my own jottings down and interpretations, and not necessarily the views of the speakers.

Repeated Themes and Standout Points

  • Focus on concrete examples and outcomes rather than abstract user stories (See Dan North And Jenny)
  • Confirm scenarios with data and experimentation
  • BDD is for helping the team understand what a system actually should be doing, not for testing the system in detail (Dan North and other speakers)
  • Agile has become a ‘tribe’ and a buzzword, relying on certification and ‘one size fits all’ solutions rather than experience and good judgement.
  • User story mapping is a very powerful way of splitting your application into ‘open, mid and end game’ stages; this allows you to get confirmation at the early ‘opening’ stage (Jenny and Cosmina Bradu).
  • Agile lends itself to an Adhocracy, where action, experimentation and achievement is valued, rather than a meritocracy, where knowledge and mutual agreement are valued or a bureaucracy, where position and business hierarchy are.
  • Adhering to Agile, and sharing stories/scenarios, doesn’t meant that the team and stakeholders are sharing understanding. Test the shared understanding of your system.
  • A decoupled, modular, contract based approach to unit test, integration tests and acceptance tests is essential. The worst thing you can do is rely on many long integration tests that take hours to run, and test everything from the UI to the database. A suite of many, quick, single concern, isolated unit tests that take milliseconds to run, coupled with a few long integration tests is a much better way.
  • For mobile development:
    • Adopt a more server-side approach, similar to MVC, and separate the UI from the business logic, testing them separately; the UI is susceptible to much change, the logic not so much so you don’t want to tie them together (Paul Stringer).
    • Wireframes do not capture requirements.

Dan North

  • Testers feel left behind with Agile
  • BDD is a tester driven movement
  • Cucumber and BDD in general shouldn’t be used for integration/acceptance tests
  • BDD should really be BGD -Behavior Guided Development
  • BDD captures and constrains business scenarios, with feature files serving as a description of the systems behaviour.
  • After a certain point, you can throw your scenarios away. They should not be used for integration testing.
  • Definition of testing: increasing confidence of Stakeholders through eveidence.
  • Stakeholder: Anyone who’s life is touched by your code.
  • TDD build confidence of programmer, but BDD build confidence for Stakeholders.
  • BDD
    • Acknowledges several Stakeholders
    • Discussed acceptance criteria from outset
  • Scenarios start as an enabling constraint to guide design
  • There’s a difference between scenarios and automated testing.
    • Scenario: tests what the general case is, how to gain confidence into that scenario, and what stakeholders have we not yet served.
    • Acceptance tests: gains confidence other stakeholders (eg security, devops, stability etc).
  • Cucumber was not designed for testing -it has bad logging, is removed from actual code (via steps), a good testing framework is near the action.
  • Python is better for testing! (as it is still easily readable by non technical people).


REST: What It Is (and What It Isn’t!)

REST (Representational State Transfer) has been bandied around as a term for a long time when it comes to API design, and it’s not hard to see the attraction. It relies on the pre-existing HTTP protocol, which is pretty much what the web already runs on, and if implemented well it can make use of a lot of features of HTTP (such as caching).  This is a huge boon to application developers, saving them having to learn/implement a new technology/protocol and implement these features themselves. Continue reading

Six Pillars Of Security, #6: Appropriate Escalation and Containment

  • In the event of a breach or an infringement of your companies responsibilities, timely and appropriate escalation is required.
  • During one breach I witnessed at a company I used to work for, inappropriate and untimely escalation made the situation a lot worse; the dev team and their managers failed to escalate a serious issue (users credentials being logged in a log file) quickly and appropriately, and as result the situation escalated.
    • access to files is often logged. In the case of a breach, the lower the number of people who accessed the compromised resources the smaller the aftermath (e.g in the case of sensitive data being logged to a file, it’s easier to deal with five people who accessed the compromised file, than thirty). Reducing initial propagation helps this.

Continue reading

Six Pillars Of Security, #5: Controlling External Risk

Once data leaves your system, your ability to control it rapidly diminishes. However there are steps you can take to mitigate risks:

  • Only giving clients the data they require
    • For example, with a centralized service application this would involve analyzing what each client needs, and applying logic so that they only receive that information.
  • Actively engaging with client teams, asking them about security, guiding them. Even though the data has left your system, it is still your data and you need to ensure others are being careful with it.

Continue reading

Six Pillars Of Security, #4: Not Helping The Bad Guys

  • Systems often accidentally reveal their workings and vulnerabilities:
  • Revealing the server/component type and version in their error response -this allows the attacker to search for known vulnerabilities against that version number.
  • Logging full stack traces, or worse showing them in the error response -again, this tells the attacker what libraries the system uses, and if any of those libraries have known vulnerabilities, the attacker can exploit this.
  • Just because your client is internal and not ‘user facing’ don’t assume that your error responses won’t filter through to the outside -in a highly distributed system, you never know where your response will end up.
  • Same goes for logging; you don’t know who will end up reading your error logs.
  • Basic Action Points For A Team:
    • Never log full stack traces, unless absolutely necessary.
    • Confirm that non of your responses contain information about the server products you use or their versions.

Six Pillars Of Security, #2: Configuration, Internal Processes And Human Error


  • 1-mistakeCare with configuration. Misconfiguration is one of the top 5 reasons behind companies getting hacked.
  • For sensitive data and functionality, consider incorporating a per-role based permissions system to reduce risk, and help track what happened in the event of an attack.
  • Principle of least privilege again helps secure a system; a leaked password isn’t any use if there is no way to invoke important processes with it.

Continue reading