Agile Testing & BDD eXchange 2016 (Part 2)

A few months ago I had the opportunity to go to the Agile Testing & BDD eXchange 2016 conference ( I’ve finally had the chance to write up the notes I took (if any of the speakers are reading this and feel that I’m ‘stealing their lunch’ or infringing on their rights, please let me know and I’ll remove the offending content.)

The notes below reflect my own jottings down and interpretations, and not necessarily the views of the speakers. 

Aslak Hellesøy

  • Good tests
    • Inspire Confidence -no third party service
    • Fast -milliseconds or second
    • Lowcost -easy to write and maintain: decoupled architecture, don’t try to test a UI.
  • Use Decoupled architecture, direct injection and ‘contracts’ so that you have a load of smaller, cheap unit tests, and a few deeper, full stack ones.
  • Testing doesn’t find defects, testing prevents them

User Stories, OOPSI Method (Jenny Martin)

  • User stories are deceptive, they look too simple
  • There is no ‘scope creep’; the scope was already there, just not known about.
  • JIRA is not a documentation system
  • Focus on concrete examples and outcomes, not stories
  • Use User Story Mapping to chunk work into Opening, Mid and End games (stages). Get an MVP for your outcomes as soon as possible.
  • Use the OOPSI system to focus on the following, in this order:
    • Outcomes -what outcomes do we want (in business terms)
    • Outputs -what outputs are needed
    • Processes -what processes generated those outputs
    • Scenarios -what scenarios trigger those processed
    • Inputs -what input do those scenarios need?
  • Keep
    • Conversations
    • Collaborations
    • Chunking
  • Stop
    • Up front analysis
    • Worrying about the format of the user story, its not important
    • Writing stories in JIRA.
  • More
    • Checking outcome
    • Knowing what level we’re at
    • Documentation
    • BDD

Problems With Agile (Martine Devos)

  • What is bad:
    • Everyone wants to do Agile!
    • It’s all courses, certification
    • Example quotes: “We are uncovering easier ways of getting certificates on how to help others get certificates”
    • Software is very tribal; belonging to the agile tribe is note delivering good software!
    • Everything is ‘scalable’, this is bad
    • Courses are being favoured over experience
    • We’re still having fixed scope, time, cost.
    • Calling Continuous Integration ‘DevOps’
  • Why is this happening:
    • When we find a pattern that works, we feel good and try to reproduce this pattern everywhere.
    • Confirmation bias -we only notice when it works, not when it doesn’t.
    • Apophany -false epiphany.
    • People want predictability: they assume that predictability exists.
  • Organizational structure has a big say.
  • Bureaucracy doesn’t value knowledge or action; Agile thrives in an ‘Adhocracy’.
Bureaucracy Meritocracy Adhocracy
Values Position Knowledge Action
Driven By Rules Mutual Adjustment Opportunity
Decides By Hierarchy Logical Argument Experimentation
Strives towards Extrinsic Rewards Personal Mastery Achievement
  • Well formed outcomes are essential.
  • Scale Agile where failure is survivable. Experiment.
  • Do safe to fail probing of new agile practices.
  • Cynefin framework: Four domains of complexity plus a central nexus, Disorder:
    • Obvious: Self apparent, simple
    • Complicated: Less simple, but mechanical, ordered
    • Complex: cause and effect only knowable in retrospect
    • Chaotic: no order, generally adverse, transient, need to act quickly.
    • Disorder: when you don’t know which domain you are in

Cosmina Bradu: A BDD Test Case

  • Anti-patterns
    • Scenario Overwhelm
    • Difficulty in sharing
    • Shared scenarios do not equal shared understanding
    • It was all about HOW, not WHAT and WHY
  • Solution
    • Confirm as soon as possible
    • Itereative discover/deliver/confirm cycle, with confirmation (based on evidence) at all stages
    • Test shared understanding.

Agile Testing & BDD eXchange 2016 (Part 1)

A few months ago I had the opportunity to go to the Agile Testing & BDD eXchange 2016 conference ( I’ve finally had the chance to write up the notes I took (if any of the speakers are reading this and feel that I’m ‘stealing their lunch’ or infringing on their rights, please let me know and I’ll remove the offending content.)

The notes below reflect my own jottings down and interpretations, and not necessarily the views of the speakers.

Repeated Themes and Standout Points

  • Focus on concrete examples and outcomes rather than abstract user stories (See Dan North And Jenny)
  • Confirm scenarios with data and experimentation
  • BDD is for helping the team understand what a system actually should be doing, not for testing the system in detail (Dan North and other speakers)
  • Agile has become a ‘tribe’ and a buzzword, relying on certification and ‘one size fits all’ solutions rather than experience and good judgement.
  • User story mapping is a very powerful way of splitting your application into ‘open, mid and end game’ stages; this allows you to get confirmation at the early ‘opening’ stage (Jenny and Cosmina Bradu).
  • Agile lends itself to an Adhocracy, where action, experimentation and achievement is valued, rather than a meritocracy, where knowledge and mutual agreement are valued or a bureaucracy, where position and business hierarchy are.
  • Adhering to Agile, and sharing stories/scenarios, doesn’t meant that the team and stakeholders are sharing understanding. Test the shared understanding of your system.
  • A decoupled, modular, contract based approach to unit test, integration tests and acceptance tests is essential. The worst thing you can do is rely on many long integration tests that take hours to run, and test everything from the UI to the database. A suite of many, quick, single concern, isolated unit tests that take milliseconds to run, coupled with a few long integration tests is a much better way.
  • For mobile development:
    • Adopt a more server-side approach, similar to MVC, and separate the UI from the business logic, testing them separately; the UI is susceptible to much change, the logic not so much so you don’t want to tie them together (Paul Stringer).
    • Wireframes do not capture requirements.

Dan North

  • Testers feel left behind with Agile
  • BDD is a tester driven movement
  • Cucumber and BDD in general shouldn’t be used for integration/acceptance tests
  • BDD should really be BGD -Behavior Guided Development
  • BDD captures and constrains business scenarios, with feature files serving as a description of the systems behaviour.
  • After a certain point, you can throw your scenarios away. They should not be used for integration testing.
  • Definition of testing: increasing confidence of Stakeholders through eveidence.
  • Stakeholder: Anyone who’s life is touched by your code.
  • TDD build confidence of programmer, but BDD build confidence for Stakeholders.
  • BDD
    • Acknowledges several Stakeholders
    • Discussed acceptance criteria from outset
  • Scenarios start as an enabling constraint to guide design
  • There’s a difference between scenarios and automated testing.
    • Scenario: tests what the general case is, how to gain confidence into that scenario, and what stakeholders have we not yet served.
    • Acceptance tests: gains confidence other stakeholders (eg security, devops, stability etc).
  • Cucumber was not designed for testing -it has bad logging, is removed from actual code (via steps), a good testing framework is near the action.
  • Python is better for testing! (as it is still easily readable by non technical people).


REST: What It Is (and What It Isn’t!)

REST (Representational State Transfer) has been bandied around as a term for a long time when it comes to API design, and it’s not hard to see the attraction. It relies on the pre-existing HTTP protocol, which is pretty much what the web already runs on, and if implemented well it can make use of a lot of features of HTTP (such as caching).  This is a huge boon to application developers, saving them having to learn/implement a new technology/protocol and implement these features themselves. Continue reading

Six Pillars Of Security, #6: Appropriate Escalation and Containment

  • In the event of a breach or an infringement of your companies responsibilities, timely and appropriate escalation is required.
  • During one breach I witnessed at a company I used to work for, inappropriate and untimely escalation made the situation a lot worse; the dev team and their managers failed to escalate a serious issue (users credentials being logged in a log file) quickly and appropriately, and as result the situation escalated.
    • access to files is often logged. In the case of a breach, the lower the number of people who accessed the compromised resources the smaller the aftermath (e.g in the case of sensitive data being logged to a file, it’s easier to deal with five people who accessed the compromised file, than thirty). Reducing initial propagation helps this.

Continue reading

Six Pillars Of Security, #5: Controlling External Risk

Once data leaves your system, your ability to control it rapidly diminishes. However there are steps you can take to mitigate risks:

  • Only giving clients the data they require
    • For example, with a centralized service application this would involve analyzing what each client needs, and applying logic so that they only receive that information.
  • Actively engaging with client teams, asking them about security, guiding them. Even though the data has left your system, it is still your data and you need to ensure others are being careful with it.

Continue reading

Six Pillars Of Security, #4: Not Helping The Bad Guys

  • Systems often accidentally reveal their workings and vulnerabilities:
  • Revealing the server/component type and version in their error response -this allows the attacker to search for known vulnerabilities against that version number.
  • Logging full stack traces, or worse showing them in the error response -again, this tells the attacker what libraries the system uses, and if any of those libraries have known vulnerabilities, the attacker can exploit this.
  • Just because your client is internal and not ‘user facing’ don’t assume that your error responses won’t filter through to the outside -in a highly distributed system, you never know where your response will end up.
  • Same goes for logging; you don’t know who will end up reading your error logs.
  • Basic Action Points For A Team:
    • Never log full stack traces, unless absolutely necessary.
    • Confirm that non of your responses contain information about the server products you use or their versions.