Agile Testing Days 2015 – At the Lab


This year I attended the Agile Testing Days for the second time and I can say for sure that I will be attending in 2016. It is worth to return to this conference, because it has as delightful mix of topics across the Agile spectrum. The program offers technical talks and workshops that focus on coding, debugging, frameworks and test automation. It also features talks and workshops on Agile leadership and collaboration, with focus on human aspects, motivation and learning. And if offers sessions on many aspects of testing, such as exploratory testing, note-taking, modeling and the relation of testing to Agile methodologies. There is more than enough to go round for the modern tester.



From this year’s conference I took away some particular things. I spend quite some time in the Anything Build Party & TestLab, expertly hosted by James Lyndsay and Bart Knaack. I played around with CodeBug, which is a programmable and wearable device with 25 leds (see picture). The device can be programmed using a Blockly-based programming interface, which makes it easy to build controls for the LEDs. I spend a while thinking about what I would like to create and then got stuck halfway through coding. At that point I paired up with another participant trying to improve the program that I wrote. We seemed to run into the limits of the programming interface, but it was fun doing this together and we were both energized by the experience.

I also finally took a serious look at some black box puzzles, created by James Lyndsay. At the Belgium Testing Days last year I struggled with a black box puzzle, turned into a physical puzzle by Altom. Now I wanted to look at the other puzzles. With some hints from James I found a satisfying description of the functional behavior of puzzle 8. After that success I picked up puzzle 7 which I was unable to fully figure out, also because the conference was drawing to an end.

I learned the following things from the puzzles.

  • Taking notes helps. Using the notes as a guideline, it is easier to explain your testing. At least, that was how I was able to explain my testing when people who stopped by asked what I had been testing so far. Taking notes in a notebook with a pen or pencil is quicker, much more flexible and much more intuitive than taking notes in the form of a mind map. I tried both the mind map and the pencil. Creating a mind map (even tough it appears to be very light weight tool), seemed to interrupt the flow of thoughts and the flow of testing exactly when I needed that flow. I chucked the mind map and grabbed the pencil. I always thought of a mind map as a ‘informal’ tool that enables creativity. I do so now to a lesser degree.Black box puzzle 8
  • Having a mental model helps you devise theories and experiments. The mental model is the idea you have about how the subject under test might function. Whether the idea is right or wrong doesn’t matter that much. I got pretty far in explaining the behavior of puzzle 8 using a wrong (but very useful!) mental model. The beauty is that once you have an idea about how an application might function, you can test that theory. Often this provides you with more information about its behavior and about the validity of your model. Having gained that information I was able to move to a more advanced model. I think I moved through several mental models during puzzle 7 and 8.
  • Frequent interaction with the subject under test is important. With regards to the puzzles I tried, I found it not very useful to sit back and philosophize a whole lot about the excercise. Reflection is needed in order to move ahead through the theories about the functioning of the device, but you need data to reflect on. Interacting with the subject under test makes you notice certain behavior and it will generates test data. This is the data that you need in order to build a theory. I observed people playing the dice game during the Agile Games Market on Wednesday evening and one participant in this game took long pauses (of up to a minute!) between ‘throws’. I was almost yelling ‘Come on, test, test, test!!’ to urge him to gather data faster. The brain needs something to work with.
  • And yet, taking a break helps. This has been mentioned time and again in testing literature and blog posts. It is true that when one detaches himself from the situation, usually the mind starts offering clues as to how the application functions. I took a cigarette break and it helped me. James told me a story about a man who tried one of his puzzles at a conference and got stuck. During the ensuing keynote he had an epiphany, went back to the lab and presented, in one go, the answer to the puzzle.
  • Use tools. Since puzzles 7 & 8 are about timing, it is nice to have a timer and use it. I took my smartphone and used the timer to at least get an idea of the time between the flashing of the red light. It helped me to be a bit more confident about the reproducibility of certain situations and it helped me to falsify (or confirm) my theories. Yet at my current assignment quite a number of testers seem to be very wary of using tools, whether it be a SQL query tool, a tool to transfers loan data from one database to another, to make notes, to call a SOAP interface, to view logs, to drive a GUI, to debug the environment etc… I think it is inexperience in using such tools that is keeping them.

Since I didn’t ‘solve’ puzzle 7, James gave me a final hint and he urged me to look at it at home. I will do this, but in the meantime I want to thank James for creating these puzzles, and James and Bart for running the testlab at so many conferences, making it possible for testers to reflect on their testing and to learn from it.

On Being Part of the Problem


He that is not with me is against me; and he that gathereth not with me scattereth abroad.

Matthew 12:30 King James Version

Two weeks ago I visited the Agile Testing Days in Potsdam. It is an awesome conference and if you have the opportunity to be there in 2015, do consider attending. I am sifting through the talks that I attended and the conversations that I had, and will try to come up with some form of retrospective. Today something struck me about ‘being agile’ that actually never occurred to me before. I think it was my visit to the Agile Testing Days that actually set the train of thought in motion that lead me to this realization.

Actually, the thing that struck is me probably as revealing as the fact that the earth rotates around the sun. And yet it takes an actual experience to really know what it means. The realization is that when you are a member of an Agile team, this implies that you are part of the solution for the problem that the team is trying to solve. It means that you are enlisted to throw all the capabilities you have at the problem. You may not have the capabilities to solve the problem entirely so that is why we have teams of people who complement each other. But you are part of the solution.

The thing that made me feel this was that I was pleading my case before a team member, inviting him to help, assist or guide me in whichever way possible. The way I actually said it was: “Please feel free at any time to correct my insights, come up with a better plan, teach me things that I do not know or do anything else to make this solution that we as a team are working on, a success.” I made this plea because this same person provided me with some critical remarks in the days before. Remarks such as “I do not think this is documented in the right way”, “I think there is a better way to report these results”, “I think there are persons who are in a better position to evaluate this situation” and “I think we should take another look at the decisions that we made.” Without expanding on what he actually thought.

So this plea was probably my final attempt to get this person on board, a final invitation to show me what was on his mind and how things could be improved. I probably could have handled a 45 minute rant detailing how he thought we should move forward from this. Yet the sphinx-like answer I got was that I should be asking him the right questions, based on which he might offer me some of his insights. Which is a game that was last played in ancient Greece.

In an Agile team, when you criticize the work of a team member and after that vigorously and consistently refrain from offering help, then you’re not part of the solution. And if you’re not part of the solution, you’re not part of the team.