Friday, February 26, 2010


Coverage helps us to know what we are not testing. It doesn't helps us much to know if we are testing well what we are testing.

There are discussions about the usefulness of coverage. For example see the summary of a discussion on the subject, where I took the idea that the only thing that coverage shows are the untested areas.

For my part, I did a rant (in Spanish) about using the % coverage as a metric, especially by people who learned the concept only as a byproduct of learning TDD.

But if we take into account the characteristics of coverage, it can be an important aid for the test activity (whoever do this activity).

There are many types of coverage. For example, test coverage on business objectives or requirements, on identified risks, on the code or the inputs and outputs of the program.

Code coverage is the most common coverage metric, probably because it is easier to measure than others. Even within the code coverage, there are many possible coverage: class, method, line, statement, decision, path, etc.

In any coverage metric, we first define the universe, and then we measured how many of these points are being covered by some test case.

For example, in line of code coverage, each line is a point. If that line is executed by running a test, this point is said to be covered.

You can take the percentage of coverage as the number of points covered over the total amount of points.

But as mentioned earlier, is more valuable to know the points not covered.

The tool

The tool jXmlCoverage measures coverage based on the XML used by the System Under Test (SUT).

Many SUT uses XML as input, output or configuration. For example, Web Service. In these SUT it is available, or you can create, an XSD that defines a contract that the corresponding XML must comply.

We are interested in the degree to which the tests exercise the different values in the XML, specially the values that are not used at all.

For this, we must define the universe that we want to measure. What we do is to decide for each element defined in the XSD, the interesting equivalence partitions. For example, for an integer, it could be positive, zero, and negative integers and values out of range. We call this a sub-domain. Each sub-domain is a point on which coverage is measured.

After the definition and configuration phase, we have a testing universe, on which we can measure coverage.

All the XML used by the tests are evaluated against the sub-domains, counting how many times a sub-domain is used (covered) by the test cases.

As a result, we can obtain the sub-domains that were not covered.

As in all coverage metrics, one must avoid the temptation to treat a sub-domain as a sub-domain covered well tested.

Tuesday, January 12, 2010

Scrum week in Buenos Aires

Tobias Mayer and Alan Cyment will facilitate a serie of activities between January 25 and 28.

January 25: Scaling Scrum (free) by Tobias
This is a full-day workshop that works as a 3rd day of a CSM. See an example.

January 26: CSP Q&A (free) by Alan and Tobias
Help for filling your CSP application form

January 27: The Spirit of Scrum (*) by Tobias
This full-day workshop will allow Scrum practitioners to reach the next level of Scrum by exploring some of these underlying foundations in a highly experiential way. It will consist of a series of interactive exercises and facilitated discussion designed to help participants not just understand, but embody these principles and values at a deep level.

January 28: Improvisation for agile teams (*) by Alan Cyment and Tobias
The skill of teamwork is complex, multi-faceted, and sometimes paradoxical. Team members need to create for themselves environments that are both safe and challenging, both structured and flexible. A good team member should be able to respect organization and embrace chaos; be part of the team, yet able to have the courage of their own, individual convictions; be able to plot a steady course and be willing to stray from it. Team members need to be present; they need to be dynamic; they need compassion and empathy - and they need to be agile.
Improv is very much a conscious, kinesthetic application of complexity theory to the creative process. The exercises illuminate the things individuals may do to halt the creative process and forward motion of the team. The exercises offer new, simple methods of basic communication that help to overcome these behaviors, and keep the future limitless. The exercises heighten a person's sense of "permission" to be absolutely available to the project and the group. And they provide a template for moving a project relentlessly forward in increments small and thorough enough to bring the whole group along.

(*) For cost and payment, please see the link below

More info and registration at (some of the information in Spanish)

Activities organized by Agiles Argentina, Agilar Argentina and Southworks

Note: Tobias' activities will be in English, Alan's will be in Spanish.