Flow Diagram | Testing Goals | Scenarios
| Results Sheet | Consolidation | Solution Brainstorming
Our testing methodology is largely structured around the prototype iterations. Getting design information from
our prototype, the sequence of steps in the user testing process generates new or revised ideas, which in turn
provides feedback to the redesign iterations. Think-aloud sessions are the main usability testing method we used for our
testing. Heuristic evaluation is also performed on some prototypes to eliminate any glaring usability problems before
the think-aloud sessions.
After designing each prototype, we identify specific areas of the design and formulate testing goals based on
these features. These goals set a focus to each testing session, and help testers identify critical incidents
specific to these goals.
The scenarios were constructed to reflect the graphing needs of our actual users in the city. With the testing goals
in mind, we wrote the scenarios to give users motivation to use certain aspects of the interface during the
testing session. For man-on-the-street user testing, we used instructive scenarios which required the users to
accomplish certain tasks. When we user-tested with our actual users, we included scenarios which were more investigative
and required users to explore the metric data for interesting points in a free-form manner.
As we did the think-aloud user testing sessions, we realized that in addition to keeping track of whether certain tasks
are accomplished, we also had to take note of a lot of critical incidents. As a result, we created a Results Recording
sheet to record certain factual observations easily. The sheet consists of sets of requirements with checkboxes for
each scenario. It also includes certain features like 'Search' and 'Expand All' functions that we were interested in seeing
if the users used during the sessions. This Results Recording sheet has been very helpful for us to track the extent of tasks
that were accomplished.
After each user testing, we wrote Usability Aspect Reports (UARs) to record specific critical incidents from the
testing session. We also held consolidation meetings to categorize the problems and good incidents into the specific
areas of the design. This allowed us to manage a huge amount of information from each testing, and also helped us
focus on major design issues.
When we identified major redesign areas, we started brainstorming for various solutions for the problems. These
redesigns will drive the testing goals of the next user testing round.