Isolating and Predicting Risks in Architectural Design - Meaningful Experiment Data

An on-line experiment was used to test hypothesis H5 (if Design Rule, Resource, and Use Case Containers are meaningful to architects and developers, they should help to better manage mitigations used in practice) and answer RQ2 in order to assess whether any of the three risk container types satisfy the final meaningful criterion M3 (help to better manage mitigations used in practice). The on-line experiment is based around an architecture review scenario for a toy architecture because performing an architecture review is an example of a mitigation that is commonly used in practice. The experiment provides qualitative and quantitative evidence about whether presenting an architecture as a series of container diagrams helps practitioners to locate error inducing flaws and impact changes during the review. Participants are randomly assigned to one of four different groups (control, Design Rule Container, Resource Container, and Use Case Container) and each group must undertake the review exercise using a series of container diagrams or a single control diagram according to their assigned group. Participants also answer an open question to capture their attitudes towards completing the exercise. Section 7.1 of the thesis describes the experiment designed to answer RQ2 in more detail. This csv file contains the participant responses collected during the experiment.