Abstract
This paper describes the development and application of a cognitively-based performance methodology for assessing human performance on computer-aided architectural design (CAAD) tasks. Two CAAD tasks were employed that were hypothesized to be different in terms of the underlying cognitive processes required for these tasks to be performed. Methods of manipulating task complexity within each of these tasks were then developed. Six architectural graduate students were trained on a commercially available CAAD system. Each student performed the two experimental design tasks at one of three levels of complexity. The data collected included protocols, video recordings of the computer screen, and an interactive script (time-stamped record of every command input and the computers textual response). Performance measures and methods of analysis were developed which reflected the cognitive processes used by the human during design (including problem-solving techniques, planning times, heuristics employed, etc.) and the role of the computer as a design aid. The analysis techniques used included graphical techniques, Markov process analysis, protocol analysis, and error classification and analysis. The results of the study indicated that some measures more directly reflected human design activity while others more directly reflected the efficiency of interaction between the computer and the human. The discussion of the results focuses primarily on the usefulness of the various measures comprising the performance methodology, the usefulness of the tasks employed including methods for manipulating task complexity, and the effectiveness of this system as well as CAAD systems in general for aiding human design processes.