An experiment on the cognitive complexity of code

Hansen, M. E., Lumsdaine, A., & Goldstone, R. L. (2013).  An experiment on the cognitive complexity of code.  Proceedings of the Thirty-Fifth Annual Conference of the Cognitive Science Society. Berlin, Germany: Cognitive Science Society.

What simple factors impact the cognitive complexity of code? We present an experiment in which participants predict the output of ten small Python programs. Even with such simple programs, we find a complex relationship between code, expertise, and correctness. We use subtle differences between program versions to demonstrate that small notational changes can have profound effects on comprehension. We catalog common errors for each program, and perform an in-depth data analysis to uncover effects on response correctness and speed.

Download PDF version of this paper

Cognitive Architectures: A Way Forward for the Psychology of Programming

Hansen, M. E., Lumsdaine, A., & Goldstone, R. L. (2012).  Cognitive Architectures: A Way Forward for the Psychology of Programming.  Onward! Workshop at the Third Annual SPLASH Conference 2012.

Programming language and library designers often debate the usability of particular design choices. These choices may impact many developers, yet scientific evidence for them is rarely provided. Cognitive models of program comprehension have existed for over thirty years, but lack quantitative definitions of their internal components and processes. To ease the burden of quantifying existing models, we recommend using the ACT-R cognitive architecture: a simulation framework for psychological models. In this paper, we provide a high-level overview of modern cognitive architectures while concentrating on the details of ACT-R. We review an existing quantitative program comprehension model, and consider how it could be simplified and implemented within the ACT-R framework. Lastly, we discuss the challenges and potential benefits associated with building a comprehensive cognitive model on top of a cognitive architecture.

 Get PDF version of the paper