5th Year Master's Thesis Presentation - Archan Das

— 2:30pm

Location:
In Person - Gates Hillman 7501

Speaker:
ARCHAN DAS , Master's Student
Computer Science Department
Carnegie Mellon University
https://www.linkedin.com/in/archan-das-801b92154/

Analyzing Student Debugging Using Programming Process Data

BACKGROUND: Debugging is an important part of the software development workflow. Understanding the cognitive process through which programmers debug is important for improving the techniques and instruction of debugging. Previous research has used a variety of methodologies for studying the debugging process, including concurrent verbal protocols, quantitative analyses, and neural imaging. One frontier in this research is the use of process data to study debugging. This process data consists of logs collected from integrated development environments (IDEs) that describes the process by which programmers work on code.

AIM: We aim to: a) create a framework for analyzing process data captured from an IDE, b) analyze the process data to observe patterns across student debugging behavior, and c) use the collected data to identify efficient and inefficient habits exhibited by students while debugging.

DATA: We collected process data across three exercises from 315 students in an introductory programming class. This data consists of an event log of every keystroke, code execution, and submission attempt students made while working on their exercises.

METHODS: We extracted a timeline of cognitive phases from the process data for each student and validated our model with a panel of experts. We tested the effect of two behavioral features (use of print statements and time in locate-error phase) as a novel measure of student struggle in debugging (count of run-program events). We also observed patterns across the subject population of our extracted cognitive phases

RESULTS: We found that the frequency of print statements had a positive correlation with debugging struggle across all exercises. Increased time spent in locate-error phase had a statistically significant impact on student debugging struggle in some exercises, but not others. Subjects tended to perform faster and more focused repairs to their code later in debugging episodes. Finally, debugging struggle had a weak negative correlation with average exam scores in the course.

CONCLUSION: Results suggest that students should be encouraged to spend more time reasoning about their code while debugging. Process data also shows promise as a tool for evaluating and giving feedback on the student debugging process. In addition, our framework can be widely useful for experiments on student debugging behavior.

Thesis Committee
David Kosbie (Chair)
Mark Stehlik
Roy Maxion

Additional Information 

For More Information:
tracyf@cs.cmu.edu

Add event to Google
Add event to iCal