Files
Abstract
The inherently complex nature of evaluating teamwork calls for methods to measure and predict students’ performance and provide timely feedback. Analyzing students’ individual performance, particularly in low-stake teams, is a challenge since the main goal in such team settings is knowledge acquisition and social skill development rather than final artifact production. Students’ attitude plays an important role in their cognition process and is commonly measured by self-report tools or asynchronous communications in textual discussion platforms. These methods have certain drawbacks such as distracting the learning process, demanding time and commitment from students or lack of emotional awareness which reduces the reliability of such tools. Research suggests speech is the best way to measure attitude accurately since it captures behavior directly rather than self-report. This study focuses on operationalizing attitude constructs of affect, self-efficacy, and personality and analyzing their correlation with students’ performance in order to identify which attitude constructs serve as performance predictor metrics. Personality and self-efficacy are measured using standard self-report tools, and affective states are captured from students’ conversations as they work in low-stake teams in a CS1 active learning class and discuss course content with peers. The novelty of this research is its focus on students' speech in class and how to operationalize affective states from verbal conversations as a metric that is related to individual performance. We record students’ conversations during teamwork in multiple sessions throughout the semester. By applying Natural Language Processing (NLP) algorithms we conduct sentiment analysis to detect valence, polarity and multiple classes of affective states from the conversations. The result of data analysis shows that students with higher levels of positive sentiment during the semester had higher performance scores. The result of personality and self-efficacy self-report tools, however, does not indicate a statistically significant correlation with performance. This outcome supports the research argument that self-report tools are not reliable in capturing attitude constructs. The result of data analysis in this study helps in identifying the attitudinal components that are correlated with performance to be applied in developing predictive models.