Sentiment Analysis on Verbal Data from Team Discussions As an Indicator of Individual Performance

1 online resource (112 pages) : PDF

2020

University of North Carolina at Charlotte

The inherently complex nature of evaluating teamwork calls for methods to measure and predict students’ performance and provide timely feedback. Analyzing students’ individual performance, particularly in low-stake teams, is a challenge since the main goal in such team settings is knowledge acquisition and social skill development rather than final artifact production. Students’ attitude plays an important role in their cognition process and is commonly measured by self-report tools or asynchronous communications in textual discussion platforms. These methods have certain drawbacks such as distracting the learning process, demanding time and commitment from students or lack of emotional awareness which reduces the reliability of such tools. Research suggests speech is the best way to measure attitude accurately since it captures behavior directly rather than self-report. This study focuses on operationalizing attitude constructs of affect, self-efficacy, and personality and analyzing their correlation with students’ performance in order to identify which attitude constructs serve as performance predictor metrics. Personality and self-efficacy are measured using standard self-report tools, and affective states are captured from students’ conversations as they work in low-stake teams in a CS1 active learning class and discuss course content with peers. The novelty of this research is its focus on students' speech in class and how to operationalize affective states from verbal conversations as a metric that is related to individual performance. We record students’ conversations during teamwork in multiple sessions throughout the semester. By applying Natural Language Processing (NLP) algorithms we conduct sentiment analysis to detect valence, polarity and multiple classes of affective states from the conversations. The result of data analysis shows that students with higher levels of positive sentiment during the semester had higher performance scores. The result of personality and self-efficacy self-report tools, however, does not indicate a statistically significant correlation with performance. This outcome supports the research argument that self-report tools are not reliable in capturing attitude constructs. The result of data analysis in this study helps in identifying the attitudinal components that are correlated with performance to be applied in developing predictive models.

doctoral dissertations

Computer science
Education

Ph.D.

Atitude Constructs
CS Education
Natural Language Processing
Sentiment Analysis
Speech Analysis
Team Discussion in Active Learning

Computer Science

Maher, Mary Lou

Dorodchi, Mohsen
Gero, John
Shaikh, Samira
Cukic, Bojan
Schlueter, Jessica

Thesis (Ph.D.)--University of North Carolina at Charlotte, 2020.

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). For additional information, see http://rightsstatements.org/page/InC/1.0/.

Copyright is held by the author unless otherwise indicated.

Dehbozorgi_uncc_0694D_12521

http://hdl.handle.net/20.500.13093/etd:2364