FIRST: Finding Interesting stoRies about STudents - An Interactive Narrative Approach to Explainable Learning Analytics
Analytics
12 views ◎4 downloads ⇓
Abstract
Learning Analytics (LA) has had a growing interest by academics, researchers, and administrators motivated by the use of data to identify and intervene with students at risk of underperformance or discontinuation. Typically, faculty leadership and advisors use data sources hosted on different institutional databases to advise their students for better performance in their academic life. Although academic advising has been critical for the learning process and the success of students, it is one of the most overlooked aspects of academic support systems. Most LA systems provide technical support to academic advisors with descriptive statistics and aggregate analytics about students' groups. Therefore, one of the demanding tasks in academic support systems is facilitating the advisors' awareness and sensemaking of students at the individual level. This enables them to make rational, informed decisions and advise their students. To facilitate the advisors' sensemaking of individual students, large volumes of student data need to be presented effectively and efficiently. Effective presentation of data and analytic results for sensemaking and decision-making has been a major issue when dealing with large volumes of data in LA. Typically, the students' data is presented in dashboard interfaces using various kinds of visualizations like scientific charts and graphs. From a human-centered computing perspective, the user's interpretation of such visualizations is a critical challenge to design for, with empirical evidence already showing that 'usable' visualizations are not necessarily effective and efficient from a learning perspective. Since an advisor's interpretation of the visualized data is fundamentally the construction of a narrative about student progress, this dissertation draws on the growing body of work in LA sensemaking, data storytelling, creative storytelling, and explainable artificial intelligence as the inspiration for the development of FIRST, Finding Interesting stoRies about STudents, that supports advisors in understanding the context of each student when making recommendations in an advising session. FIRST is an intelligible interactive interface built to promote the advisors' sensemaking of students' data at the individual level. It combines interactive storytelling and aggregate analytics of student data. It presents the student's data through natural language stories that are automatically generated and updated in coordination with the results of the aggregate analytics. In contrast to many LA systems designed to support student awareness of their performance or support teachers in understanding the students' performance in their courses, FIRST is designed to support advisors and higher education leadership in making sense of students' success and risk in their degree programs. The approach to interactive sensemaking has five main stages: (i) Student temporal data Model, (ii) Domain experts' questions and queries, (iii) Student data reasoning, (iv) Student storytelling model, and (v) Domain experts' reflection. The student storytelling stage is the main component of the sensemaking model and it composes four tasks: (i) Data sources, (ii) Story synthesis, (iii) Story analysis, and (iv) User interaction. The contributions of this dissertation are: (i) A novel student storytelling model to facilitate the sensemaking of complex, diverse, and heterogeneous student data, (ii) An anomaly detection model to enrich student stories with interesting, yet, insightful information for the domain experts and (iii) An explainable and interpretable interactive LA model to inspire advisors' trust and confidence with the student stories. This study reports on four ethnographic studies to show the potential of the proposed LA sensemaking model and how it affects the advisor's sensemaking of students at the individual level. The user studies considered for this dissertation were focus group discussions, in-depth interviews, and diary study- in-situ and snippet technique. These studies investigate if FIRST can improve and facilitate the advisor's sensemaking of students' success or risk by presenting individual student's heterogeneous data as a complete and comprehensive story.