In recent years, machine learning systems, particularly deep learning architectures, transformer models, and large language models (LLMs), have achieved remarkable success across domains such as healthcare, law, and education. As these systems increasingly support or automate decision-making, the challenge of responsibly communicating their performance and limitations to both expert and non-expert users has grown more urgent. Misinterpretation of model outputs, especially by lay audiences, can lead to misguided trust, poor decisions, and serious consequences. This dissertation investigates how data visualizations and contextual framing influence user attitudes, comprehension, confidence, and decision-making in AI-assisted contexts.Three empirical studies investigate how specific visualization techniques and recommendation sources influence human perception and interaction with data visualizations. The first study explores how eliciting prior beliefs and presenting contrasting narratives alongside data visualizations affect user engagement, recall accuracy, and attitude change in a news media setting. While belief elicitation increased engagement and reduced recall error in some conditions, it did not significantly shift attitudes. In contrast, narratives that boosted interest occasionally impaired memory of the focal data.The second study investigates how the presence and source of AI-generated recommendations influence decision-making in a content moderation task. Results show that participants suspended accounts with more offensive content and fewer reviews of evidence. AI recommendations led to faster but less confident decisions than human (crowd-sourced) recommendations, highlighting a trade-off between decision efficiency and perceived reliability.The third study examines the effectiveness of various representations of confusion matrix visualizations in communicating AI binary classification performance. By comparing classic tabular formats with Sankey diagrams and frequency-framed versions, the study shows that design choices significantly impact comprehension accuracy and confidence calibration among non-experts. Visualizations using frequency framing improved understanding and helped participants form more appropriately calibrated confidence in model performance.Together, these studies contribute to a deeper understanding of how visualization design and contextual framing shape user cognition and behavior in AI-supported environments. Together, these studies show that well-designed visualizations can meaningfully enhance non-experts’ ability to understand and appropriately use AI systems. The findings offer practical recommendations for creating more accessible, transparent, and responsible human-AI interactions by supporting informed and calibrated decision-making among lay users.
