Snapshot-Driven Deep Reinforcement Learning

1 online resource (111 pages) : PDF

2022

University of North Carolina at Charlotte

Deep reinforcement learning (DRL) has suggested many effective solutions to complex problems. Despite the impressive achievements of DRL, we have limited insights into why DRL is effective. DRL is also known as a black-box model with high complexity which makes DRL becomes difficult to be interpreted in a human-understandable way to discover the rationale behind DRL's predictions. Many papers have proposed different techniques intended to improve interpretability and measured the effect of different interpretability methods on user trust, the ability to simulate models, and the ability to detect mistakes. Understanding DRL through interpretation methods can unlock knowledge to improve the quality of prediction, understand internal errors to be able to fix, answer why a model does not work and improve the overall learning process.Most existing interpretation approaches only assume a converged model, which can't interpret the learning process of DRL and important concepts shaped. Therefore, the previous approaches are slow to produce a robust interpretation instantly. The sparsity of interpretation is another problem in the previous approaches because most of the known interpretation requires enormous human effort. Also, interpretation frameworks have not been used for supporting the learning process to have better performance. To address these challenges in DRL, a snapshot-driven approach, extracting a small number of examples for interpretation and using these snapshots for learning improvement, is proposed. Utilizing the snapshots, we can interpret the change in the learning process's behavior. The snapshots can also provide instant feedback to new samples for a faster understanding of the decisions of DRL. With the understanding of DRL, the snapshots can provide improvement to the learning methods in different settings. In order to fully accomplish the snapshot-driven learning, I propose methods 1) to extract snapshots from a DRL agent and understand the behavior, 2) to improve the learning process of the DRL agent and evaluate the effectiveness of the snapshots on it, 3) to reduce the number of snapshots for easier interpretation, and 4) to apply snapshots usage in continual DRL settings for latent stabilization to prevent catastrophic forgetting problem.

doctoral dissertations

Artificial intelligence

Ph.D.

Computer Science

Lee, Minwoo

Bunescu, Razvan
Terejanu, Gabriel
Rooshenas, Pedram
Uddin, Mesbah

Thesis (Ph.D.)--University of North Carolina at Charlotte, 2022.

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). For additional information, see http://rightsstatements.org/page/InC/1.0/.

Copyright is held by the author unless otherwise indicated.

Dao_uncc_0694D_13030

http://hdl.handle.net/20.500.13093/etd:2846