Impacts of connected and autonomous vehicles on deep reinforcement learning controlled intersection systems

Search for this publication on Google Scholar
Song, L. (2022). Impacts of connected and autonomous vehicles on deep reinforcement learning controlled intersection systems. Unc Charlotte Electronic Theses And Dissertations.
Download PDF

Analytics

13 views ◎
3 downloads ⇓


Abstract

Connected and autonomous vehicle (CAV) technologies could significantly change the car-following behaviors and affect the performance of the intersection systems. As it is expected to have a long transition time during which human driven vehicles (HDVs) and CAVs will coexist, it is important to investigate the impacts of CAVs on the intersection systems under different market penetration rates (MPRs). Also, the currently used Highway Capacity Manual does not consider the impacts of CAVs when calculating the intersection capacity. Though highly needed, a new guideline for estimating the intersection capacity under different MPRs of CAVs is becoming a critical issue for transportation planners and engineers. Furthermore, combining the intersection traffic signal control (TSC) systems with deep reinforcement learning (DRL) provides a new potential solution to improve the efficiency, safety, and sustainability of the intersection system. However, the training procedure of the DRL TSC system requires large samples and takes a long time to converge. Furthermore, it is common to have several intersections along corridors or in networks. A single DRL agent is unable to control several intersections as this may result in exponential explosion in the action space. Hence, a modification of the DRL TSC framework to improve the training efficiency and a multi-agent control framework to control several intersections are needed.To better prepare and guide both intersection planning and operations under different MPRs of CAVs and traffic demands, this dissertation provides an intensive evaluation of the impacts of CAVs in several signal intersection systems, as well as an in-depth analysis on intersection capacity adjustments that consider varying MPRs of CAVs. Also, a transfer-based DRL TSC framework is proposed and tested at different MPRs of CAVs and traffic demand levels. A multi-agent DRL TSC with shared traffic states between downstream and upstream intersections is investigated in a corridor. It is concluded that 100% MPR of CAVs can increase the saturation flow rate of the through-only lane by 126.8%. Meanwhile, transfer-based models could significantly improve training efficiency and model performance. The multi-agent DRL TSC also enables coordination between intersections. The insights of this research should be helpful and valuable to transportation researchers and traffic engineers in calculating intersection capacity, designing intelligent intersections, improving intersection efficiency, and implementing DRL-controlled traffic signals under the mixed flow with CAVs.

Details
Author

Song, Li

Title

Impacts of connected and autonomous vehicles on deep reinforcement learning controlled intersection systems

Physical Description

1 online resource (131 pages) : PDF

Date

2022

Degree Granting Institution

University of North Carolina at Charlotte

Abstract

Connected and autonomous vehicle (CAV) technologies could significantly change the car-following behaviors and affect the performance of the intersection systems. As it is expected to have a long transition time during which human driven vehicles (HDVs) and CAVs will coexist, it is important to investigate the impacts of CAVs on the intersection systems under different market penetration rates (MPRs). Also, the currently used Highway Capacity Manual does not consider the impacts of CAVs when calculating the intersection capacity. Though highly needed, a new guideline for estimating the intersection capacity under different MPRs of CAVs is becoming a critical issue for transportation planners and engineers. Furthermore, combining the intersection traffic signal control (TSC) systems with deep reinforcement learning (DRL) provides a new potential solution to improve the efficiency, safety, and sustainability of the intersection system. However, the training procedure of the DRL TSC system requires large samples and takes a long time to converge. Furthermore, it is common to have several intersections along corridors or in networks. A single DRL agent is unable to control several intersections as this may result in exponential explosion in the action space. Hence, a modification of the DRL TSC framework to improve the training efficiency and a multi-agent control framework to control several intersections are needed.To better prepare and guide both intersection planning and operations under different MPRs of CAVs and traffic demands, this dissertation provides an intensive evaluation of the impacts of CAVs in several signal intersection systems, as well as an in-depth analysis on intersection capacity adjustments that consider varying MPRs of CAVs. Also, a transfer-based DRL TSC framework is proposed and tested at different MPRs of CAVs and traffic demand levels. A multi-agent DRL TSC with shared traffic states between downstream and upstream intersections is investigated in a corridor. It is concluded that 100% MPR of CAVs can increase the saturation flow rate of the through-only lane by 126.8%. Meanwhile, transfer-based models could significantly improve training efficiency and model performance. The multi-agent DRL TSC also enables coordination between intersections. The insights of this research should be helpful and valuable to transportation researchers and traffic engineers in calculating intersection capacity, designing intelligent intersections, improving intersection efficiency, and implementing DRL-controlled traffic signals under the mixed flow with CAVs.

Genre

doctoral dissertations

Subjects--Topics

Transportation

Degree

Ph.D.

Keywords

Cav
Market Penetration Rates
Reinforcement Learning
Traffic Signal Control

Subject Area

Civil Engineering

Advisor(s)

Fan, Wei

Committee Members

Kane, Martin
Weggel, David
Wu, Jay

Degree Note

Thesis (Ph.D.)--University of North Carolina at Charlotte, 2022.

Rights Statement

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). For additional information, see http://rightsstatements.org/page/InC/1.0/.

Rights Holder Information

Copyright is held by the author unless otherwise indicated.

Identifier

Song_uncc_0694D_13162

Permalink

http://hdl.handle.net/20.500.13093/etd:3072