Algorithmic Optimization of First Convolution Layer in CNNs for Hardware Accelerator Design

Search for this publication on Google Scholar
Chamarthi, R. V. (2019). Algorithmic Optimization of First Convolution Layer in CNNs for Hardware Accelerator Design. Unc Charlotte Electronic Theses And Dissertations.
Download PDF

Analytics

147 views ◎
103 downloads ⇓


Abstract

This thesis proposes "1D Convolution replacement layer ", a novel optimization for first convolution layer in CNN. This optimization enables edge friendly streaming accelerator design with a minimum drop in accuracy. This optimization reduces the number of convolution parameters in the first convolution layers of CNN, reducing the number of multiplications performed in convolution operation. In CNNs first convolution is the most memory and compute intensive as the first layer operates on input. In a streaming accelerator design, the first layer operates on streaming data, the complexity of operations and memory demand of the first layer will proportionally affect the latency of complete accelerator design. Using 1D Convolution replacement in a CNN on a N x N convolution layer after 1D replacement number of operations in each convolution window gets reduced by N times. To show the effect of 1D convolution in accelerators, streaming accelerator design for SqueezeNet is compared with 1D-SqueezeNet, SqueezeNet with 1D convolution replacement in the first layer is discussed. 1D replacement enabled edge friendly design reducing the dynamic power consumption by 7.3X, with 0.6\% drop in accuracy in SqueezeNet real-time edge accelerator.

Details
Author

Chamarthi, Ramachandra Vikas

Title

Algorithmic Optimization of First Convolution Layer in CNNs for Hardware Accelerator Design

Physical Description

1 online resource (65 pages) : PDF

Date

2019

Degree Granting Institution

University of North Carolina at Charlotte

Abstract

This thesis proposes "1D Convolution replacement layer ", a novel optimization for first convolution layer in CNN. This optimization enables edge friendly streaming accelerator design with a minimum drop in accuracy. This optimization reduces the number of convolution parameters in the first convolution layers of CNN, reducing the number of multiplications performed in convolution operation. In CNNs first convolution is the most memory and compute intensive as the first layer operates on input. In a streaming accelerator design, the first layer operates on streaming data, the complexity of operations and memory demand of the first layer will proportionally affect the latency of complete accelerator design. Using 1D Convolution replacement in a CNN on a N x N convolution layer after 1D replacement number of operations in each convolution window gets reduced by N times. To show the effect of 1D convolution in accelerators, streaming accelerator design for SqueezeNet is compared with 1D-SqueezeNet, SqueezeNet with 1D convolution replacement in the first layer is discussed. 1D replacement enabled edge friendly design reducing the dynamic power consumption by 7.3X, with 0.6\% drop in accuracy in SqueezeNet real-time edge accelerator.

Genre

masters theses

Subjects--Topics

Electrical engineering
Computer engineering

Degree

M.S.

Keywords

CNN
Deep Learning
Hardware Accelerator
Machine Learning

Subject Area

Electrical Engineering

Advisor(s)

Tabkhi, Hamed

Committee Members

Willis, Andrew
Conrad, James

Degree Note

Thesis (M.S.)--University of North Carolina at Charlotte, 2019.

Rights Statement

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). For additional information, see http://rightsstatements.org/page/InC/1.0/.

Rights Holder Information

Copyright is held by the author unless otherwise indicated.

Identifier

Chamarthi_uncc_0694N_11979

Permalink

http://hdl.handle.net/20.500.13093/etd:2543