Deep learning-based techniques have been widely employed in solving various medical image analytical problems. Currently, most of these methods directly employ deep architectures from natural image scenarios without considering the specific structures in the input/output variables, resulting in a suboptimal solution. In this dissertation, we systematically discuss deep structured learning in medical image analysis. Particularly, this dissertation is organized by answering the following questions: 1) how to model complex dependencies among the input/output variables with deep neural networks, 2) how to enforce prior structural knowledge in deep structured learning, and 3) how to model certain special structures in medical image analytical problems. More specifically, we first introduce a formal formulation of structured learning in medical imaging and present a general deep structured learning framework to address this problem. Second, we enforce the prior structural knowledge in the loss function to further improve the analytical performance. Third, as an example of special structures in medical imaging, we introduce how to model the tree structures in coronary arteries with tree-structured convolutional long short-term memory. Finally, we further introduce a special structured learning problem in medical imaging which involves sequential decision making. Accordingly, a deep reinforcement learning-based solution is proposed. To put our discussion in the context of medical imaging, we evaluated our approaches on several medical image analytical tasks, i.e., cardiac recognizing from MRI sequences, metastasis detection in whole-slide images (WSIs), coronary artery segmentation from 3D computed tomography angiography (CTA) volumes, and axon tracing. The superior performance demonstrates the effectiveness of the proposed methods.
