Natural language processing has become a very popular tool in many areas and has also drawn great attention in the community of health informatics. It is a series of processes that allows informaticists to take advantage by extracting and understanding the information hidden in the unstructured text. Such a process can help clinicians making more accurate decisions, filtering more useful information, and better understand public health-related social norms. However, due to the uniqueness of the health-related content, the regular workflow of NLP has limited application because of the challenges in the annotation. The thesis primarily focuses on shortening the gap in annotation by integrating deep learning NLP approaches in the workflow to reduce the task in annotation, or to realize semi-automatic and automatic annotation in certain tasks. In this dissertation, I first present a deep learning-based phenotyping system that allows extraction of blood pressure readings from unstructured clinical notes. The workflow employs a pre-filtering approach that can reduce the workload in annotation, and can be applied in different domains. The second part presents an extractive text summarization system that utilizes the information in the abstract of scientific publications. The system uses a self-supervised approach that does not require any annotation while generating a classifier that can detect the content in the body text of the publication which should be extracted. In the third part, I proposed a workflow that performs info-surveillance on social media about COVID-19. By using a small group of annotated social media posts, the workflow will be able to monitor the trend and sentiment of the different topics being discussed on social media based on different times and locations.