The availability of large amounts of labeled training data is a major contributing factor (and a bottleneck) to the recent progress in the field of Deep Learning. However, collecting and labeling data is a time consuming and expensive process. Oftentimes, the data cannot be collected due to privacy reasons or is just not available. This has led to an emergence of research in Few-Shot Learning, a new sub-domain of machine learning that focuses on building models that can learn from a few number of training examples. The recent progress in few-shot learning has shown promising results of achieving up to 90% accuracy on the task of 5-way image classification using just five training examples per class. The success of few-shot learning, however, is too much concentrated on image classification and the emergent field requires strict scrutiny. For example, 1) its behavior on speech data is unknown; 2) the nature of few-shot models to continually update when new category of data is witnessed remains untested; and 3) finally the privacy issues surrounding the data used in the few-shot model have been unaddressed. Therefore, this dissertation study develops few-shot model for audio data (Few-Shot Keyword Spotting), explores how few-shot learning models can continuously learn from the new incoming data (Few-Shot Continual Learning), and discusses how privacy can be an inbuilt part of few-shot learning (Privacy-Enhanced Few-Shot Learning).