Neural networks are a class of machine learning models which have recently revolutionized many applied machine learning domains such as natural language understanding, image/video processing, bioinformatics, time series analysis. This course teaches students to develop new neural network architectures from scratch and customize them. The course covers all necessary foundations of neural networks including gradient descent optimization and vector calculus. Students will learn how to design models using idioms such as observed variables, latent variables, gate variables and different functions as well as a wide range of state-of-the-art architectures as design examples. Students are expected to have background in data structures.