In this talk I will focus on the possibilities that arise from recent advances in the area of deep learning for accelerating and improving physics simulations. I will focus on fluids, which encompass a large class of materials we encounter in our everyday lives. In addition to being ubiquitous, the underlying physical model, the Navier-Stokes equations, at the same time represent a challenging, non-linear advection-diffusion PDE that poses interesting challenges for deep learning methods. I will explain and discuss several research projects from our lab that focus on temporal predictions of physical functions, temporally coherent adversarial training, and predictions of steady-state turbulence solutions. Among other things, it turns out to be useful to make the learning process aware of the underlying physical principles. Here, especially the transport component of the Navier-Stokes equations plays a crucial role. I will also give an outlook about open challenges in the area of deep learning for physical problems. Most importantly, trained models could server as priors for a variety of inverse and control problems.