Integration of mathematical modeling with experimental investigations plays a key role in identifying recurring motifs and control mechanisms in biological systems, which is necessary for understanding disease mechanisms and development of new therapeutics. We will use three case studies to exemplify the utility of mathematical modeling. First, we will show that analyses of deterministic and stochastic models unravel design principles of signaling pathways in different organisms. To this end, we will examine the signaling pathway archetypes through their steady-state response profiles, response times, and abilities to process noise. Second, we will use a first-passage time framework to study hourglass-like timekeeping mechanisms that govern important cellular events like cell-division in bacteria and killing of a host bacterium by its virus, bacteriophage Finally, we will discuss our efforts to develop ab initio models of immune response in the plant Arabidopsis thaliana using an iterative wet-dry methodology.