Warming oceans are affecting the growth rates of fishes, with potential flow-on effects for population productivity and ecosystem functioning. Though we can make general predictions about how temperature affects fish growth, applying these to wild populations is difficult. This is because the relationship between temperature and growth can differ depending on whether we are looking at spatial variations in temperature across different locations, or temporal warming trends. Additionally, these effects may shift as fish develop from juveniles to adults. Here, we investigate long-term patterns in the thermal sensitivity of age-dependent growth across 33 temperate fish species from a ‘warming hotspot’, southeastern Australia, using otolith-derived growth chronologies. Across species, average annual growth was generally faster in warmer conditions, especially in young fish. Specifically, growth was faster in warmer years at colder sites, while responses to ocean warming at warmer sites were weak or absent, particularly in older fish. We show that warming effects on fish growth depend on both ontogeny and thermal context, and that responses across spatial temperature gradients can aid in understanding the direction of warming responses, but not the magnitude of local responses to warming through time. Understanding how fish growth responds to rising ocean temperatures is a first step in our ability to predict the impacts of future climate change on population dynamics and assist us in developing effective management strategies to maintain healthy ecosystems.