Aftershocks rates seem to follow a power law decay, but the question of the aftershock frequency immediately after an earthquake remains open. We estimate an average aftershock decay rate within one day in southern California by stacking in time different sequences triggered by main shocks ranging in magnitude from 2.5 to 4.5. Then we estimate the time delay before the onset of the power law aftershock decay rate. For the last 20 years, we observe that this time delay suddenly increase after large earthquakes, and slowly decreases at a constant rate during periods of low seismicity. In a band-limited power law model such variations can be explained by different patterns of stress distribution at different stages of the seismic cycle. We conclude that, on regional length scales, the brittle upper crust exhibits a collective behavior reflecting to some extent the proximity of a threshold of fracturing.