We present the first relativistic electron pitch-angle and momentum diffusion rates for scattering by whistler-mode waves in the low density regieme. Diffusion rates are strongly dependent on the ratio between the electron plasma and gyro-frequencies ωpe/Ωe. For conditions typical of storm times, diffusion rates at a few MeV increase by more than 3 orders of magnitude as ωpe/Ωe is reduced from 10 to 1.5. Diffusion rates are extremely sensitive to energy and become ineffective above 3 MeV. At energies below 100 keV pitch-angle diffusion approaches strong diffusion loss to the atmosphere, while loss at higher energies is much weaker. For storm-time whistler-mode chorus amplitudes near 100 pT, and ωpe/Ωe ≤ 2.5, acceleration timescales can be less than a day at 1 MeV. This indicates that chorus diffusion could provide an important mechanism for local acceleration during the recovery phase of storms outside the plasmapause.