We analyze the response of relativistic electrons to the 276 moderate and intense geomagnetic storms spanning the 11 years from 1989 through 2000. We find that geomagnetic storms can either increase or decrease the fluxes of relativistic electrons in the radiation belts. Surprisingly, only about half of all storms increased the fluxes of relativistic electrons, one quarter decreased the fluxes, and one quarter produced little or no change in the fluxes. We also found that the pre-storm and post-storm fluxes were highly uncorrelated suggesting that storms do not simply pump up the radiation belts. We found that these conclusions were independent of the strength of the storm (minimum Dst) and independent of L-shell. In contrast, we found that higher solar wind velocities increase the probability of a large flux increase. However, for all solar wind velocities both increases and decreases were still observed. Our analysis suggests that the effect of geomagnetic storms on radiation belt fluxes are a delicate and complicated balance between the effects of particle acceleration and loss.