An ~6-year variation in the length of day has been observed for which there is no corresponding signal in records of atmospheric angular momentum. We find that this signal can be explained by an exchange of angular momentum with the core arising from gravitational coupling between the mantle and inner core. We develop a theoretical model of the core-mantle system that includes a combination of gravitational and electromagnetic couplings that allow angular momentum to be transferred between the solid inner core, fluid outer core, and mantle of the Earth. This model is used to calculate the natural period of interannual oscillations in the axial rotation of the core and mantle. The period and quality of the free oscillations depend on physical properties of the core and mantle. Time-varying geodynamo processes produce torques on the inner core which are capable of exciting the predicted free oscillations to observable levels. The observed length-of-day variation is used to constrain the strength of gravitational coupling between the mantle and inner core and hence the degree 2, order 2 component of equipotential surfaces at the inner core boundary and core mantle boundary. This constraint has implications for models of mantle flow.