It has been proposed that solar cycle irradiance variations may affect the whole planet's climate via the stratosphere, the Quasi-Biennial Oscillation (QBO) and Arctic Oscillation (AO). We test this hypothesis by examining causal links between time series of sunspot number and indices of QBO, AO and ENSO activity. We use various methods: wavelet coherence, average mutual information, and mean phase coherence to study the phase dynamics of weakly interacting oscillating systems. All methods clearly show a cause and effect link between Southern Oscillation Index (SOI) and AO, but no link between AO and QBO or solar cycle over all scales from biannual to decadal. We conclude that the 11-year cycle sometimes seen in climate proxy records is unlikely to be driven by solar forcing, and most likely reflects other natural cycles of the climate system such as the 14-year cycle, or a harmonic combination of multi-year cycles.