A number of sedimentary records of relative paleointensity have been reported to contain modes of variability that could have been driven by changes in the Earth's orbital or climatic state. Proposed relationships are often based on visual correlation in the time domain or the presence of peaks in relative paleointensity frequency spectra close to those of the orbital components. When discussing hypothesized connections, it is not common for the phase relationship between paleointensity and orbital change to be addressed, so a true mechanistic link cannot be confirmed. The existence of a direct link between records of paleointensity and orbital/climatic variation is tested using the cross-wavelet transform and squared wavelet coherence. It is found that while all the records show common power at certain periods, the geomagnetic and orbital variations do not exhibit a consistent phase relationship, suggesting there is no direct physical link between them. This interpretation is supported by a lack of significant correlation in the time-frequency plane between the studied records as measured by the squared wavelet coherence.