A significant and very similar annual variation in solar wind speed and in geomagnetic activity was recently found around all the four solar cycle minima covered by direct SW observations since mid-1960's. We have shown that the phase of this annual variation reverses with the Sun's polarity reversal, depicting a new form of 22-year periodicity. The annual variation results from a small north-south asymmetry in SW speed distribution where the minimum speed region is shifted toward the northern magnetic hemisphere. Here we study the very long-term evolution of the annual variation using early registrations of geomagnetic activity. We find a significant annual variation during the high-activity solar cycles in mid-19th century and since 1930's. Most interestingly, the SW speed asymmetry in mid-19th century was opposite to the present asymmetry, i.e., the minimum speed region was then shifted toward the southern magnetic hemisphere. This change of asymmetry suggests for a possible new form of century-scale oscillation in the north-south asymmetry of the Sun. We explain the asymmetry in terms of a relic magnetic field dislocated slightly in the north-south direction from the heliographic equator. The change in the asymmetry would result from the century-scale north-south oscillation of the location of the relic field across the ecliptic. ¿ 2001 American Geophysical Union