The two most striking features of the martian surface topography are the Tharsis rise and the crustal dichotomy. Closely associated with Tharsis are several large volcanoes, active in the geologically recent past, indicating a longevity of volcanic activity at Tharsis that is unique in the solar system. Using analogue laboratory experiments we examine the dynamical effects of the crustal dichotomy and a layered mantle on the thermal evolution of Mars. We show that in combination these two effects lead to the formation of a large-scale upwelling under the southern highlands that appears early and endures for many billions of years. The upwelling comprises several persistent narrow plumes, which can explain the long-lived Tharsis volcanoes.