It is widely accepted that substantial relative motion has occurred between the Indo-Atlantic and Pacific hot spots since the Late Cretaceous. At the same time, a fixed Indo-Atlantic hot spot reference frame has been argued for and used since the advent of plate tectonics, implying relatively little motion between the hot spots in this domain since about 130 Ma. Most plumes purported to have caused these hot spots, while being advected in the global-scale mantle flow field, are assumed to move an order of magnitude more slowly than plates. However, the lifetime of a plume may be over ~100 Myr, and the integrated motion of a plume is expected to be significant over these times. The uncertainties inherent in hot spot reconstructions are of a magnitude similar to the expected plume motion, and so any differences between a fixed and moving frame of reference must be discernible beyond the level of these uncertainties. We present a method for constraining hot spot reconstruction uncertainties, similar to that in use for relative plate motion. We use a modified Hellinger criterion of fit for the hot spot problem, using track geometries and radiometric dating, and derive covariance matrices for our Indo-Atlantic rotations for the last 120 Myr. However, any given mantle convection model introduces additional uncertainties into such models, based on its model parameters and starting conditions (e.g., choice of global tomography model, viscosity profile, nature of mantle phase transitions). We use an interactive evolutionary approach, where we constrain the hot spot motion resulting from convection models to fit paleomagnetic constraints, and converge on an acceptable motion solution by varying unknowns over several generations of simulations. Our hot spot motion model shows large motion (5--10¿) of the Indo-Atlantic hot spots for times >80 Ma, consistent with available paleomagnetic constraints. The differences between the fixed and moving hot spot reference frames are not discernible over the level of uncertainty in such rotations for times <80 Ma.