We investigate the physics of multiscale convection in Earth's mantle, characterized by the coexistence of large-scale mantle circulation associated with plate tectonics and small-scale sublithospheric convection. In this study, conditions for the existence of small-scale convection beneath oceanic lithosphere are investigated by deriving a scaling law for the onset of convection in a fluid whose surface is instantaneously cooled. We employ two dimensional finite element convection modeling to solve this intrinsically time dependent problem for the Rayleigh number of 105--107 and with a range of temperature-dependent viscosity. Two different forms of temperature dependency, the Arrhenius law and its linear exponential approximation, are used. We present a new scaling analysis, on the basis of the concept of a differential Rayleigh number, to derive a general scaling law covering from constant viscosity to strongly variable viscosity. Compared to previous studies, our scaling law predicts significantly shorter onset time when applied to Earth's mantle. Possible reasons for this discrepancy are also discussed.