We discuss a possible existence of small-scale convection under the back arc, which may be a cause of recently proposed along-arc variation of thermal structure under northeastern Japan. Based on numerical simulations, we find that, under the situation of opening of Japan Sea, a possible existence of low viscosity layer (LVL) below the back arc makes the existence of small-scale convection be likely. A shear produced by the movement of subducting slab aligns the axes of convection roll perpendicular to the arc. To explain the observed wavelength of along-arc variation (~50 to 80 km), our preferred model requires a similar thickness of the LVL whose viscosity is two-orders of magnitude less than that of surrounding. Another notable feature is the upward movement of the mantle associated with the small-scale convection. We discuss the water released from the subducting crust as a possible origin of the LVL.