We report geochemical and Hf-Pb-Sr-Nd isotopic analyses of a suite of picrites and associated high-MgO volcanic rocks (6--17 wt.% MgO) from the Kerguelen Archipelago, which are rare compared to other oceanic islands, to better constrain the nature and the origin of components present in the Kerguelen mantle plume source. The Sr and Nd isotopic compositions of the transitional to mildly alkalic picrites and high-MgO basalts closely match those of the 24 Ma mildly alkalic basalts from the Courbet Peninsula, whose compositions are considered to reflect the present geochemical expression of the enriched component of the Kerguelen mantle plume. However, linear trends in Pb isotopic compositions in the studied samples reflect involvement of a component with lower 206Pb/204Pb and 208Pb/204Pb than that inferred for the enriched Kerguelen plume. Contamination of the MgO-rich magmas by the Kerguelen Plateau cannot account for the observed Hf-Pb-Sr-Nd isotopic variations. Isotopic systematics in the picrites and the high-MgO basalts are inconsistent with simple binary mixing between two distinct end-members and indicate the presence of small-scale heterogeneities within the Kerguelen plume itself as has been observed in other hot spot environments such as Hawaii and Iceland. The 34 to 26 Ma Kerguelen plume-related basalts that formed when the archipelago was close to the ridge axis (~50 to 250 km) show geochemical evidence for significant involvement of a Southeast Indian Ridge (SEIR)-like source. In contrast, the 24--25 Ma mildly alkalic basalts from the eastern and southeastern parts of the archipelago, which erupted about 400 km away from the SEIR, and the picrites and high-MgO basalts from this study show little or no contribution from a SEIR-like component. Thus chemical interaction between the SEIR axis and the Kerguelen plume effectively ceased prior to 25 Ma.