Relativistic electron fluxes in the outer radiation belt exhibit highly variable complex behavior. Previous studies have established a strong correlation of electron fluxes and the inner magnetospheric ULF waves in the Pc 3--5 frequency range. Resonant interaction of ULF waves with the drift motion of radiation belt electrons violates their third adiabatic invariant and consequently leads to their radial transport. If the wave-particle interaction has a stochastic character, then the electron transport is diffusive. The goal of this paper is to analyze the impact of toroidal ULF waves on radiation belt electrons. The study is based on direct measurements of ULF electric fields on the CRRES spacecraft. We show that the electric fields of inner magnetospheric toroidal ULF waves exhibit high asymmetry in magnetic local time and have narrow-band frequency spectra. Such narrow-band waves can induce radial diffusion of energetic electrons, if an extrinsic stochasticity is introduced in the system. The quasi-periodic variations in the solar wind dynamic pressure are identified as a possible source of extrinsic stochasticity. In the asymmetric magnetic field, drifting electrons can interact with both azimuthal and radial electric field components. We derive analytically and then calculate numerically the diffusion rates associated with azimuthal and radial electric field components of the waves. It is shown that even under highly disturbed geomagnetic conditions, when the background field asymmetry is large, the diffusion rates due to the radial field component are small. At the same time, the resonant scattering of energetic electrons by the azimuthal electric field of the waves provides an efficient form of radial diffusion and therefore can play an important role in the dynamics of the outer radiation belt.