The transport of energetic charged particles in the heliosphere depends on the structure of the interplanetary magnetic field. The field has both a stochastic part due to turbulence and a causal part due to the regular motions of the plasma. These give rise to the variety of transport effects observed. The stochastic component affects mostly the random walk or diffusive motion of the particles whereas the regular field affects primarily the various drift motions. Of particular interest is the latitudinal transport, which has been constrained by Ulysses observations to the highest latitudes, and which is related closely to the latitudinal magnetic field, both the causal and stochastic components. The observations of corotating-interaction-region-accelerated, low-rigidity energetic particles on Ulysses are consistent with propagation in a stochastic field superimposed on a Parker spiral, giving some support to the concept. The observations would require a significant cross-field diffusion. More-recent observations of energetic particles from impulsive solar flare events show structure on a fine scale which seems also to reflect the field-line meandering. Large small-scale gradients normal to the local magnetic field are shown to be completely consistent with significant cross-field diffusion on larger timescales. A causal latitudinal magnetic field, such as that proposed by Fisk [1996>, will also enhance latitudinal transport. This is briefly discussed. ¿ 2001 American Geophysical Union