With the increase in resolution and accuracy of gravity measurements it is becoming possible and increasingly important to concentrate on local analysis and on the role of elongated geophysical features. This paper deals with the statistical analysis of such features and develops for this purpose a new tool, the three- and four-point correlation functions. Using an integration over the manifold of the rotation group, expressions are derived for these functions in terms of the spherical harmonic coefficients; I also discuss the corresponding quantities for a flat earth and show how this approximation is recovered for small features. To gain an understanding of the structure of these functions, a simple, heuristic model is constructed in which the surface gravity field is expressed as a random superposition of elliptical elementary disturbances with random orientation. The three-point correlation function shows a characteristic behavior when the angle between its two vector arguments is of the order of the ratio of the axes of the ellipse. Another important piece of information contained in this function is the skewness of the gravity field.