What is the relationship between the visual attributes of a rock sample and its permeability, and how might such information be used in aquifer/reservoir characterization? We investigated these questions by way of three meter-scale blocks of rock, two sandstones, and one volcanic tuff. Each rock exhibits visual and permeability patterns that differ in geometry, intensity, and scale. Using a gas minipermeameter, hundreds to thousands of permeability values were collected on grids at centimeter-scale resolution from each block face and compared to the digital visual image of the same face. We found no significant direct correlation between the permeability measurements and any objective textural measure drawn from the corresponding digital images, despite the obvious resemblance of the permeability maps and visual images. Hypothesizing that the human eye relies on spatial correlation to synthesize information, we calculated and compared semivariograms for both the maps and the images. Semivariogram shape, anisotropy ratio, and principal axes orientation were consistent for the maps and images of each rock but different between rocks. To explore this similarity further, we compared discrete features appearing in map/image pairs, using a difference of Gaussian edge filter to segment (i.e., delineate) the spatial features. The filter consistently colocated visually sharp features, such as sedimentological bounding surfaces, in both the map and the image. Additionally, cumulative distribution functions characterizing the size, shape, and orientation of the segmented features were statistically indistinguishable between a permeability map and its corresponding visual image but statistically different between rocks. These results suggest that it may not always be possible to use visual images (photographs or scans) of outcrops to directly map permeability, even with a training set of punctual permeability data. In particular, defining such relations may be difficult where the scales of analysis and/or intensity of heterogeneity are limited (e.g., within a given facies) as is the case here. Where direct mapping is not possible, these results suggest that visual images can still be used to delineate spatial permeability patterns, especially to locate boundaries where sharp permeability contrasts occur. Other surrogate measurements of permeability, in particular the various geophysical methods, may exhibit similar behavior.