The search continues for coronal environmental factors that determine whether a fast coronal mass ejection (CME) results in a solar energetic particle (SEP) event at 1 AU. From a plot of peak 20 MeV SEP intensities versus associated CME speeds we select for comparison two groups of fast, wide, western hemisphere CMEs observed with the LASCO coronagraph from 1998 to 2002. The SEP-rich CME group produced the largest observed SEP events, and the SEP-poor CME group produced the smallest or no observed SEP events. The major differences are that the SEP-rich CMEs are brighter and more likely to be streamer blowouts and to follow colocated CMEs within 12 or 24 hours. The SEP-poor CMEs are fainter and less likely to interact with streamers or to follow preceding colocated CMEs. Thus we confirm the recent result that the SEP event peak intensities are higher when the associated driver CMEs are preceded within a day by wide CMEs at the same locations. However, the enhanced brightness, and therefore mass, of the SEP-rich CMEs appears to be their most dominant characteristic and suggests that either large longitudinal and latitudinal extents or high densities are needed for fast CMEs to produce SEPs.