We investigate the use of multiple simultaneous solar wind plasma compositional anomalies, relative to the composition of the ambient solar wind, for identifying interplanetary coronal mass ejection (ICME) plasma. We first summarize the characteristics of several solar wind plasma composition signatures (O7/O6, Mg/O, Ne/O, Fe charge states, He/p) observed by the ACE and Wind spacecraft within the ICMEs during 1996--2002 identified by Cane and Richardson <2003>, hereafter CR03. We then develop a set of simple criteria that may be used to identify such compositional anomalies and hence potential ICMEs. To distinguish these anomalies from the normal variations seen in ambient solar wind composition, which depend on the wind speed, we compare observed compositional signatures with those expected in ambient solar wind with the same solar wind speed. This method identifies anomalies more effectively than the use of fixed thresholds. The occurrence rates of individual composition anomalies within ICMEs range from ~70% for enhanced iron and oxygen charge states to ~30% for enhanced He/p (>0.06) and Ne/O and are generally higher in magnetic clouds than other ICMEs. Intervals of multiple anomalies are usually associated with ICMEs and provide a basis for the identification of the majority of ICMEs. We estimate that CR03, who did not refer to composition data, probably identified ~90% of the ICMEs present. However, around 10% of their ICMEs have weak compositional anomalies, suggesting that the presence of such signatures does not provide a necessary requirement for an ICME. We note a remarkably similar correlation between the Mg/O and O7/O6 ratios in hourly-averaged data within both ICMEs and the ambient solar wind. This universal relationship suggests that similar processes produce the first-ionization potential bias and enhanced ion freezing-in temperatures in the source regions of both ICMEs and the ambient solar wind.