This study examines how the asymmetry in the low-latitude geomagnetic field evolves during sheath and magnetic cloud domains of interplanetary coronal mass ejections using the Dst, SYMH and ASYH indices. For all investigated storms the station that contributed most to Dst was located in the dusk sector while the smallest disturbance field was concentrated around dawn. We found that sheath region storms are associated with larger morning/afternoon asymmetry than magnetic cloud storms. For sheath storms the station in the dusk sector contributed almost four times as much to Dst as the station in the dawn sector. Furthermore, the disturbance field is strongly variable during sheath storms. The results of this study suggest that for magnetic cloud storms the asymmetry arises mainly from ions drifting on open trajectories whereas in a case of a sheath driven storm the sudden intensifications of the substorm associated current systems add significantly to the asymmetry.