The interactions between aerosols and solar radiation are determined by a combination of aerosol properties (i.e., types), surface properties (i.e., albedo) and clouds. These determining factors vary for different regions. We examine how these differences contribute to the impact of aerosols on the top-of-atmosphere (TOA) and surface radiation. In this study, the AERONET (Aerosol Robotic Network) aerosol climatology is used, in conjunction with surface albedo and cloud products from Moderate Resolution Imaging Spectroradiometer (MODIS), to calculate the aerosol direct radiative effect (ADRE) and its normalized form (NADRE). The NADRE is defined as the ADRE normalized by optical depth at 550 nm and is mainly determined by internal aerosol optical properties and geographical parameters. These terms are evaluated for cloud-free and cloudy conditions and for all-mode and fine-mode aerosols. Single-scattering albedo is an important variable determining ADRE of biomass burning. Because of stronger absorption by the smoke over South Africa the average NADRE over South America is ~35% larger at the TOA but ~38% smaller at the surface than that over South Africa. The surface albedo is another important factor in determining ADRE, especially for mineral dust. As the surface albedo varies from ~0.1 to ~0.35, it is observed that the dust NADRE ranges from -44 to -17 Wm-2 τ-1 at the TOA and from -80 to -48 Wm-2 τ-1 at the surface over the Saharan deserts, Arabian Peninsula, and their surrounding oceans. We also find that the NADRE of fine-mode aerosol is larger at the TOA but smaller at the surface in comparison to that of all-mode aerosol because of its larger single-scattering albedo and smaller asymmetry factor. Cloudy-sky ADRE is usually not negligible for the observed cloud optical thickness, but the TOA ADRE with clouds is sensitive to the relative location of aerosols and cloud layer.