Particulate organic carbon (POC) sinking out of the sunlit euphotic zone at the surface of the ocean feeds the deep sea and alters the CO2 concentration of the atmosphere. Most of the sinking POC is reoxidized to dissolved inorganic carbon (DIC) before it hits the sea floor, but the mechanism for this is poorly understood. Here we develop a global model of the microbial loop in the aphotic zone based on new measurements of deep ocean bacterial metabolism. These together imply that a significant fraction of the decreasing POC flux with depth is converted to dissolved organic carbon (DOC) rather than directly to DIC as is commonly assumed, thereby providing the substrate for free-living bacteria in the deep ocean. The model suggests the existence of a substantial DOC-pool with a relatively fast turnover time in the deep sea. By implementing the microbial loop in a model of the global ocean circulation, we show that the observed gradient of DOC in the deep North Atlantic can be explained by the temperature dependence of bacterial metabolic activity in conjunction with the formation of deep-water at high latitudes.