The decadal variation of thermohaline circulation (THC) is investigated in a simple coupled ocean-atmospheric two-box model with the new approach of conditional nonlinear optimal perturbation (CNOP). First, the nonlinear optimal initial perturbations with different constrains are found by this approach. There are two different types of perturbations in the nonlinear regime. One is the freshwater flux perturbation, which is the CNOP of THC and has stronger amplification. The other is salinity flux perturbation, whose amplification is weaker. Freshwater (salt) perturbations weaken (enhance) the mean circulation and hence weaken (enhance) the stability of THC. Second, the passive variabilities of THC are investigated by superposing the initial perturbations to the thermohaline circulation. The passive variabilities found in this model are due to nonnormal and nonlinear growth of initial perturbations. These variabilities, measured as recovering time of perturbations, can cause decadal variability of THC. The results of this paper suggest that CNOP approach is applicable to the investigation of the dependence of the THC sensitivity on the background climate state and is of potential application to the interpretation of past climate change linked to THC variations in nonlinear regime.