The work of Nobel Laureates Banerjee, Duflo and Kremer has centered around the use of randomized control trials to help solve development problems. To date, however, few field experiments have been undertaken to evaluate the effects of humanitarian assistance. The reasons may lie in challenges related to logistics, fragility, security and ethics that often loom large in humanitarian settings. Yet every year, billions of dollars are spent on humanitarian aid, and policymakers are in need of rigorous evidence. In this paper, we reflect on the opportunities and risks of running experiments in humanitarian settings, and provide, as illustration, insights from our experiences with recent field experiments of large-scale humanitarian aid programs in the Democratic Republic of Congo.