01512nas a2200205   4500000000100000008004100001260001200042100001600054700001500070700001500085700001300100700002000113700002600133700001600159700001000175700002500185700001700210245007100227520100800298       2020                            d  c12/20201 aXinrun Wang1 aTarun Nair1 aHaoyang Li1 aYuh Wong1 aNachiket Kelkar1 aSrinivas Vaidyanathan1 aRajat Nayak1 aBo An1 aJagdish Krishnaswamy1 aMilind Tambe00aEfficient Reservoir Management through Deep Reinforcement Learning3 aDams impact downstream river dynamics through flow regulation and disruption of upstream-downstream linkages. However, current dam operation is far from satisfactory due to the inability to respond the complicated and uncertain dynamics of the upstream-downstream system and various usages of the reservoir. Even further, the unsatisfactory dam operation can cause floods in downstream areas. Therefore, we leverage reinforcement learning (RL) methods to compute efficient dam operation guidelines in this work. Specifically, we build offline simulators with real data and different mathematical models for the upstream inflow, ie, generalized least square (GLS) and dynamic linear model (DLM), then use the simulator to train the state-of-the-art RL algorithms, including DDPG, TD3 and SAC. Experiments show that the simulator with DLM can efficiently model the inflow dynamics in the upstream and the dam operation policies trained by RL algorithms significantly outperform the human-generated policy.