,5,1 OPTIMAL OPERATION OF THE SELECTIVE WITHDRAWAL SYSTEM IN TONO DAM RESERVOIR 1, Andrea Casteetti,3 Rodofo Soncini-Sessa 3 Hiroshi YAJIMA, Andrea CASTELLETTI and Rodofo SONCINI-SESSA 1 () 8-855 -11 Centre for Water Research 35, Stiring Hwy, Crawey WA 9, Austraia 3 Piazza Leonardo da Vinci, 3, I-133, Miano, Itay A reinforcement earning approach was deveoped and appied to design efficient management poicies for seective withdrawa system with the purpose of meeting estabished water quaity/quantity targets both in-reservoir and downstream. Structured design of experiment simuations was performed by a 1D couped hydrodynamic-ecoogica mode (DYRESM-CAEDYM) to generate a earning dataset over which a daiy management poicy was trained using a fitted-q agorithm based on extremey randomized trees. The approach was demonstrated on the management of Tono Dam reservoir, which is now under construction. Preiminary resuts indicated that a potentia great contro over reservoir imnoogy and reease quaity can be gained by effectivey expoiting - through the management poicy - the operationa fexibiity provided by the seective withdrawa structures. Key Words : Tono dam reservoir, seective withdrawa system, optima operation, fitted-q earning, DYRESM-CAEDYM 197 SWS 1) ) SWS 3) ) 5) Stochastic Dynamic Programming (SDP) ) SDP 19 (deterministic) Dynamic Programming (DP) 7) 8) 9) 198 SDP 1) 11) SDP 1) 13 Incrementa DP 1)
SDP SDP 15) 1). Casteetti Q 17).5 1 (km) fitted-qq 18 fitted-q fitted-q SDP F = x { t, u t x t+1 g t+1, = 1,,#F} (1) fitted-q #Ffitted- Q x t u t h Q * h (x t, u t ) * * Q h (x t, u t ) = g(x t, u t ) + γ max Q h 1 (x t+1, u t+1 ) u t+1 () t =, 1,, 3 t state x t S (S ) u t A(x t ) 1Q * h-1 (x t, u t ) Q * h (x t, u t )<x t, u t >, = 1,, #F s x s u Q * h fitted-q F Q * h Q * h-1 (A(x t ) )1 g t+1 R (R ) x t+1 N
375m 9m1m 3 38.1km.km 19.8m ) xt ut 1 ut 5 u t u t = u 1 n t,, u t (3) u n t nswstt+1 nsws x t 1DYRESM- CAEDYM 3) 3) ) 3m 13m 3m13m DYRESM-CAEDYM 3m13m u t 3m 13m Tota Suspended Soid (TSS) 5x t 3 1J sed TSS J env J irr h 1 out E TSS t+1 ε 1 ε h 1 t= h 1 E T out t+1 T in t+1 ε 1 ε h 1 t= h 1 J sed = () J env = ( ) (5) J irr = E ε 1 ε h 1 ( w t r t+1 ) 1.3 + () t= th 35TSS out TSS T out T in wr 535.9 m 3 /s551.795 m 3 /s9.98 m 3 /s939.5 m 3 /s955.39 m 3 /s
ut + 3 J sed J env J irr.1.1.8 3 x t, u t x t+1 g t+1 < t, xt, ut, t+1, xt+1, gt+1 > 3) ) 199 1995 SWS MatabMathWorks <u t > DYRESM-CAEDYMFortran Matab3 fitted-q fitted-q s x s u C++ DYRESM-CAEYDM DYRESM-CAEYDM 118 m3 /s 1 3 5year 1 3 5year (a) (b) SWS 3m (c) SWS 13m m3 /s (d) 1 3 5year 1 3 5year (e) SWS 3m (f) SWS 13m 3m13m 199199 53m13m 1 3 5year 1 3 5year () 3m 3m13m 5.%3m 9.9%13m.7% 3. 13m3m J env 3m J sed TSS13m
ο (a) 3m13m TSS (a) J sed (b) 3m13mTSS (b) J env (a) (b) (c) J irr J sed J env J irr (a) (b) 5 TSS J irr 3 J sed J sed.8 g/m 3.38 g/m 3 J env.1. J irr.3 m 3 /s.3 m 3 /s J sed TSS 11SWS J env.1. 199
J irr 11 fitted-q 1) Fortran1 DYRESM-CAEDYM fitted-q C++ ) 3 3) 11 Enrico WeberGiovanni Garbarini Centre for Water Research3-HY 1) :,, No.7, pp.1-33, 3. ) :, 11, pp.3-37,. 3) 9pp.1135-11, 5. ) 5. 5) Labadie, J.: Optima operation of mutireservoir systems: State-ofthe-art review, Journa of Water Research Panning and Management - ASCE, vo.13 (), pp.93 111,. ) Soncini-Sessa, R., A. Casteetti, and E. Weber: Integrated and participatory water resources management. Theory, Esevier, Amsterdam, 7. 7) Ha, W., and N. Buras: The dynamic programming approach to water resources deveopment, Journa of Geophysica Research, vo. (), pp.51 5, 191. 8) Ha, W., W. Butcher, and A. Esogbue: Optimization of the operation of a muti-purpose reservoir by dynamic programming, Water Resources Research, vo. (3), pp.71 77, 198. 9) Trott, W., and W. Yeh: Optimization of mutipe reservoir systems, Journa of the Hydrauic Division ASCE, vo.99, pp.185 188, 1973. 1) Yakowitz, S.: Dynamic programming appications in water resources, Water Resources Research, vo.8 (), pp.73 9, 198. 11) Yeh, W.: Reservoir management and operations modes: a state of the art review, Water Resources Research, vo.1 (1), pp.1797 1818, 1985. 1) Beman, R.: Dynamic Programming, Princeton University Press, Princeton, 1957. 13) Bertsekas, D., and J. Tsitsikis: Neuro-Dynamic Programming, Athena Scientific, Boston, 199. 1) Larson, R.: State Incrementa Dynamic Programming, American Esevier, New York, 198. 15) Kaebing, L., M. Littman, and A. Moore: Reinforcement Learning: a survey, Journa of Artificia Inteigence Research, vo., pp.37 85, 199. 1) Gosavi, A.: Simuation-based optimization: parametric optimization techniques and reinforcement earning, Kuwer Academic Pubishers, Boston, 3. 17) Casteetti, A., G. Corani, A. Rizzoi, R. Soncini-Sessa, and E. Weber: A reinforcement earning approach for the operationa management of a water system, in Proceedings of IFAC Workshop Modeing and Contro in Environmenta Is-sues, August -3, Esevier, Yokohama, 1. 18) Ernst, D., P. Geurts, and L. Wehenke: Tree-based batch mode reinforcement earning, Journa of Machine Learning Research, vo., pp.53 55, 5. 19) Sutton, R. S. and A.G.Barto ():.