Enhanced Decision Making in Smart Grid Management by Optimizing Adaptive Multi-Agent Reinforcement Learning with Vehicle-to-Grid Systems

Pannee Suanpang; Pitchaya Jamjuntr

doi:10.31181/dmame7120241257

Authors

Pannee Suanpang Department of Information Technology, Faculty of Science & Technology, Suan Dusit University, Bangkok, Thailand
Pitchaya Jamjuntr Department of Electrical Engineering, Faculty of Engineering, King Mongkut's University of Technology Thonburi, Bangkok, Thailand

DOI:

https://doi.org/10.31181/dmame7120241257

Keywords:

Decision making; Smart grid management; Optimization, Multi-Agent Reinforcement Learning; Vehicle-to-Grid Systems

Abstract

This research proposes a decision-making framework in which the Adaptive Multi-Agent Reinforcement Learning (MARL) model and the concept of Vehicle-to-Grid (V2G) interactivity are employed to improve the effective management of smart grids. The research hypothesis introduces innovations for improving the efficiency and security of power systems in the global south, primarily by controlling the net energy transmission between the defined electric vehicles (EVs) and the grid. Other issues that require attention to ensure the proper functioning of smart grids include demand response, load management, and energy storage optimization. In this instance, these gaps are filled by the system’s proposed framework. With the help of MARL, the system dynamics' autonomous learning aspects allow the system to adapt to the capacity of renewable energy sources and electricity demand, which is also time-dependent. Because of the MARL, the autonomous coordination of decision-making has resulted in very positive changes in the system's effectiveness. In particular, this framework permitted an increase of 13.6% in the total energy exchange between EVs and the grid, and the grid stability index improved from 0.84 to 0.87 compared to what would have been achieved with the conventional methods. Enhanced energy management and pricing rehabs added another 22% to net savings. Further, it is stated that deploying MARL-based V2G systems in developing areas has many benefits, including more robust grid reliability and energy security and better integration of renewable energy resources. Such changes aid in reducing fossil fuel use and greenhouse gas emissions

Downloads

Download data is not yet available.

References

[1] Suanpang, P., & Jamjuntr, P. (2024). Optimizing electric vehicle charging recommendation in smart cities: A multi-agent reinforcement learning approach. World Electric Vehicle Journal, 15(2), 67. https://doi.org/10.3390/wevj15020067

[2] Bauer, C., Hofer, J., Althaus, H.-J., Simons, A., & Cox, B. (2021). The environmental performance of current and future passenger vehicles: Life cycle assessment based on a novel scenario analysis framework. Applied Energy, 291, 116766. https://doi.org/10.1016/j.apenergy.2021.116766

[3] Hardman, S., Jenn, A., Tal, G., & Axsen, J. (2021). The future of electric vehicles: Lessons from California’s zero-emission vehicle program. Energy Policy, 148, 111929. https://doi.org/10.1016/j.enpol.2020.111929

[4] Suanpang, P., Jamjuntr, P., Kaewyong, P., Niamsorn, C., & Jermsittiparsert, K. (2023). An intelligent recommendation for intelligently accessible charging stations: Electronic vehicle charging to support a sustainable smart tourism city. Sustainability, 15(1), 455. https://doi.org/10.3390/su15010455

[5] Muratori, M., Alexander, M., Arent, D., & Ward, J. (2021). The rise of electric vehicles—2020 status and future expectations. Progress in Energy, 3(2), 022002. https://doi.org/10.1088/2516-1083/abe0ad

[6] Chen, Z., Carrel, A. L., Gore, C., & Shi, W. (2021). Environmental and economic impact of electric vehicle adoption in the U.S. Environmental Research Letters, 16(4), 1–12. https://doi.org/10.1088/1748-9326/abe2d0

[7] Buhmann, K. M., & Rialp, J. (2022). Consumers' preferences for electric vehicles: The role of status and reputation. Transportation Research Part D: Transport and Environment, 114(1), 103530. https://doi.org/10.1016/j.trd.2022.103530

[8] Murali, N., M. V. P., and S. Ushakumari. "Electric Vehicle Market Analysis and Trends." 2022 IEEE 19th India Council International Conference (INDICON), Kochi, India, 2022, pp. 1-6. IEEE, doi:10.1109/INDICON56171.2022.10040056.

[9] Xu, X., Jia, Y., & Lai, C. S. (2020). A multi-agent reinforcement learning-based data-driven method for home energy management. IEEE Transactions on Smart Grid, PP(99), 1–1. https://doi.org/10.1109/TSG.2020.2971427

[10] Sarker, S., & Moore, R. (2021). Smart cities and electric vehicles: Integrating renewable energy and autonomous transportation. Journal of Urban Planning and Development, 147(1), 04020061. https://doi.org/10.1061/(ASCE)UP.1943-5444.0000657

[11] Okoh, A. S., & Onuoha, M. C. (2024). Immediate and future challenges of using electric vehicles for promoting energy efficiency in Africa’s clean energy transition. Global Environmental Change, 84, Roesch, M., Linder, C., Zimmermann, R., Rudolf, A., Hohmann, A., & Reinhart, G. (2020). Smart grid for industry using multi-agent reinforcement learning. Applied Sciences, 10(19), 6900. https://doi.org/10.3390/app10196900

[12] Anwar, A., Ali, A., & Ahmed, S. (2022). Vehicle-to-grid technology for energy management and grid support in developing countries. Journal of Sustainable Energy Systems, 15(4), 123–135.

[13] Timilsina, L., Moghassemi, A., Buraimoh, E., & Edrington, C. S. (2024). Impact of vehicle-to-grid (V2G) on battery degradation in a plug-in hybrid electric vehicle. In WCX SAE World Congress Experience. https://doi.org/10.4271/2024-01-2000

[14] Ferreira, R., Liu, X., & Martinez, A. (2020). Bidirectional energy flow and grid stabilization with vehicle-to-grid technology: A case study. Journal of Energy Engineering, 146(5), 04020063. https://doi.org/10.1061/(ASCE)EY.1943-7897.0000684

[15] Bibak, B., & Tekiner-Mogulkoc, H. (2021). Influences of vehicle to grid (V2G) on power grid: An analysis by considering associated stochastic parameters explicitly. Sustainable Energy, Grids and Networks, 26, 100429. https://doi.org/10.1016/j.segan.2020.100429

[16] Boodoo, C. (2024). Optimizing grid performance in Trinidad and Tobago: The role of vehicle-to-grid (V2G) technology. European Journal of Energy Research, 4(2), 36–43. https://doi.org/10.24018/ejenergy.2024.4.2.142

[17] Suanpang, P., & Jamjuntr, P. (2024). Machine learning models for solar power generation forecasting in microgrid application: Implications for smart cities. Sustainability, 16(14), 6087. https://doi.org/10.3390/su16146087

[18] Suanpang, P., & Jamjuntr, P. (2023). Optimizing tourism service intelligent recommendation system by multi-agent reinforcement learning for smart cities destination. Operational Research in Engineering Sciences: Theory and Applications, 6(3), 336–359. https://doi.org/10.31181/oresta/060317

[19] Roesch, M., Linder, C., Zimmermann, R., Rudolf, A., Hohmann, A., & Reinhart, G. (2020). Smart grid for industry using multi-agent reinforcement learning. Applied Sciences, 10(19), 6900. https://doi.org/10.3390/app10196900

[20] Luo, Y., Huang, C., & Feng, Z. (2022). Optimization of V2G operations with multi-agent systems: A review of current approaches. Journal of Cleaner Production, 367, 133237. https://doi.org/10.1016/j.jclepro.2022.133237

[21] Park, K., & Moon, I. (2022). Multi-agent deep reinforcement learning approach for EV charging scheduling in a smart grid. Applied Energy, 328, 120111. https://doi.org/10.1016/j.apenergy.2022.120111

[22] Han, C., & Han, H. (2021). Energy flow optimization in V2G systems: A decentralized approach. Journal of Power Sources, 501, 230–238. https://doi.org/10.1016/j.jpowsour.2021.230892

[23] Shah, P., Mehta, A., & Patel, S. (2023). Exploring the role of V2G in enhancing energy efficiency and sustainability in developing regions. Energy Reports, 16(3), 459–470. https://doi.org/10.1016/j.egyr.2023.01.018

[24] Goncearuc, A., De Cauwer, C., Sapountzoglou, N., Van Kriekinge, G., Huber, D., Messagie, M., & Coosemans, T. (2024). The barriers to widespread adoption of vehicle-to-grid: A comprehensive review. Energy Reports, 12, 27–41. https://doi.org/10.1016/j.egyr.2024.05.075

[25] Shi, R., Li, S., Zhang, P., & Lee, K. Y. (2020). Integration of renewable energy sources and electric vehicles in V2G network with adjustable robust optimization. Renewable Energy, 153, 1–10. https://doi.org/10.1016/j.renene.2020.02.027

[26] Zaino, R., Ahmed, V., Alhammadi, A. M., & Alghoush, M. (2024). Electric vehicle adoption: A comprehensive systematic review of technological, environmental, organizational, and policy impacts. World Electric Vehicle Journal, 15(8), 375. https://doi.org/10.3390/wevj15080375

[27] Abban, A. R., & Hasan, M. Z. (2021). Revisiting the determinants of renewable energy investment: New evidence from political and government ideology. Energy Policy, 151, 112184. https://doi.org/10.1016/j.enpol.2021.112184

[28] Kaewpasuk, S., Intiyot, B., & Jeenanunta, C. (2020). Impact of electric vehicles and solar PV on future Thailand’s electricity daily demand. International Scientific Journal of Engineering and Technology (ISJET), 4(1), 21–33. Retrieved from https://ph02.tci-thaijo.org/index.php/isjet/article/view/209020

[29] Srihari, G., Naidu, R. S. R. K., Falkowski-Gilski, P., Bidare Divakarachari, P., & Penmatsa, R. K. V. (2024). Integration of electric vehicle into the smart grid: A meta-heuristic algorithm for energy management between V2G and G2V. Frontiers in Energy Research, 12, Article 1357863. https://doi.org/10.3389/fenrg.2024.1357863

[30] Al-Dhaifallah, M., Ali, Z. M., Alanazi, M., Dadfar, S., & Fazaeli, M. H. (2021). An efficient short-term energy management system for a microgrid with renewable power generation and electric vehicles. Neural Computing and Applications, 33(23), 16095–16111. https://doi.org/10.1007/s00521-021-06247-5

[31] Chen, R.-F., Luo, H., Huang, K.-C., Nguyen, T.-T., & Pan, J.-S. (2022). An improved honey badger algorithm for electric vehicle charge orderly planning. Journal of Network Intelligence, 7(2), 332–346.

[32] Cheng, H., Wang, Z., Yang, S., Huang, J., & Ge, X. (2020). An integrated SRM powertrain topology for plug-in hybrid electric vehicles with multiple driving and onboard charging capabilities. IEEE Transactions on Transportation Electrification, 6(2), 578–591. https://doi.org/10.1109/TTE.2020.2987167

[33] Egbue, O., Uko, C., Aldubaisi, A., & Santi, E. (2022). A unit commitment model for optimal vehicle-to-grid operation in a power system. International Journal of Electrical Power & Energy Systems, 141, Article 108094. https://doi.org/10.1016/j.ijepes.2022.108094

[34] Gan, W., Wen, J., Yan, M., Zhou, Y., & Yao, W. (2024). Enhancing resilience with electric vehicles charging redispatching and vehicle-to-grid in traffic-electric networks. IEEE Transactions on Industry Applications, 60(1), 953–965. https://doi.org/10.1109/TIA.2023.3272870

[35] Grasel, B., Baptista, J., & Tragner, M. (2023). The impact of V2G charging stations (active power electronics) on the higher frequency grid impedance. In 2023 International Conference on Smart Energy Systems and Technologies (SEST) (pp. 1–6). IEEE. https://doi.org/10.2139/ssrn.4528053

[36] Ismail, A. A., Mbungu, N. T., Elnady, A., Bansal, R. C., Hamid, A.-K., & AlShabi, M. (2023). Impact of electric vehicles on smart grid and future predictions: A survey. International Journal of Modelling and Simulation, 43(6), 1041–1057. https://doi.org/10.1080/02286203.2022.2148180

[37] Justin, F., Peter, G., Stonier, A. A., & Ganji, V. (2022). Power quality improvement for vehicle-to-grid and grid-to-vehicle technology in a microgrid. International Transactions on Electrical Energy Systems, 2022, 1–17. https://doi.org/10.1155/2022/2409188

[38] Liu, J., Li, Y., Wang, Q., & Zhang, L. (2021). Integration of vehicle-to-grid technology in smart grid environments: A review. Renewable and Sustainable Energy Reviews, 136, 110193. https://doi.org/10.1016/j.rser.2020.110193

[39] Tushar, H., & Gupta, S. (2021). Challenges and barriers to the integration of renewable energy and electric vehicles: A review. Renewable and Sustainable Energy Reviews, 143, 110911. https://doi.org/10.1016/j.rser.2021.110911

[40] Zhang, C., Chen, X., & Zhang, H. (2021). A distributed multi-agent reinforcement learning approach for V2G integration in smart grids. Journal of Energy Storage, 45, 103598. https://doi.org/10.1016/j.est.2021.103598

[41] Liu, Y., & Zhang, X. (2021). Smart charging and discharging of electric vehicles using multi-agent reinforcement learning techniques. International Journal of Electrical Power & Energy Systems, 127, 106679. https://doi.org/10.1016/j.ijepes.2020.106679

[42] Gogineni, K., Wei, P., Lan, T., & Venkataramani, G. (2023). Scalability bottlenecks in multi-agent reinforcement learning systems. arXiv preprint arXiv:2302.05007. https://doi.org/10.48550/arXiv.2302.05007

[43] Kumar, M. P., Kumaraswamy, D., Jayanth, R., Rohith, K., Rakesh, G., Teja, S. M., & others. (2022). Vehicle-to-grid technology in a microgrid using DC fast charging architecture. Journal of Engineering Science, 13(07), 291–295.

[44] Mojumder, M. R. H., Ahmed Antara, F., Hasanuzzaman, M., Alamri, B., & Alsharef, M. (2022). Electric vehicle-to-grid (V2G) technologies: Impact on the power grid and battery. Sustainability, 14, Article 13856. https://doi.org/10.3390/su142113856

[45] Escoto, M., Guerrero, A., Ghorbani, E., & Juan, A. A. (2024). Optimization challenges in vehicle-to-grid (V2G) systems and artificial intelligence solving methods. Applied Sciences, 14(12), 5211. https://doi.org/10.3390/app14125211

[46] Liu, B., Lu, M., Shui, B., Sun, Y., & Wei, W. (2022). Thermal-hydraulic performance analysis of printed circuit heat exchanger precooler in the Brayton cycle for supercritical CO2 waste heat recovery. Applied Energy, 305, Article 117923. https://doi.org/10.1016/j.apenergy.2021.117923

[47] Zhou, Y., Liu, Y., & Zhao, J. (2022). Electric vehicles in smart cities: Integration with smart grids and autonomous systems. Sustainable Cities and Society, 78, 103592. https://doi.org/10.1016/j.scs.2022.103592

[48] Li, J., Shi, L., & Guo, Q. (2020). Government policies and the growth of the electric vehicle market in China. Energy Policy, 142, 111542. https://doi.org/10.1016/j.enpol.2020.111542

[49] Zhang, H., Liu, W., & Yang, X. (2021). Performance comparison of multi-agent reinforcement learning and traditional methods in vehicle-to-grid systems. Energy, 213, 118774. https://doi.org/10.1016/j.energy.2020.118774

[50] Bibak, B., & Tekiner-Moğulkoç, H. (2021). A comprehensive analysis of Vehicle to Grid (V2G) systems and scholarly literature on the application of such systems. Renewable Energy Focus, 36, 1–20. https://doi.org/10.1016/j.ref.2020.10.001

[51] Hannan, M. A., Mollik, M. S., Al-Shetwi, A. Q., Rahman, S. A., Mansor, M., Begum, R. A., Muttaqi, K. M., & Dong, Z. Y. (2022). Vehicle to grid-connected technologies and charging strategies: Operation, control, issues, and recommendations. Journal of Cleaner Production, 339, Article 130587. https://doi.org/10.1016/j.jclepro.2022.130587

[52] Nazir, M. S., Almasoudi, F. M., Abdalla, A. N., Zhu, C., & Alatawi, K. S. S. (2023). Multi-objective optimal dispatching of combined cooling, heating and power using hybrid gravitational search algorithm and random forest regression: Towards the microgrid orientation. Energy Reports, 9, 1926–1936. https://doi.org/10.1016/j.egyr.2023.01.012

[53] Maheriya, A., Raval, A. S., Panchal, S., & Borisagar, K. (2024). Empowering the future of smart grids: Unveiling the role of electric vehicles in V2G integration for sustainable infrastructure. In L. D. Lakshmi, N. Nagpal, N. Kassarwani, V. Varthanan G., & P. Siano (Eds.), E-Mobility in Electrical Energy Systems for Sustainability (pp. [page range]). IGI Global. https://doi.org/10.4018/979-8-3693-2611-4.ch011

[54] Aurangzeb, M., Xin, A., Iqbal, S., Afzal, M. Z., Kotb, H., AboRas, K. M., Ghadi, Y. Y., & Ngoussandou, B. P. (2023). A novel hybrid approach for power quality improvement in a vehicle-to-grid setup using droop-ANN model. International Journal of Energy Research, 2023, Article 7786928. https://doi.org/10.1155/2023/7786928

[55] Sovacool, B. K., Axsen, J., & Kempton, W. (2017). The future promise of vehicle-to-grid (V2G) integration: A sociotechnical review and research agenda. Annual Review of Environment and Resources, 42, 377–406. https://doi.org/10.1146/annurev-environ-030117-020220

[56] Lund, H., & Kempton, W. (2008). Integration of renewable energy into the transport and electricity sectors through V2G. Energy Policy, 36(9), 3578–3587. https://doi.org/10.1016/j.enpol.2008.06.007

[57] Guille, C., & Gross, G. (2009). A conceptual framework for the vehicle-to-grid (V2G) implementation. Energy Policy, 37(11), 4379–4390. https://doi.org/10.1016/j.enpol.2009.05.053

[58] Sarker, M. R., Halder, T. R., & Lo, K. L. (2018). Internet of things-enabled vehicle-to-grid systems for distributed renewable energy resources. Renewable and Sustainable Energy Reviews, 94, 705–719. https://doi.org/10.1016/j.rser.2018.06.037

[59] Kempton, W., & Tomić, J. (2005). Vehicle-to-grid power fundamentals: Calculating capacity and net revenue. Journal of Power Sources, 144(1), 268–279. https://doi.org/10.1016/j.jpowsour.2004.12.025

[60] Wang, Y., Zou, P., & Chen, M. (2022). Technological advancements and the economic feasibility of electric vehicles. Journal of Cleaner Production, 344, Article 130991. https://doi.org/10.1016/j.jclepro.2022.130991

[61] Clement-Nyns, K., Haesen, E., & Driesen, J. (2011). The impact of vehicle-to-grid on the distribution grid. Electric Power Systems Research, 81(1), 185–192. https://doi.org/10.1016/j.epsr.2010.08.007

[62] Lund, H., & Kempton, W. (2008). Integration of renewable energy into the transport and electricity sectors through V2G. Energy Policy, 36(9), 3578–3587. https://doi.org/10.1016/j.enpol.2008.06.007

[63] Gao, L., Liu, X., & Zhang, T. (2019). Socio-economic impacts of V2G technology in the context of sustainable energy transitions. Sustainability, 11(1), 1135. https://doi.org/10.3390/su11041135

[64] Chen, Q., Xie, R., Lu, H., Zheng, X., Hang, L., Fu, C., He, Z., He, Y., Zeng, P., & Qiu, J. (2022). Hybrid injection strategy of sub-module voltage ripple suppression for MMC under low-frequency operation. Energy Reports, 8(Supplement 5), 1454–1465. https://doi.org/10.1016/j.egyr.2022.03.002

[65] Chen, Q., Qiu, P., Xiang, Z., Wang, S., Pan, W., & Xie, H. (2021). The application of Distributed Power Flow Controller in Gan Quan–Xiang Fu 220 kV AC lines in Hu Zhou. Energy Reports, 7(Supplement 1), 210–215. https://doi.org/10.1016/j.egyr.2021.01.078

[66] Woltmann, S., & Kittel, J. (2022). Developing and implementing multi-agent systems for demand response aggregators in an industrial context. Applied Energy, 314, Article 118841. https://doi.org/10.1016/j.apenergy.2022.118841

[67] Guo, Y., Zhang, P., Ding, H., & Le, C. (2021). Design and verification of the loading system and boundary conditions for wind turbine foundation model experiment. Renewable Energy, 172, 16–33. https://doi.org/10.1016/j.renene.2021.03.017

[68] Zhang, Y., Lv, Y., & Ge, M. (2021). Time-frequency analysis via complementary ensemble adaptive local iterative filtering and enhanced maximum correlation kurtosis deconvolution for wind turbine fault diagnosis. Energy Reports, 7, 2418–2435. https://doi.org/10.1016/j.egyr.2021.04.045

[69] Bilgili, L. (2021). Comparative assessment of alternative marine fuels in life cycle perspective. Renewable and Sustainable Energy Reviews, 144, Article 110985. https://doi.org/10.1016/j.rser.2021.110985

[70] Nweye, K., Liu, B., Stone, P., & Nagy, Z. (2022). Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings. Energy and AI, 10, Article 100202. https://doi.org/10.1016/j.egyai.2022.100202

[71] Rehman, U., Faria, P., Gomes, L., & Vale, Z. (2023). Future of energy management systems in smart cities: A systematic literature review. Sustainable Cities and Society, 96, Article 104720. https://doi.org/10.1016/j.scs.2023.104720

[72] Stavrev, S., & Ginchev, D. (2024). Reinforcement learning techniques in optimizing energy systems. Electronics, 13(8), Article 1459. https://doi.org/10.3390/electronics13081459

[73] Mousa, A. (2023). Extended-deep Q-network: A functional reinforcement learning-based energy management strategy for plug-in hybrid electric vehicles. Engineering Science and Technology, an International Journal, 43, Article 101434. https://doi.org/10.1016/j.jestch.2023.101434

[74] Mwasilu, F., Justo, J. J., Kim, E.-K., Do, T. D., & Jung, J.-W. (2014). Electric vehicles and smart grid interaction: A review on vehicle-to-grid and renewable energy sources integration. Renewable and Sustainable Energy Reviews, 34, 501–516. https://doi.org/10.1016/j.rser.2014.03.031

[75] Yang, Y., Yang, W., Chen, H., & Li, Y. (2020). China’s energy whistleblowing and energy supervision policy: An evolutionary game perspective. Energy, 213, Article 118774. https://doi.org/10.1016/j.energy.2020.118774

[76] Coe, R. G., Ahn, S., Neary, V. S., Kobos, P. H., & Bacelli, G. (2021). Maybe less is more: Considering capacity factor, saturation, variability, and filtering effects of wave energy devices. Applied Energy, 291, Article 116763. https://doi.org/10.1016/j.apenergy.2021.116763

[77] Knox, S., Hannon, M., Stewart, F., & Ford, R. (2022). The (in)justices of smart local energy systems: A systematic review, integrated framework, and future research agenda. Energy Research & Social Science, 83, Article 102333. https://doi.org/10.1016/j.erss.2021.102333

[78] Jang, M.-J., & Oh, E. (2024). Deep-reinforcement-learning-based vehicle-to-grid operation strategies for managing solar power generation forecast errors. Sustainability, 16(9), Article 3851. https://doi.org/10.3390/su16093851

[79] Sadeghian, O., Oshnoei, A., Kheradmandi, M., Khezri, R., & Mohammadi-Ivatloo, B. (2020). A robust data clustering method for probabilistic load flow in wind-integrated radial distribution networks. International Journal of Electrical Power & Energy Systems, 115, Article 105392. https://doi.org/10.1016/j.ijepes.2019.105392

[80] Javaid, N., Javaid, S., Abdul, W., Ahmed, I., Almogren, A., Alamri, A., & Niaz, I. A. (2017). A hybrid genetic wind-driven heuristic optimization algorithm for demand side management in smart grid. Energies, 10(3), Article 319. https://doi.org/10.3390/en10030319

[81] Wu, C., Gao, S., Liu, Y., Song, T. E., & Han, H. (2021). A model predictive control approach in microgrid considering multi-uncertainty of electric vehicles. Renewable Energy, 163, 1385–1396. https://doi.org/10.1016/j.renene.2020.08.137

[82] Hosny, M., Kamel, S., Dominguez-Garcia, J. L., & Molu, R. J. J. (2023). Integrating renewable energy and V2G uncertainty into optimal power flow: A gradient bald eagle search optimization algorithm with local escaping operator. IET Renewable Power Generation. https://doi.org/10.1049/rpg2.12874

[83] He, Q., Yang, Y., Luo, C., Zhai, J., Luo, R., & Fu, C. (2022). Energy recovery strategy optimization of dual-motor drive electric vehicle based on braking safety and efficient recovery. Energy, 248, Article 123543. https://doi.org/10.1016/j.energy.2022.123543

[84] Singh, R. A., Kumar, R. S., Bajaj, M., & et al. (2024). Machine learning-based energy management and power forecasting in grid-connected microgrids with multiple distributed energy sources. Scientific Reports, 14, Article 19207. https://doi.org/10.1038/s41598-024-70336-3

[85] Perera, A. T. D., & Kamalaruban, P. (2021). Applications of reinforcement learning in energy systems. Renewable and Sustainable Energy Reviews, 137, Article 110618. https://doi.org/10.1016/j.rser.2020.110618

[86] Weil, J., Bao, Z., Abboud, O., & Meuser, T. (2024). Towards generalizability of multi-agent reinforcement learning in graphs with recurrent message passing. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). arXiv:2402.05027. https://doi.org/10.48550/arXiv.2402.05027

[87] Aladdin, S., El-Tantawy, S., Fouda, M.M., & Tag Eldien, A.S. (2020). MARLA-SG: Multi-Agent Reinforcement Learning Algorithm for Efficient Demand Response in Smart Grid. IEEE Access, 8, 210626-210639.

[88] Antonini, E. G. A., & Caldeira, K. (2021). Atmospheric pressure gradients and Coriolis forces provide geophysical limits to power density of large wind farms. Applied Energy, 281, Article 116048. https://doi.org/10.1016/j.apenergy.2020.116048

[89] Suresh, V., Muralidhar, M., & Kiranmayi, R. (2020). Modeling and optimization of an off-grid hybrid renewable energy system for electrification in rural areas. Energy Reports, 6, 594–604. https://doi.org/10.1016/j.egyr.2020.01.013

[90] Kim, S., Kim, K., & Son, C. (2020). Transient system simulation for an aircraft engine using a data-driven model. Energy, 196, Article 117046. https://doi.org/10.1016/j.energy.2020.117046

[91] Qi, C., Zhu, Y., Song, C., Yan, G., Xiao, F., Wang, D., Zhang, X., Cao, J., & Song, S. (2022). Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicles. Energy, 238, Part A, Article 121703. https://doi.org/10.1016/j.energy.2021.121703

[92] Ahadi, R., Ketter, W., Collins, J., & Daina, N. (2022). Cooperative learning for smart charging of shared autonomous vehicle fleets. Transportation Science, 57(3), 613–630. https://doi.org/10.1287/trsc.2022.1187

[93] Albogamy, F. R., Khan, S. A., Hafeez, G., Murawwat, S., Khan, S., Haider, S. I., Basit, A., & Thoben, K.-D. (2022). Real-time energy management and load scheduling with renewable energy integration in smart grid. Sustainability, 14(3), Article 1792. https://doi.org/10.3390/su14031792

[94] Martone, T., Iob, P., Schiavo, M., & Cenedese, A. (2024). Smart fleet solutions: Simulating electric AGV performance in industrial settings. Paper presented at the 2024 IEEE International Conference on Emerging Technologies and Factory Automation (ETFA2024). arXiv:2408.12498.

[95] Preedakorn, K., Butler, D., & Mehnen, J. (2023). Challenges for the adoption of electric vehicles in Thailand: Potential impacts, barriers, and public policy recommendations. Sustainability, 15(12), Article 9470. https://doi.org/10.3390/su15129470

[96] Chonsalasin, D., Champahom, T., Jomnonkwao, S., Karoonsoontawong, A., Runkawee, N., & Ratanavaraha, V. (2024). Exploring the influence of Thai government policy perceptions on electric vehicle adoption: A measurement model and empirical analysis. Smart Cities, 7(4), 2258–2282. https://doi.org/10.3390/smartcities7040089

[97] Kokchang, P., Chattranont, N., Menaneatra, T., & Srianthumrong, S. (2023). Economic feasibility of hybrid solar-powered charging station with battery energy storage system in Thailand. International Journal of Energy Economics and Policy, 13(3), 342–355. https://doi.org/10.32479/ijeep.14258

[98] Salazar, E. J., Jurado, M., & Samper, M. E. (2023). Reinforcement learning-based pricing and incentive strategy for demand response in smart grids. Energies, 16(3), Article 1466. https://doi.org/10.3390/en16031466

[99] Wattana, B., & Wattana, S. (2022). Implications of electric vehicle promotion policy on the road transport and electricity sectors for Thailand. Energy Strategy Reviews, 42, Article 100901. https://doi.org/10.1016/j.esr.2022.100901

[100] Suanpang, P., & Jamjuntr, P. (2024). Optimal electric vehicle battery management using Q-learning for sustainability. Sustainability, 16(16), 7180. https://doi.org/10.3390/su16167180

[101] Suanpang, P., Niamsorn, C., Pothipassa, P., Chunhapataragul, T., Netwong, T., & Jermsittiparsert, K. (2022). Extensible metaverse implication for a smart tourism city. Sustainability, 14(21), 14027. https://doi.org/10.3390/su142114027

[102] Zhou, Z., Liu, G., & Tang, Y. (2023). Multi-agent reinforcement learning: Methods, applications, visionary prospects, and challenges. ACM Transactions on Intelligent Systems and Technology, 1(1), 43 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

[103] Dantzig, G. B. (1963). Linear programming and extensions. Princeton University Press.

[104] Karmarkar, N. (1984). A new polynomial-time algorithm for linear programming. Combinatorica, 4(4), 373–395. https://doi.org/10.1007/BF02579150

[105] Fletcher, R. (1987). Practical methods of optimization (2nd ed.). John Wiley & Sons.

[106] Gomory, R. E. (1963). An algorithm for integer solutions to linear programs. Administrative Science Quarterly, 8(3), 245–262.

[107] Lawler, E. L., Lenstra, J. K., Rinooy Kan, A. H. G., & Shmoys, D. B. (1986). The traveling salesman problem: A guided tour of combinatorial optimization. John Wiley & Sons.

[108] Bellman, R. (1957). Dynamic programming. Princeton University Press.

[109] Lauinger, D., Caliandro, P., Van Herle, J., & Kuhn, D. (2016). A linear programming approach to the optimization of residential energy systems. Journal of Energy Storage, 7, 24–37. https://doi.org/10.1016/j.est.2016.04.009

[110] Farajnezhad, Mohammad, Jason See Toh Seong Kuan, and Hesam Kamyab. "Impact of Economic, Social, and Environmental Factors on Electric Vehicle Adoption: A Review." Eidos, vol. 17, no. 24, June 2024, pp. 39-62. Eidos, doi:10.29019/eidos.v17i24.1380.

[111] Yilmaz, M., Valluri, P., & Pagadrai, S. (2012). An integer programming power optimization in storage systems. In C. H. Dagli (Ed.), Complex adaptive systems, Publication 2 (pp. 326–331). Procedia Computer Science, 12. https://doi.org/10.1016/j.procs.2012.09.079

[112] Huffman, D. A. (1952). A method for the construction of minimum-redundancy codes. Proceedings of the IRE, 40(9), 1098–1101. https://doi.org/10.1109/JRPROC.1952.273898

[113] Holland, J. H. (1975). Adaptation in natural and artificial systems. University of Michigan Press.

[114] Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671–680. https://doi.org/10.1126/science.220.4598.671

[115] Cerny, V. (1985). Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45(1), 41–51. https://doi.org/10.1007/BF00940843

[116] Glover, F., & Laguna, M. (1997). Tabu search. Kluwer Academic Publishers.

[117] Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 26(1), 29–41. https://doi.org/10.1109/3477.484436

[118] Dorigo, M., & Stützle, T. (2004). Ant colony optimization. MIT Press.

[119] Yan, N., Zhao, H., Ma, S., & Yan, T. (2021). Research on energy management and control method of microgrid considering health status of batteries in echelon utilization. Energy Reports, 7(Supplement 1), 389–395. https://doi.org/10.1016/j.egyr.2021.01.055

[120] Yu, Z., Cheng, S., Gu, R., Li, Y., Dai, S., & Mao, Z. (2021). Interfacial solar evaporator for clean water production and beyond: From design to application. Applied Energy, 299, 117317. https://doi.org/10.1016/j.apenergy.2021.117317

[121] Kusunda, K. (2019). Optimal electricity cost minimization of a grid-interactive pumped hydro storage using ground water in a dynamic electricity pricing environment. Energy Reports, 5, 159–169. https://doi.org/10.1016/j.egyr.2019.01.004

[122] Xie, J., Ajagekar, A., & You, F. (2023). Multi-agent attention-based deep reinforcement learning for demand response in grid-responsive buildings. Applied Energy, 342, 121162. https://doi.org/10.1016/j.apenergy.2023.121162

[123] Zafar, S., Blavette, A., Camilleri, G., Ben Ahmed, H., & Agbodjan, J.-J. P. (2023). Decentralized optimal management of a large-scale EV fleet: Optimality and computational complexity comparison between an adaptive MAS and MILP. International Journal of Electrical Power & Energy Systems, 147, 108861. https://doi.org/10.1016/j.ijepes.2022.108861

[124] Wu, H., Qiu, D., Zhang, L., & Sun, M. (2024). Adaptive multi-agent reinforcement learning for flexible resource management in a virtual power plant with dynamic participating multi-energy buildings. Applied Energy, 374, 123998. https://doi.org/10.1016/j.apenergy.2024.123998

[125] Zhang, J., Zhang, Z., Han, S., & Lü, S. (2022). Proximal policy optimization via enhanced exploration efficiency. Information Sciences, 609, 750–765. https://doi.org/10.1016/j.ins.2022.07.111

[126] Geng, Sijia, Thomas Lee, Dharik Mallapragada, and Audun Botterud. "An Integer Clustering Approach for Modeling Large-Scale EV Fleets with Guaranteed Performance." Electric Power Systems Research, vol. 236, November 2024, 110650. https://doi.org/10.1016/j.epsr.2024.110650.

[127] Charbonnier, F., Morstyn, T., & McCulloch, M. D. (2022). Scalable multi-agent reinforcement learning for distributed control of residential energy flexibility. Applied Energy, 314, 118825. https://doi.org/10.1016/j.apenergy.2022.118825

[128] Wiedemann, N., Xin, Y., Medici, V., Nespoli, L., Suel, E., & Raubal, M. (2024). Vehicle-to-grid for car sharing: A simulation study for 2030. Applied Energy, 372, 123731. https://doi.org/10.1016/j.apenergy.2024.123731

[129] Hao, X., Chen, Y., Wang, H., Wang, H., Meng, Y., & Gu, Q. (2023). A V2G-oriented reinforcement learning framework and empirical study for heterogeneous electric vehicle charging management. Sustainable Cities and Society, 89, 104345. https://doi.org/10.1016/j.scs.2022.104345

[130] Ali, H., Hussain, S., Khan, H. A., & Khan, I. (2020). Economic and environmental impact of vehicle-to-grid (V2G) integration in an intermittent utility grid. In 2020 2nd International Conference on Smart Power & Internet Energy Systems (SPIES). Bangkok, Thailand. https://doi.org/10.1109/SPIES48661.2020.9242992