Applications of Deep Reinforcement Learning for Home Energy Management Systems: A Review

1. Introduction

The transition towards sustainable energy practices has led to an increased emphasis on Home Energy Management Systems (HEMS) as integral components of energy-efficient residential buildings. These systems are designed to monitor, control and optimize energy usage within residences, thereby aligning with broader goals of reduced environmental impact and operational cost savings [1,2]. As residential energy demands increase, driven by the growing utilization of smart devices, electric vehicles and distributed energy resources (DERs) such as solar panels and battery storage, HEMS are becoming indispensable components of modern homes [3,4,5,6]. The importance of HEMS is further emphasized by the advent of smart grids, as they not only facilitate the management of energy within the context of individual households but also provide support for grid stability and efficiency through the implementation of demand side management (DSM) and demand side response (DSR) mechanisms [6,7,8].

In conjunction with these technological developments, new regulatory frameworks, policies and standards are fostering the evolution of HEMS, thereby rendering energy management functions obligatory for residential and prosumer premises. The revised Energy Performance of Buildings Directive (EPBD 2024) [9], places an emphasis on energy efficiency within the context of the building sector and mandates improvements in energy performance across EU member states. The directive’s objective is to achieve a highly energy-efficient and decarbonized building stock by 2050, which will necessitate updates to existing HEMS to meet the requisite energy-saving and emission-reducing targets [10]. Furthermore, the recently introduced in EPBD 2024 the Smart Readiness Indicator (SRI) assesses a building’s capacity to support a range of smart services, including the integration of renewable energy sources (RES) and storage solutions, the enabling of vehicle-to-grid (V2G) connections, and effective interfacing with the smart grid. The SRI is therefore of great importance in the promotion of buildings that are not only energy-efficient but also adaptable to future energy requirements [11,12,13]. New technical standards, such as ISO 52120 [14] for Building Automation and Control Systems (BACS), provide further specification of the functional and performance requirements for energy management. This standard provides comprehensive guidance for optimizing energy usage within residential and commercial buildings, requiring the integration of BACS with HEMS to achieve enhanced energy efficiency and responsiveness to grid signals. In combination, these regulatory changes are propelling HEMS towards more sophisticated functionalities that not only facilitate energy management at the household level but also actively contribute to grid stability [11,15,16].

1.1. Modern and Future Home Energy Management Systems – Complexity and Advancements

It is expected that modern and future HEMS will manage a wide array of interconnected devices, including HVAC systems, lighting, appliances, as well as RESs and local prosumers microgrids [8,17]. These devices are often integrated with distributed control networks and Internet of Things (IoT) technologies that enable real-time monitoring and automated control [18,19,20]. The utilization of the IoT within smart homes provides a plethora of data, yet simultaneously introduces a multitude of complexities. The system must adeptly handle diverse data streams from a multitude of sources, process this data in real-time, and make optimal control decisions under varying conditions [21,22,23,24]. The heterogeneity of data, which encompasses weather forecasts, energy prices, user habits, and the operational status of devices, gives rise to a highly complex problem space. The complexity is further amplified in interconnected environments, where homes operate as nodes within a smart grid. Adaptive and coordinated responses are therefore essential to ensure grid stability and efficiency [25,26,27,28].

Bearing in mind all the discussed aspects, conventional optimization techniques for HEMS, such as rule-based algorithms, static scheduling and linear programming, frequently demonstrate shortcomings in addressing the dynamic characteristics of smart homes comprising interconnected devices and fluctuating demand profiles. A Reinforcement Learning (RL) offers a promising approach for dynamically managing energy resources within complex environments. Its ability to learn and improve policies through interaction and reward makes it a compelling solution for these scenarios. In particular, RL enables HEMS to adapt continuously to changing conditions by adjusting control policies in response to real-time feedback [29,30,31,32]. As HEMS applications become increasingly complex and diverse, Deep Reinforcement Learning (DRL), a subfield of RL that employs deep neural networks to process high-dimensional data, has emerged as a powerful tool for managing these systems. The abilities of DRL algorithms to manage extensive data inputs and learn intricate relationships between variables make them particularly well-suited to smart home environments with extensive IoT devices and diverse energy sources [32,33,34,35].

The implementation of RL and DRL in HEMS presents a number of benefits with regard to DSM and load management. To illustrate, DRL algorithms are capable of anticipating peak load periods based on historical data and current usage patterns, thereby enabling systems to preemptively shift loads, control appliance usage, or draw on stored energy to reduce strain on the grid [6,15,36,37]. Furthermore, the adaptability of RL-based methods enables HEMS to autonomously respond to DSM signals from utility providers, thereby adjusting loads in a manner that optimizes both homeowner cost savings and grid efficiency. This adaptability is crucial, as HEMS are increasingly expected to balance energy consumption and storage within the household while synchronizing with smart grid demands. This will contribute to broader goals of energy resilience and sustainability as well as support transactive energy model [38,39,40,41,42].

The BACS and smart home systems plays a pivotal role within HEMS by orchestrating the control of an array of home subsystems, including lighting, climate, and security, through centralized or distributed control platforms [43,44,45]. The integration of advanced BACS with RL and DRL models facilitates a more unified and effective operation of HEMS. This is achieved by providing a comprehensive view of the user’s activity, home’s energy demands and environmental conditions, which enables the management of the building’s diverse systems in a coordinated manner. This integration supports an optimized approach to energy management, where, for instance, Heating, Ventilation and Air Condition (HVAC) systems can be programmed to adjust based on occupancy patterns detected by IoT sensors or energy pricing data, thereby reducing operational costs [5,22,46,47].

Despite the considerable potential of RL and DRL in the field of HEMS, a number of challenges remain. One significant constraint is the data-intensive nature of DRL models, which necessitate extensive training data to learn optimal policies, often resulting in high computational overheads. Furthermore, DRL models necessitate a high degree of safety and reliability, as suboptimal decisions could result in increased energy costs or compromise user comfort. Furthermore, the potential for integrating DRL with IoT-based HEMS gives rise to concerns regarding data security and user privacy, given that these systems process sensitive personal data from smart home devices [26,48,49,50,51].

1.2. An Original Contribution and the Paper Structure

This review provides a comprehensive examination of the current applications of RL and DRL in HEMS, with a particular focus on the analysis of their benefits, limitations, and the state-of-the-art methodologies. By outlining the contributions of RL and DRL to energy demand forecasting, load scheduling, and peak load management, as well as their role in smart grid integration and DSM, this review aims to demonstrate the value of these technologies in advancing smart home capabilities. Furthermore, it discusses ongoing research and future directions, with particular emphasis on areas where RL and DRL have the potential to facilitate the development of autonomous, efficient, and sustainable home energy systems that will benefit both individual households and the broader energy grid.

Moreover, the review makes a novel contribution by exploring the intersection of RL with IoT technologies, DSR and DSM strategies, scheduling optimization, and the integration of RESs with energy storage systems. The focus on these interrelated domains represents a cutting-edge direction in smart building technologies development, especially in smart home automation and energy management, which remains underdeveloped in existing literature. The novelty of this review lies in the following key aspects:

• Synergy between RL and IoT for Real-Time Smart Home systems. While RL and IoT have individually shown promise in home and building automation [43,52,53,54], this review is among the first to extensively analyze how RL can be leveraged with IoT networks to achieve real-time monitoring and adaptive control in energy management. Moreover, it demonstrates the potential for more efficient and autonomous building operations through the utilization of IoT sensors to feed RL systems with real-time data on energy usage, occupancy, and environmental conditions;
• Innovative approaches to DSR optimization. This review identifies a novel application of RL in enhancing DSR programs, enabling homes, in particular prosumers, to dynamically respond to fluctuations in energy prices and grid conditions. By utilizing RL, homes buildings can autonomously learn optimal strategies for shifting or reducing energy loads, contributing to grid stability and energy cost savings, particularly in the context of peak demand periods. The ability of RL to adapt to varying DR signals and building-specific constraints presents a significant advancement over traditional rule-based approaches;
• Advanced scheduling for energy and resource optimization. A unique focus of this review is the application of RL in scheduling algorithms for home automation systems, particularly in relation to energy consumption, occupancy prediction, and appliance usage. This review explores how RL and DRL can optimize multiobjective scheduling problems, balancing comfort, energy efficiency, and operational costs. Such applications are critical for ensuring flexible home and prosumers systems, capable of responding to dynamic energy demands and varying occupant needs;
• Integration of RL and DRL with RES and energy storage systems. One of the most novel aspects of this review is the examination of how RL and DRL techniques can be used to manage RESs, such as solar and wind, in conjunction with energy storage systems, especially important for modern and future prosumer applications. By enabling intelligent decision-making about when to store, use, or sell generated energy, RL and DRL algorithms can help maximize the self-consumption of renewables and ensure grid or microgrid independence. This is particularly important in homes and buildings aiming for net-zero energy performance, as RL-driven strategies can optimize the use of intermittent RES in real time;
• Bridging the gap between theory and practice. While much of the existing research on RL in building automation remains theoretical or simulation-based, this review uniquely emphasizes need for practical case studies and real-world implementations. It identifies key challenges such as scalability, data availability, and heterogeneous system integration, offering insights into how these challenges can be overcome when deploying RL-based systems in operational environments.

Section 2 sets forth the principal assumptions and methodology of the review, whereas Section 3 contains a synthetic analysis of selected publications. The following Section 4 provides detailed information and characterization of the application areas of RL and DRL techniques and algorithms in homes, along with the identification of gaps and challenges. On this basis, in Section 5, the authors identify potential opportunities and development trends, with particular attention to the role of RL and DRL in the organization of advanced energy management systems for homes and prosumers in local microgrids. In the final Section 6, the conclusions are presented, along with an outline of future work.

2. Methodology of the Review

The methodology of organizing this review is informed by the latest developments in home and building automation systems, including an analysis of the potential for incorporating RL and DRL algorithms into the various functional areas of energy management, considering the guidelines for the SRI [55] introduced firstly by the EPBD 2018 [56]. Consequently, the initial stage of the selection process entailed verifying the number of publications addressing the subjects of building automation, home automation and reinforcement learning in the primary bibliographic databases of scientific and technical literature. The outcomes of this preliminary selection process are presented in Table 1.

It is notable that there is a paucity of scientific publications from the area that combine two threads: home and building automation and RL. In particular, the number of articles and individual review studies on the use of RL in home automation and smart homes is limited, indicating a relatively low level of interest among researchers and scientists in this application area for RL. Conversely, a substantial number of publications in these thematic areas were identified in the Google Scholar database, which encompasses a comprehensive range of academic materials, including those that may not meet traditional bibliographic and scientific standards. This suggests that ongoing processes of technological and technical development are occurring regarding solutions based on RL that are dedicated to applications in building and home automation.

However, it is important to note that the factors driving this development are relatively recent. They have emerged in recent years, largely driven by the necessity to align building infrastructure with new standards governing the smart grid, integration of RES, energy storage units and charging stations for electric vehicles. Furthermore, in Europe, the necessity to develop and implement novel, sophisticated energy and demand management mechanisms in residential and commercial buildings is driven by the requirements set forth in the aforementioned EPBD directive and the SRI indicator [11,13,57,58].

To gain insight into the current state of knowledge and latest developments in RL solutions and algorithms in home and building automation, a literature review was conducted. This involved selecting relevant publications from technical science publishers. These included Springer, ScienceDirect, MDPI, IEEE Xplore (journals and conferences), and additionally Taylor and Francis, the ACM Digital Library, and the Wiley Online Library. The findings are presented in Table 2.

A detailed examination of the data presented in Table 2 clearly demonstrates that the subject of RL implementation in the functional structures of home and building automation systems is a developing area of research and is being addressed in scientific publications by leading publishers. However, it should be noted, however, that most publications are not indexed in the Web of Science and Scopus databases. Furthermore, the authors of this review, who have been engaged with the subject of supporting BACS building automation systems and the impact of automation functions on the energy efficiency of buildings for several years [41,59,60], highlight the necessity for the development and implementation of new, efficient mechanisms for the processing of large amounts of data in such applications. This is a direct consequence of the continual expansion of the infrastructure of both residential and non-residential buildings, particularly in response to the evolving requirements and needs of their users and managers. Thus, there is an increasing number of diverse types of sensors, monitoring modules, energy and media meters that generate an ever-increasing volume of data, as well as fieldbus-level controllers, which in turn process this data and based on it implement usage scenarios for rooms, buildings or entire building campuses, along with public spaces in their surroundings.

This is an area of application in which BACS and Building Management Systems (BMS) have been functioning effectively for a considerable period. They have been installed in a range of larger public, commercial and office buildings, among other locations. However, the growing trend of developing local energy microgrids and smart grid solutions has led to a requirement for the active inclusion of the infrastructure of these facilities, both commercial and residential, in the energy management network. This, in turn, necessitates the implementation of a novel and ingenious methodology for the organization of monitoring and automation functions with data processing. One of the solutions in this domain is the utilization of external data servers and cloud computing resources [44,46,61]. Moreover, an essential aspect is also the provision of data services and effective processing at the local automation network level (fieldbus) [62,63], with the potential for implementing procedures directly in network nodes (IoT) [52,64] or local automation servers (edge computing) [44,50,65]. It is in this application area that the potential of machine learning mechanisms and algorithms, as well as RL and DRL, with various learning models, data analysis are seen to be particularly promising. Accordingly, the authors selected several dozen scientific papers directly concerning the simulation and application studies of such algorithms in the field of building automation, with a particular focus on home automation. The decision to focus on home automation was motivated by the limited number of publications, particularly reviews that simultaneously addressed RL and DRL as well as home automation. Additionally, the growing trend of this application field for RL and DRL further reinforced the choice of home automation as a key area of interest.

3. State of the Art and Practice

In the assumptions for the analysis of the existing state of knowledge in the area of the use of RL and DRL techniques and algorithms in home automation systems, the focus for this review was on threads supporting the implementation of energy management functions. This is a specific area of application in the context of home automation, as so far, the focus has been mainly on ensuring comfort, convenience and security in the use of the home infrastructure [18,66,67,68,69]. On the other hand, the trends and changes identified in sections 1 and 2 indicate the need to carry out research and development work on the integration of mechanisms in home automation systems to support effective energy management, the organization of the functioning of these facilities as prosumers, and so on [8,26,70,71,72,73,74,75,76]. The authors selected four key application areas that support integration, energy and device management, and the operation of prosumer installations in the home. These are the Internet of Things, Demand Response, Scheduling and RES + Storage. As a result, the Table 3 presents a unique collection of several dozen publications in the form of articles in journals and conference materials (without reviews) from the last five years, which constitute the main core of this review analysis, considering the issues indicated previously. In addition, the graph in Figure 1 shows the link between the research and development work on the application of RL and DRL methods discussed in these publications and the main publishers. The developmental nature of this work is evidenced by the large number of publications by the IEEE Xplore publisher, particularly in the form of conference proceedings and individual scientific articles.

The publications collected in the Table 3 have been divided into four groups according to the areas of application of RL and DRL algorithms identified in them. It should be emphasized that for each of the application groups, the authors of these publications analyzed the possibilities of using different RL and DRL algorithms and techniques. They also assumed slightly different objectives and methods for verifying the effectiveness of the algorithm implementation, which can be summarized as follows:

• IoT applications
  • Algorithms used: DRL, Deep Q-learning, Q-learning, DDPG
  • Objectives: Focuses on optimizing cost and comfort, with additional considerations for autonomy, personalization, and privacy
  • Verification: All experiments and models are verified through simulations
• DSR applications
  • Algorithms used: A variety including MORL, Q-learning (and its variations with Fuzzy Reasoning), DQN, MARL, PPO, Actor-Critic methods, among others
  • Objectives: Primarily target cost and comfort optimization
  • Both simulations and some evaluations using real-world data or physical testing setups (e.g., MATLAB and Arduino Uno)
• Scheduling applications
  • Algorithms used: Q-learning, DQN, PPO, MADDPG, among others
  • Objectives: Focus on cost and comfort optimization, with several entries solely targeting cost
  • Verification: Predominantly simulations, with some studies using practical data from real-world networks
• Data security and privacy
  • Algorithms used: TRPO, SAC, Q-learning, PPO, DDPG, and others
  • Objectives: Aimed at optimizing cost and comfort, with a specific focus on energy systems integrating renewable sources and storage
  • Verification: All studies verify their findings through simulations, with some using real-world data from energy markets and PV profiles.

It is noteworthy that the efficacy of RL and DRL algorithms has been evaluated through simulations conducted within the development environment for each analyzed case. It is notable that only a limited number of the simulations included data sets derived from real-world objects. Furthermore, there is a notable absence of detailed descriptions of studies conducted on real-world objects in the subject literature. Additionally, there is a dearth of case studies that would provide insight into the tangible impact of RL and DRL algorithm techniques on the efficacy of automation systems, particularly in the context of energy management in residential and commercial buildings. This is one of the issues identified by the authors of this review as a gap and challenge for further research and development work. Furthermore, they conducted a comprehensive analysis of the characteristics of the solutions and application areas for RL and DRL proposed in the literature, with a particular focus on smaller home automation applications.

4. Applications of Reinforcement Learning for Home Automation

As previously stated, the architectural framework of HEMS can be based on IoT modules, as they facilitate the collection of data on energy consumption, weather conditions, or user presence in real time and remotely, enabling the automatic control of receivers and energy flows [79]. The straightforward integration of IoT devices and the relatively simple control algorithms implemented in them prompt the consideration of more effective control methods, such as DRL [34,112]. In the context of HEMS applications, DRL algorithms are employed with the objective of rationalizing energy consumption in households. In particular, the application of these techniques is beneficial in the context of new home infrastructure related to RES, electric car charging systems and energy storage installations. In such circumstances, the incorporation of DRL mechanisms facilitates the formulation of intelligent decisions pertaining to energy management, with considerations of variables such as energy costs, environmental conditions and user preferences [77,78]. DRL agents learn through interactions with their environment, receiving rewards or penalties for their actions, thereby enabling them to gradually enhance their real-time decision-making capabilities [77]. Moreover, DRL models are employed to forecast future energy availability, consumption patterns, and storage levels by analyzing historical data, weather forecasts, and real-time sensor data. This enables the automation system and users to make decisions that are based on informed reasoning, thereby maximizing the use of renewable energy sources and minimizing dependence on external energy sources. RL-based HEMS systems are capable of adapting energy management strategies in accordance with individual user preferences and changing system conditions, including tariffs and DSM/DSR signals. This enables the achievement of greater energy savings and user comfort. Furthermore, these systems can provide users with recommendations regarding the optimal management of energy consumption [78].

From the technical and functional organization point of view, energy consumption management through optimal work planning (i.e., switching on and off or changing the mode) of devices, particularly those included in the technical infrastructure of the building (e.g., heating, ventilation, air conditioning, etc.), represents a primary task of modern HEMS. The utilization of control algorithms employing DRL within HEMS facilitates the incorporation of dynamic planning (scheduling) functions. The use of DRL is important for the optimization of scheduling strategies in HEMS systems. This is since DRL algorithms learn optimal energy management strategies through interactions with the environment, receiving rewards for actions that result in energy and cost savings. This is evidenced by the literature, which cites numerous studies on the subject [3,6,94,95,96,97,98,99,100,101,102,103].

The adaptability and flexibility of DRL algorithms enable management systems based on monitoring and control functions to adjust their operations in response to changing market and environmental conditions, including fluctuations in energy prices and changes in demand. Concurrently, they facilitate the optimal utilization of renewable energy and energy stored in energy storage tanks, thereby rationalizing energy storage management processes. As previously stated, a key consideration for the implementation of scheduling in HEMS is the incorporation of user preferences, with the objective of aligning energy management strategies with individual requirements. This approach is essential for the acceptance of such a mode of operation by users. Such preferences can be collated through the utilization of user interfaces, which permit the configuration of the system in accordance with user specifications [6,94,95,98,99,101,103]. Alternatively, they can be employed to facilitate the adaptive realignment of the energy management strategy, thereby reducing instances of user dissatisfaction and enhancing overall system efficiency [94,96,98,99,102].

It is becoming increasingly common to integrate RES with HEMS. These sources of energy include photovoltaic (PV) panels, small wind turbines and associated energy storage devices. The primary challenge facing HEMS is the rationalization of energy supplied to buildings by disparate sources, in addition to the effective integration of PV systems and energy storage devices. Subsequently, energy management strategies must be adjusted in a dynamic manner, with a view to ensuring the safety and efficiency of the systems. A significant number of research and technical teams have already indicated that the use of DRL may offer a potential solution to these issues. In the field of energy management optimization, DRL-based algorithms are being developed and implemented with the objective of managing the charging and discharging of energy storage devices in response to changing market conditions and energy demand. A variety of DRL algorithms are employed, including PPO, LSTM, and DDPG. These facilitate the forecasting of prospective energy conditions and the real-time adaptation of energy management strategies. The primary objective of optimization in such procedures is to minimize operating costs and maximize savings and energy efficiency [104,108,109,110,111].

In the context of the evolving energy storage market and its integration into residential settings, the dynamic adjustment of energy management strategies stored in these systems represents a pivotal challenge for HEMS. Systems based on DRL can anticipate future price conditions and making optimal decisions regarding the management of energy storages and RES in accordance with these predictions. These approaches facilitate flexible and efficient energy management, which is of particular importance in the context of evolving market conditions and energy demand [105,106,107,110].

The implementation of advanced DRL algorithms is contingent upon the assurance of user data protection and the safeguarding of systems against potential cyber threats. Furthermore, challenges related to the scalability and interoperability of these systems necessitate the development of standards and communication protocols that will facilitate the integration of diverse devices and platforms. The issues of security and scalability were addressed in the article [111]. The utilization of DRL in the management of energy storage and RES in HEMS systems offers substantial benefits in terms of energy consumption optimization, dynamic adjustment of energy management strategies and integration of renewable energy sources.

The utilization of DRL in HEMS also facilitates the implementation of the DSR concept, which represents a promising solution for dynamic energy management. The implementation of DR in HEMS is achieved through a variety of techniques, including load shifting, peak load reduction, and energy storage management [76,80,81,85,87,91]. Furthermore, such systems gather data on energy consumption, energy prices, and environmental conditions, thereby facilitating the optimal management of energy consumption and the maximization of user comfort [82,91,92].

It is of paramount importance to consider user preferences in DR systems integrated with HEMS to enhance user acceptance and satisfaction. Such preferences can be gathered via user interfaces, which permit the configuration of the system in accordance with user requirements [74,80,82,85,88,92]. It is of paramount importance to ascertain user feedback regarding their satisfaction with the energy management strategy, as this enables the adaptive adjustment of the strategy and the minimization of user dissatisfaction, thus optimizing system efficiency. The deployment of RL algorithms, including Q-learning, DRL, PPO, and Primal-Dual DDPG is of paramount importance for the optimization of DR strategies in HEMS systems. These algorithms learn optimal energy management strategies through interactions with the environment, receiving rewards for actions that result in energy and cost savings [76,81,83,85,87,89,92]. This enables the systems to modify their operational procedures in response to alterations in conditions, such as fluctuations in energy prices and demand [35,74,81,82,88,90]. Mobile applications and other user interfaces afford users the ability to interact with the system, configure preferences, and receive recommendations for optimal energy management. The adaptability of DRL algorithms enables their effective utilization in response to changing market and environmental conditions, such as fluctuations in energy prices and changes in demand. These algorithms facilitate the optimization of renewable energy utilization and energy storage management, thereby minimizing costs and maximizing savings [31,83,84,87,90,93]. Furthermore, DRL models can be employed to forecast future energy requirements and to plan energy consumption in an optimal manner, thus facilitating more effective resource management and cost reduction [35,74,82,88,89].

4.1. Problems, Gaps and Challenges

As mentioned previously, the conjunction of IoT and DRL technologies in HEMS exhibits considerable promise. Nevertheless, numerous challenges and deficiencies necessitate attention. Further research is required to ascertain the feasibility of implementing these technologies in real-world settings, with particular attention paid to data security, scalability, and integration with existing energy infrastructure. The overcoming of these challenges will facilitate the development of more efficient, safe, and widely available energy management systems in households.

The results of the application analyses indicate that HEMS systems implementing DRL-based scheduling show good results, particularly in simulations. However, there are several significant gaps and challenges that need to be addressed. Firstly, there is a paucity of empirical research on the implementation of DLR applications in HEMS. To assess how these systems cope with changing conditions and user preferences, it is necessary to conduct tests in real conditions. Furthermore, additional research is required on data security, scalability and integration with existing infrastructure, as well as real economic analyses. The authors of numerous publications related to the research on the application possibilities of DR technology and advanced DRL algorithms in HEMS systems have highlighted these issues. In particular, the papers [31,80,81,82,85,88,90,92,93] indicate a requirement for economic analyses and the development of more cost-effective solutions to facilitate the scaling of the technology to a larger number of households. In this context, the research and technical teams also identify challenges from a technological and system security perspective. The collection and processing of substantial data sets pertaining to energy consumption and user preferences carries an inherent risk of privacy violations and vulnerability to cyber-attacks. Accordingly, as the authors of the papers [31,35,81,83,87,89,91,93] indicate, advanced security methods should be developed to ensure the protection of user data and the security of systems against potential threats. This remains a current and significant challenge, particularly considering the monotony of available communication technologies and the trend of data processing in cloud applications. A further significant limitation is the scalability of DR systems and their interoperability with a range of devices and platforms. It is imperative to develop standards and communication protocols that will facilitate the seamless integration of diverse devices and systems [31,80,82,83,88,90,113].

A further two problematic issues emerge from the analysis of the literature. The initial challenge pertains to the integration of sophisticated HEMS systems that incorporate DR mechanisms based on DRL with the extant energy infrastructure and smart grid monitoring and control systems. The authors of the papers [31,81,84,86,88,90,92,93] indicate that, given the current technical and standardization conditions, the process may prove to be both complicated and expensive. This necessitates further investigation into methodologies that will facilitate seamless integration of novel technologies with existing energy management systems, thereby enabling the establishment of consistent standards within this domain. A further challenge is that the efficacy of DRL algorithms is dependent on the quality and quantity of data collected by DR systems. In the absence of sufficiently accurate data, suboptimal decisions and system actions may result. Consequently, research is required to develop methods to enhance data quality and algorithms to address the issue of missing or incomplete data, as it is suggested in papers [35,74,82,83,86,89,90,93].

5. Opportunities in Application of Reinforcement Learning in Home Automation and Home and Building Energy Management

Considering the identified gaps and challenges pertaining to the implementation of RL algorithmic methodologies in buildings, it is imperative to consider the diverse nature of such facilities when contemplating the prospective avenues for their advancement. While the principal objective is energy efficiency, there are several fundamental differences between smart home automation systems and larger building automation systems, which have an impact on the application and development of RL algorithms in each case. The home automation systems are often focused on the management of energy-consuming appliances, lighting, HVAC systems, as well as RES, such as rooftop solar PV panels. In some cases, they may also include home energy storage systems, which are batteries that are used to store energy generated from renewable sources. In comparison to large commercial or industrial buildings, these systems are generally smaller in scale, involve fewer interconnected devices, and have more predictable occupant behavior [94,99]. In contrast, building automation systems, particularly in commercial or multi-dwelling environments, manage a more complex array of equipment, including centralized HVAC systems, elevators, large-scale lighting networks, security systems, and RES integrated into microgrids [32,114,115].

Furthermore, recent scientific and technical publications have identified several key application areas, creating new opportunities for the development of the aforementioned RL and DRL mechanisms in energy management and infrastructure of homes and buildings. Table 4 provides a summary of the most important opportunities, assigned to these areas, divided into applications in homes and buildings.

DRL approaches demonstrate considerable potential for enhancing the efficiency of HEMS, thereby improving the energy efficiency of residential buildings under their monitoring and control. Another crucial domain is facilitating the integration of such buildings with smart grid platform functions and DSM/ DSR mechanisms operated by distribution system operators (DSO).

Considering the aforementioned opportunities, the authors have identified key applications of RL and DRL in HEMS that facilitate effective energy management while maintaining the comfort and safety of residential buildings:

• Adaptability and optimization The utilization of DRL models in HEMS facilitates the dynamic realignment of energy management strategies (storage and consumption) in accordance with fluctuating market and weather conditions, as well as evolving user preferences. This approach has the potential to significantly enhance energy and cost savings [138,139];
• Integration with renewable energy sourcesRL-based HEMS systems facilitate the integration of renewable energy sources, such as photovoltaic panels and wind turbines, thereby enhancing energy independence and reducing reliance on the power grid [138];
• Demand management and flexibilityThe implementation of DRL models in HEMS can facilitate demand management by enabling a more flexible and responsive management of energy consumption, which is of paramount importance in the context of dynamic tariffs and demand response programs [75,139];
• Data security and privacyThe implementation of DRL in HEMS systems requires the utilization of sophisticated data protection and security methodologies to guarantee the confidentiality of user data and the integrity of the system against the threat of cyberattacks [29].

6. Conclusions

This review has detailed the significant potential of RL and DRL-based algorithms and methods for improving the efficiency of HEMS, particularly in light of the growing complexities of home infrastructure and energy systems as well as their integration with smart grids. The capacity of RL and DRL for real-time, adaptive decision-making introduces novel approaches to energy management in smart homes, offering benefits over conventional, static control methods. The authors have made a distinctive contribution by examining the applications of RL and DRL methods across several key areas, including load scheduling, DSM/DSR, integration with IoT networks, and energy storage management. Furthermore, the authors emphasize that RL and DRL, through their capacity for continuous learning and data-driven control, can enhance flexibility and responsiveness of HEMS to factors such as fluctuating energy tariffs, renewable energy availability, and user preferences [140,141,142].

One of the key contributions of this review is the synthesis of RL and DRL techniques in the context of BACS and IoT frameworks, which demonstrates how interconnected systems can utilize real-time data to autonomously balance energy loads, optimize renewable energy use and adjust consumption patterns for cost savings [31,98]. This integration represents a future-oriented direction for HEMS development, supporting not only household energy optimization but also contributing to grid stability through DSR mechanisms.

Notwithstanding the aforementioned opportunities, the review also identifies challenges that require future research work. Most notably is the scalability of DRL solutions and their integration within diverse residential infrastructures. A crucial next stage in the development of DRL-based HEMS is the validation of the technology through case studies conducted in real-world applications and scenarios. This approach will facilitate a practical evaluation of the algorithms’ performance, economic impact, and user satisfaction. Furthermore, future research should address data analysis, processing and security concerns. It is needed to investigate methods to enhance algorithmic efficiency in processing high volumes of real-time data and its integration, availability. Such studies will facilitate the transition from theoretical models to practical implementations, thereby advancing RL and DRL-based HEMS towards scalable, resilient, and secure applications in residential energy management.