All research

Intelligent energy systems · 2024–2025

Edge AI and deep reinforcement learning for intelligent energy management

Occupancy sensing, privacy-preserving edge analytics and a broad review of deep reinforcement learning for energy systems.

E-Tawfeer project · Qatar University · Qatar National Research Fund NPRP14S-0401-210122 · Published studies and project development

Illustrated energy network linking power plants, a factory, office buildings, houses, wind turbines, solar panels, battery storage, electric vehicles and a grid control centre. Labels list deep reinforcement learning applications in generation, transmission, distribution, storage, markets and local energy management.
Figure 4 from Mahmud et al. (2026). Deep reinforcement learning applications across key components of the energy internet.View full-size figureSource articleCC BY 4.0

Approach

How can AI support adaptive energy decisions while keeping sensitive building and household data protected?

E-Tawfeer is a research project on a secure, edge-based energy-efficiency platform that uses AI to support behavioural change. Useful energy decisions depend on knowing how buildings are used, yet detailed occupancy and appliance data can expose private routines. This work moved from compact occupancy sensing, to smart plugs and privacy-preserving analytics, to a wider review of how learning agents can optimise energy systems.

  1. Sense building use
  2. Analyse data near its source
  3. Optimise energy decisions

Compact occupancy detection from environmental sensors

Neural Computing and Applications · 2025

Knowing when a room is occupied helps buildings avoid wasted energy, but cameras and wearables raise cost, usability and privacy concerns. This study used environmental measurements such as temperature, humidity, light and carbon dioxide. Each short time window was converted into a small greyscale image for a convolutional network.

BOD-Net adapts MobileNetV2 inverted residual blocks and adds a global skip connection that joins early and late features. It was evaluated with fivefold cross-validation on three public datasets. Accuracy improved over the earlier image-based CNN on every dataset, with the largest gains on the student-room and living-room data.

The network has about 0.26 million parameters, compared with 2.2 million for MobileNetV2, and its reported inference time was 4.5 ms, compared with 129 ms. On the imbalanced living-room data, oversampling raised sensitivity from 92.58% to 96.88%, and XGBoost remained a strong alternative.

Accuracy on UCI-ODD
99.05%
Model parameters
262,866
Reported inference time
4.5 ms
Occupancy detection accuracy reported by Sayed et al. (2025).
Encoded samplesEarlier CNN1D-CNNBOD-Net
Student room (SRD)30,73395.56%98.02%98.45%
Office room (UCI-ODD)20,56099.00%98.91%99.05%
Living room (LRD)29,75095.73%96.91%97.32%
Network diagram. A 32 by 32 greyscale input passes through a stem convolution and four levels of inverted residual blocks. A max-pooled skip connection is concatenated with their output before further convolution, adaptive pooling, a dense layer and a log-sigmoid output. Lower panels show the block structures.
Figure 1 from Sayed et al. (2025). BOD-Net architecture, inverted residual settings and block structures.View full-size figureSource articleCC BY 4.0
Three confusion matrices. Vacant and occupied samples are correctly classified at 0.91 and 0.99 for LRD, 0.98 and 0.98 for SRD, and 0.99 and 1.00 for UCI.
Figure 3a from Sayed et al. (2025), cropped from the full figure. Confusion matrices for vacant (0) and occupied (1) rooms, combined across five folds.View full-size figureSource articleCC BY 4.0

Smart plugs and privacy-preserving analytics

Project development · August 2024–June 2025

Appliance and occupancy data can reveal daily routines, so sending every measurement to a central server creates a privacy risk. The project therefore explored keeping more of the analysis close to where household data is produced.

I contributed to smart-plug design and development and to AI methods for residential energy management. I also implemented federated-learning concepts, in which devices train on local data and share model updates rather than raw measurements.

Engineering focus of this project stage.
Purpose
Smart plugsBring energy sensing and control to individual appliances
Edge analyticsProcess residential data close to its source
Federated learningLearn across devices without pooling raw data

Mapping deep reinforcement learning across the energy internet

Renewable and Sustainable Energy Reviews · 2026

Modern energy systems must coordinate renewable generation, storage, flexible demand and markets under constant uncertainty. Deep reinforcement learning can learn sequential decisions in these settings, but earlier surveys usually focused on one part of the system. This review connects the full chain, from centralised generation to local energy management, security and privacy.

It organises methods by algorithmic foundations, learning paradigms, representation learning, exploration, multi-agent learning, advanced techniques and optimisation strategies, then links them to practical energy tasks. Reported gains include lower costs, reduced emissions and better operating efficiency, but they often depend on data quality, reward design and the chosen benchmark.

The main barriers are scarce or biased data, weak generalisation across assets and seasons, computational cost, safety constraints and the gap between simulation and field deployment. Promising directions include hierarchical and hybrid methods, graph-based multi-agent learning, transfer learning, and federated approaches that keep sensitive energy data local.

Peer-reviewed studies synthesised
500+
Publication window
2020–2026
Energy application domains
8
Examples from the review. Each result is reported by the cited study under its own test conditions.
Decision problemsExample reported finding
GenerationSolar and wind power tracking, turbine control, reservoir operationA DQN framework increased annual hydropower production by 15% (Xu et al.)
Distributed resourcesHybrid scheduling, microgrid resources, virtual power plantsMATD3 bidding increased virtual power plant revenue by 65% (Jiang et al.)
TransmissionDynamic line rating, optimal power flow, HVDC controlEmergency frequency control reduced generator tripping on a modified IEEE 39-bus system (Li et al.)
DistributionLoad management, voltage and frequency control, vehicle-to-everything, microgridsDistributed load scheduling cut peak-to-average ratio by 12% and load variance by 11% (Chung et al.)
Energy storageHybrid storage, renewable integration, arbitrage, degradationGrid-scale arbitrage achieved a 2.4-fold profit increase and 30% less on-peak grid use (Han et al.)
Energy marketsTrading, bidding, dynamic pricing, energy sharingPPO-based dynamic pricing improved demand-response aggregator profit by 35% (Fraija et al.)
Local energy managementBuilding and home energy systems, storage, demand-side controlOnline transfer learning achieved 20–40% cost savings in building energy control (Coraci et al.)
Security and privacyCyber-attack defence, resilient control, private data useAdversarial learning strengthened inverter-based microgrids against cyberattacks on an IEEE 123-bus system (Wang et al.)
Tree diagram of deep reinforcement learning for the energy internet, branching into algorithmic foundations, learning paradigms, representation learning, exploration strategies, multi-agent learning, advanced techniques and optimisation strategies.
Figure 2 from Mahmud et al. (2026). Taxonomy of deep reinforcement learning used in the review.View full-size figureSource articleCC BY 4.0

My contribution

I redesigned the occupancy CNN and contributed to the experiments, analysis and manuscript. I contributed to smart-plug design and development and implemented federated-learning concepts. I led the reinforcement learning review, including its design, literature synthesis, taxonomy, comparative analysis, visualisations and manuscript drafting and revision.

Study scope

The occupancy results come from three public datasets, and wider testing across building types remains future work. The smart-plug and federated-learning work is presented as development activity, without performance claims. Findings in the review are those reported by the cited studies, and many depend on their simulation settings and benchmarks.