Hybrid AI-Optimized FACTS Controllers for Enhancing Small-Signal and Transient Stability in Multi-Machine Power Systems

Abstract

Modern power systems are undergoing rapid transformation due to the large-scale integration of renewable energy sources, rising electricity demand, and the increasing interconnection of regional grids. While these developments enhance flexibility and efficiency, they also expose networks to severe stability challenges, particularly low-frequency inter-area oscillations and transient instabilities. Conventional stabilization mechanisms such as Power System Stabilizers (PSS) and classical FACTS-based controllers (PI, lead–lag) provide acceptable damping under nominal conditions, but their effectiveness diminishes when subjected to nonlinear dynamics, high loading scenarios, and severe contingencies. Intelligent controllers such as fuzzy logic and Adaptive Neuro-Fuzzy Inference Systems (ANFIS) have offered improved adaptability, yet their scalability and parameter tuning remain major bottlenecks. Metaheuristic optimization algorithms, such as Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Grey Wolf Optimizer (GWO), have shown promise for optimizing FACTS controllers. However, their implementation in real-time systems has been hindered by issues like premature convergence and a lack of real-time validation. For FACTS devices (STATCOM, TCSC, UPFC), this work suggests a hybrid AI-optimized ANFIS damping controller to improve transient and small-signal stability in multimachine power systems. Damping ratio (ζ), settling time (Ts), and critical clearing time (CCT) were integrated as design indices in a multi-objective optimization framework.Hybrid metaheuristic optimizers (GA+PSO and PSO+GWO) were used, utilizing adaptive convergence and global search efficiency, to provide robustness and flexibility. The IEEE 39-bus New England network and the IEEE 68-bus NY–NE interconnected system were used as benchmark systems for evaluating the suggested architecture. MATLAB/Simulink was used for eigenvalue analysis, while PSCAD was used for electromagnetic transient validation. Beyond simulations, a complete Hardware-in-the-Loop (HIL) infrastructure was put into place using dSPACE DS1104, DSP TMS320F28335, and FPGA Spartan-6. This allowed for real-time validation in scenarios including inter-area oscillations (~0.64 Hz), three-phase failures, and load changes of ±20%. A realistic assessment framework was provided by the HIL environment, which accurately represented switching dynamics, hardware latency, and controller execution delays. Simulation and experimental results repeatedly showed that the hybrid AI-ANFIS controller outperformed standalone and traditional intelligent controllers. With simulation-to-experiment errors kept at 5%, the suggested method quantitatively improved the damping ratio by >400%, increased the CCT by ~68%, decreased the settling time by ~48%, and reduced overshoot by >70%. The efficient displacement of oscillatory modes to the left-half plane was validated by eigenvalue migration maps, and rotor angle swing curves demonstrated quicker damping and steady recovery in the face of extreme disturbances. This study fills a vacuum in the literature by integrating hybrid AI optimization with ANFIS-based FACTS controllers, which has been proven by real-time HIL experiments.These controllers can handle the dual problems of transient fault resilience and oscillatory damping, proving that they are suited for smart grid implementation. The proven scalability across various IEEE benchmark networks also highlights the possibility for practical implementation in interconnected power systems with a high renewable content. In order to bridge the gap between simulation and hardware implementation, this work offers a verified roadmap for next-generation power system stability augmentation techniques.