Abstract
Smart thermostat-based fault detection and diagnosis (FDD) for residential air-conditioning (AC) systems is gaining traction due to its low-cost prospect. However, smart thermostats measure only a limited set of variables, raising concerns about the reliability and applicability of thermostat-driven FDD. This study therefore proposes a representative co-simulation testbed and six quantitative performance metrics for assessing smart thermostat FDD methods. The testbed integrates models of a residential building, a vapor compression AC system, and unmeasurable building load disturbances (UBLD) arising from occupant behavior and weather variability. For representativeness, a co-simulation approach was used with EnergyPlus for building modeling and a Dymola for AC modeling. Additionally, components from three specialized Modelica-based libraries were used to develop the testbed, making it more representative. To demonstrate the testbed's capability to capture UBLDs due to irregular occupancy patterns and/or weather conditions (e.g., windspeed and outdoor temperature), 15 case studies of faults and UBLDs were designed and simulated. The UBLD scenarios were designed from two survey data. Simulation results were then used to analyze the impact of UBLDs and common AC faults on FDD performance. The analyses show that UBLDs could cause false alarm rates of up to 30% even with a 15% deviation threshold, and that this can be reduced with a cumulative sum (CUSUM) logic. Furthermore, a signal-to-noise ratio metric – incorporating both false positive and false negative rates – is applied to assess a commonly used FDD feature, duty factor, and revealed its limited effectiveness. Overall, this study demonstrates the value of the proposed co-simulation testbed and evaluation metrics for assessing the performance and limitations of smart thermostat-based data-driven FDD methods in residential AC systems.