This project incorporated robust statistics with B-spline function to handle outliers in the Frequency Disturbance Recorder (FDR) raw data.
The Frequency Monitoring Network (FNET) has provided a low-cost and easy way to collect wide-area frequencies with high dynamic accuracy in North American power grids. In practice, frequency measurements collected by the Frequency Disturbance Recorders (FDRs) contain outliers such as missing segments or spikes due to hardware and/or network failure, and 1-D median filtering is commonly used to eliminate these outliers; however, this filtering removes the spikes as well as the detailed information of frequency variation and is incapable of replacing the missing data. In the current project, the spikes are first identified by a preset threshold of robust statistics, and then the spikes and the missing data are replaced using B-spline smoothing, which reconstructs the FDR data by a linear combination of a family of B-spline basis functions defined by the de Boor recurrent formula. Roughness of the constructed curves is controlled to avoid the over-fitting problem of this technique. The proposed method is only used for outliers and keeps the rest of the FDR raw data intact. Test examples validate this method, and its application may be easily extended to time series data in other fields. (publisher abstract modified)