Datasets
Standard Dataset
An Effective Algorithm of Outlier Correction in Space-time Radar Rainfall Data Based on the Iterative Localized Analysis
- Citation Author(s):
-
DongkyunKim
Hongik UniversityYongchanKimHongik University - Submitted by:
- Yongchan Kim
- Last updated:
- Tue, 02/13/2024 - 12:45
- DOI:
- 10.21227/k1k8-jt58
- License:
139 Views- Categories:
- Keywords:
1 rating - Please login to submit your rating.
Abstract
This MATLAB function efficiently detects and removes outliers from a specified time slice of a radar data cube, improving data quality for analysis. It utilizes statistical thresholds to identify outliers and offers the option to visualize and save the cleaning process as a GIF, facilitating a clearer understanding of data integrity enhancements. The functionality is crucial for preprocessing in meteorological studies, ensuring accurate representations in spatial and temporal data analyses.
Instructions:
Function Overview
- This function detects and removes outlier data points from a radar data cube based on a defined upper limit. It allows for the visualization of the outlier removal process and can save these visualizations as GIFs if desired.
Input Parameters
- Cube: This is the 3D radar data cube, with dimensions representing time, X (spatial dimension), and Y (spatial dimension). The data should be organized such that each "slice" of the cube along the time axis corresponds to a 2D spatial radar image at a specific time point.
- n: A specific time frame within the radar data cube you want to analyze for outliers. This parameter selects the specific "slice" of the 3D cube for outlier detection and removal.
- Upperlimit: The threshold value used to identify outliers. Any data point with a value greater than this limit is considered an outlier and subject to removal.
- yyyy: The year of the data being processed. This parameter is used primarily for labeling and organizing output files, such as saved GIFs.
- SVpath: The path where output files (e.g., GIFs) will be saved. This needs to be a valid directory path on your system.
- SaveTF: A flag indicating whether to save the visualization of the outlier removal process as a GIF. A value of 1 enables saving, while 0 disables it.
How It Works
- Initialization: The function begins by extracting the specified time slice (n) from the radar data cube. It then initializes several matrices used in the outlier detection and removal process.
- Outlier Detection: For the selected time slice, the function searches for the maximum value and checks if it's an outlier based on the Upperlimit and surrounding values. This involves comparing the maximum value against the mean and standard deviation of nearby values to determine if it significantly deviates from its surroundings.
- Outlier Removal: Once an outlier is detected, it's marked for removal. The function then checks surrounding areas for additional outliers, expanding outwards from the initially identified point. This process repeats until no more outliers are detected based on the criteria.
- Visualization and Saving: If SaveTF is set to 1, the function visualizes the radar data slice with outliers removed alongside a mask of the removed outliers. This visualization can be saved as a GIF to the specified SVpath, providing a dynamic view of the outlier removal process.
- Output: The function returns the modified radar data slice with outliers removed. If outliers were detected and removed, their locations in the data slice are set to NaN (not a number), indicating missing or disregarded values.
