SWENET Index Quality Page

To validate the quality of forecasts and predictions of different quantities, this page generates statistics, comparing the predicted values with reference datasets.

The index quality statistics have been implemented with the kind support of Daniel Heynderickx from DH Consultancy and Petra Vanlommel from SIDC.

Generated Statistics
Name Symbol Description
Correlation Coefficient r Is a measure of the reliability of the linear relationship between the x and y values.
Coefficient of Determination r2 Gives the proportion of the variance of one variable that is predictable from the other variable.
Mean Error ME Is a measure of the overall bias of the forecast.
Mean Square Error MSE A measure of the forecast accuracy.
Root Mean Square Error RMSE Is a frequently-used measure of the differences between forecast and observed values.
Forecast Skill Index S The average accuracy of a forecast method relative to a forecast produced with a reference method.
Linear RegressionThe line with the minimized squared distance to all points.
Scatter PlotVisual comparison of forecast vs. measured value for the last month. On a perfect forecast, the green line would match the blue line, such that the linear regression is 1 (x=y).

The linear regression is the (green) line with the minimized squared distance to all points.

Box PlotShows the distribution of value deviations (forecast from measured values). Ideally, the greater part of the deviations would occur on the left hand side (smaller deviations).
Joint DistributionThe joint distribution shows the frequency of occurance of (forecast, observation) pairs.
Monthly DistributionBox plot of the monthly distribution. Stars indicate the minimum and maximum values and the red line the mean observed values. The horizontal blue line within the boxes represents the median and the box encompass the 25th and 75th percentiles, while the box 'whiskers' encompass the 1st and 99th percentiles.
Annual TrendShows the annual trend of the mean observed values, the mean error and the root mean square error as well as the correlation coefficient.
Forecast Skill IndexThe average accuracy of a forecast method over the year


Details on the forecast/nowcast comparisons presented below:

  • Dst (Disturbed Storm Time) is a measure of the ring current around earth, acting against its magnetic field. [wikipedia]
  • K is a geomagnetic activity index (0-9), quantifying disturbances in earth's magnetic field. [wikipedia]
  • F10.7 is a daily measurement of solar radiation at a wavelength of 10.7cm (F10.7), given in solar flux units (sfu). [wikipedia]
  • Ap is a planetary averaged K with exaggeration of high values, so that solar events are not smoothed out by the averaging process. (Ap = A planetary)


SWENET Metrics - Compact View    
QuantityResultsPlots

Ap

fc:BINCASTS
nc:BINCASTS
ts:7 days

Last month
Start2010-08-01
End2010-08-31
r-0.405
r20.164
ME-2.484
MSE83.452
RMSE9.135
S-0.384

Ap

fc:BINCASTS
nc:BINCASTS
ts:14 days

Last month
Start2010-08-01
End2010-08-31
r-0.311
r20.097
ME-3.516
MSE86.290
RMSE9.289
S-0.431

Ap

fc:BINCASTS
nc:BINCASTS
ts:27 days

Last month
Start2010-08-01
End2010-08-31
r-0.065
r20.004
ME-3.097
MSE80.323
RMSE8.962
S-0.332

Dst

fc:GIFINT
nc:Kyoto
ts:1 hour

Last month
Start2010-08-01
End2010-08-31
r0.616
r20.380
ME2.776
MSE123.017
RMSE11.091
S0.338

f107

fc:BINCASTS
nc:BINCASTS
ts:7 days

Last month
Start2010-08-01
End2010-08-31
r-0.228
r20.052
ME4.000
MSE51.613
RMSE7.184
S-1.595

f107

fc:BINCASTS
nc:BINCASTS
ts:14 days

Last month
Start2010-08-01
End2010-08-31
r-0.677
r20.458
ME4.452
MSE81.097
RMSE9.005
S-3.077

f107

fc:BINCASTS
nc:BINCASTS
ts:27 days

Last month
Start2010-08-01
End2010-08-31
r-0.723
r20.522
ME2.258
MSE83.935
RMSE9.162
S-3.220


If you have proposals for further index quality comparisons, please write us, naming the data sources if possible.