Meteoblue accuracy - compare forecasts feature

TopoNick

Hi there,

I really appreciate the "Compare Weather Models" feature for premium users—it's a fantastic tool. However, I've noticed something odd. In the U.S., for almost all locations I select, meteoblue (MB) is shown as the most accurate model. While I understand it has a good reputation, I find it hard to believe that it's the most accurate everywhere. This is one such spot where the difference between the MB and ICON and NAM is (supposedly) quite large:

To investigate, I took a screenshot of the next day forecasts from different models for a specific spot. For 3 p.m. the next day, MB predicted 64°F.

Then, on that day, I checked the same spot, and the weather station measured 69°F. Interestingly, Windy now shows that MB also predicted 69°F.

I understand that weather models update their data, but if a model adjusts to match the actual temperature after the fact, what's the point if the previous day's forecast was off by 5°F?

This makes me skeptical of the claim that "MB is the most reliable model at your location." What exactly is the calculation behind 'most accurate'? Has anyone else experienced something similar or have any idea what's going on?

idefix37

@TopoNick
The Observation vs forecast feature is comparing the current observed values of wind or temperature at a weather station with the forecasts of different models at same time.
Or, in the past the values observed with the predicted values at same time.
Therefore it does not evaluate the forecasts in 1, 2 or 3 days ahead.
On last several days the smallest median error gives you an idea about the best model for your area.

In mountains for instance you will see that Meteoblue in the detailed forecast panel is the best at predicting temperature.

TopoNick

@idefix37 Got it, that makes sense.

So, in other words, this feature talks about which model best 'understands' the current temperature and says nothing about which provides the best forecast. Is this correct?

Because in my example above, the day before, several other models were better at predicting the weather than MB even if MB was then better at incorporating actual conditions.

Would it make sense to incorporate a feature that compares model forecasts 1, 2, or 3 days out? In my opinion, that's much more interesting information.

idefix37

@TopoNick

Yes, the “Observation vs forecast” feature allows you to highlight the possible bias (systematic error) of a model at a location for a parameter.
For example, take a location in a valley surrounded by mountains. A model with a coarse grid (low resolution) will “see” this location at a higher altitude than reality and the temperatures will systematically be predicted to be colder. As this example.
We can assume that the forecast for the coming days will be affected by the same error - except in the case of a temperature inversion phenomenon... weather forecasting is not simple :) -
Indeed Windy could imagine an even more sophisticated tool to evaluate a model at a given location. But I fear that this will interest only a few users. I have seen very few users on the forum who knew about the "Observation vs. Forecast" tool.

jrwang

@idefix37 I was very pleasantly surprised to learn about the Observation vs. forecast tool a few months ago, but I realize now with the same question as @toponick that I misinterpreted it.

When I am looking at these forecasts, I am primarily interested in knowing how short term predictions are (even knowing that may does not necessarily mean future predictions will perform equally). Perhaps that's the wrong way to look at it?

One of main features I like about the meteoblue.com interface is the availability of short-term verification, which I believe is what both @toponick and I believed the Observation vs. forecast to mean. If there would be any shortcut to integrate the meteoblue approach to short term verification (even a link), I think it would help others better understand the feature.

e.g. https://www.meteoblue.com/en/weather/historyclimate/verificationshort/edmonton_canada_5946768

idefix37

@jrwang said

One of main features I like about the meteoblue.com interface is the availability of short-term verification,

I don't think Meteoblue's graphs are any different from Windy's comparison of “Observation vs forecast”. Meteoblue doesn't compare forecasts for day 1, day 2 or day 3, but rather the current day forecast and the previous day.