7 ms·
You say this as if you don't need he MWP models to train the AI models? The accuracy of the AI Prediction depends entirely on the quality of the training datase
by polairscience 1mo ago
You say this as if you don't need he MWP models to train the AI models? The accuracy of the AI Prediction depends entirely on the quality of the training dataset...
- geertj 1mo agoI would imagine this would be trained on actual historical weather data instead?
- plantain 1mo agoHistorical weather data is discrete. You need continuous state for weather modelling which is currently achieved through conventional reforecasts using those historical observations.
- RandomLensman 1mo agoFrom a quick read: ECMWF and IBTrACS data - the former is model based (with measurement data crunched), the latter purely observational.
- sunshinesnacks 1mo agoPretty much all of the AI weather prediction models are trained on ECMWF ERA5, which is kinda like a numerical weather prediction model run to forecast at t=0. ERA5 is historical weather data, but it’s a “reanalysis” of it.
- tcumulus 1mo agoIndeed, you can imagine this as some sort of advanced physics-based interpolation of various measurements (land stations, satellite data, ...) to fill in every cell in a latitude-longitude grid. This is not only used for ERA5 (training data for the models), but also to determine the initial conditions for every grid cell which are used to roll out the forecast. So AI weather models depend greatly on the NWP/physics used in for reanalysis and initial conditions. That being said, there is also research being conducted in training models straight from the raw data (weather stations, satellite, ...), thus bypassing the "interpolation" step.
- sunshinesnacks 1mo agoYeah. I can’t remember names off the top of my head, but there are a few companies, and I think many researchers, working on AI “data assimilation” for this.
- micro2588 1mo agoECMWF has an experimental AIFS direct observational prediction model (AIFS-DOP) that has become competitive with their physics based IFS model on certain metrics just in the past year. https://arxiv.org/html/2606.19093v1 https://arxiv.org/html/2606.19093v1
- sunshinesnacks 1mo agoAh, yes, that’s one of them! Not to be confused with AIFS and AIFS Ensemble that are competitive with IFS, but start with the same DA as IFS.
- Zacharias030 1mo agoI'm interested in understanding wheater prediction models because accurate wind forecasts make a big difference to my personal life (sports). Is there a good overview to learn about the current models, which all just seem like cryptic acronyms to me? in apps like Windy etc. WRF, TRRM, IK-HRRR-3km, ECMWF-9km,... I understand by now that small grid cells are better for local prediction and that thermic winds are mostly missing from them all.
- drusenko 1mo agoAsk your favorite AI to give you a crash course, but to start the main models you need to know are the GFS and the ECMWF. In the US where available in high res, the HRRR is excellent, but doesn’t forecast very far out. The PWG/PWE 1km PredictWind models are also very good at picking up land based features and other more precise patterns. If you are in the US everything else is probably not super relevant.
- Onavo 1mo agoA more interesting question is...does differential equations based models like mamba/state space models perform better on this sort of physics problem than pure transformer LLMs?
- c0_0p_ 1mo agoIs it? I can't imagine why a language model would do well on this sort of problem at all.