Sure, if the forecaster and the model were only used one time.
But in 538's case they use the similar methods to forecast many individual contests like presidential primaries, presidential elections, senate elections, house elections, governor elections, club soccer, college football, March Madness men and women, MLB-NBA-NFL games and playoffs, etc.
Multiple-year track records over many events so you can compare their forecasts to actual results over time.
All of those things are very different though. It’s possible you could be excellent at predicting Senate races and terrible at the electoral college, right? And you only get one data point every 4 years.
But in 538's case they use the similar methods to forecast many individual contests like presidential primaries, presidential elections, senate elections, house elections, governor elections, club soccer, college football, March Madness men and women, MLB-NBA-NFL games and playoffs, etc.
Multiple-year track records over many events so you can compare their forecasts to actual results over time.
Get the data and compute your own error rate here: https://github.com/fivethirtyeight/checking-our-work-data