Directly comparing 2000 to 2020 makes little sense given the consistent downward trend and magnitude of difference between 2001-2005 and 2015-2019.
Given the downward trend in adjusted deaths per million over time, a more reasonable approach is to average the last 5 years and compare it to 2020. Doing so, you will find that 2020 has ~750 more deaths per million in Sweden than prior periods. (Keep in mind this approach likely understates the affect of COVID on mortality rate due to the aforementioned trend).
For example, before we decide that the last 5 years are a good average we should look at what the trend of the last 5 years looked like. If for example there was an upwards trend (which it was for many countries around the world) then even without COVID you'd expect that 2020 would have continued on that trend for whatever reason it was going that way. In that case it would be fairly unexciting to say that 2020 was above the 5 year average in the context of COVID.
In my opinion the fairest comparison would be to check if 2020 continued the trend of the last 5 years, or if it had a non linear jump upwards from that trend, but looking at figures such as "above average" doesn't mean anything if every year in the last 5 years was "above average" because it started to trend upwards again.