Nitrogen fertilizer is made from natural gas, so maybe the nitrogen fertilizer plants in the EU were shut down to ensure adequate supplies for the EU's more inelastic demands for natural gas.
(More precisely, the natural gas is a cheap source of hydrogen atoms that get combined with nitrogen in the air to produce the fertilizer.)
Maybe Russia realized that it would be profitable to use some of the natural gas it could no longer sell to Europe to make nitrogen fertilizer, but it takes many months to build the capacity to produce and export it, and in the meantime of course the price remains high.
I believe Russia is also a major fertilizer exporter and nitrogen fertilizers have more or less doubled in price