It helps that you can pretty easily frame a bidirectional task in a directional way. For example, fill in the middle tasks.
You can have a bidirectional model directly fill in the middle...
Or you could just frame that as a causal task by giving the decoder llm a command to fill in the blanks, and the entire document with the sections to fill replaced by a special token/identifier all as input, and the model is trained to output the middle sections along with their identifier.
There we go, now we have a causal decoder transformer that can perform a traditionally bidirectional task.
You can have a bidirectional model directly fill in the middle...
Or you could just frame that as a causal task by giving the decoder llm a command to fill in the blanks, and the entire document with the sections to fill replaced by a special token/identifier all as input, and the model is trained to output the middle sections along with their identifier.
There we go, now we have a causal decoder transformer that can perform a traditionally bidirectional task.