On: Dust: Pretraining Transformers Without Backpropagation

[cyber_kinetist]

Support Vector Machines involve solving a large QP optimization problem... which is often done by running a variant of gradient descent (or one of the second- or quasi-second-order optimization algorithms, which involves finding the Hessian as well as the gradient)

[Maxatar → cyber_kinetist]

I was under the impression that solving QP optimization problems (subject to constraints) was largely performed by SMO.