Here is a pytorch optimizer that can maintain a matrix as orthogonal throughout optimization: https://github.com/adrianjav/pogo https://github.com/adrianjav/po

Here is a pytorch optimizer that can maintain a matrix as orthogonal throughout optimization: https://github.com/adrianjav/pogo https://github.com/adrianjav/pogo — POGO: A Proximal One-step Geometric Orthoptimizer https://arxiv.org/abs/2602.14656 https://arxiv.org/abs/2602.14656 — An Embarrassingly Simple Way to Optimize Orthogonal Matrices at Scale; Adrián Javaloy, Antonio Vergari