9 ms·
PCA is good, but you could also try playing around with Sparse (robust) PCA. The sparsification loses orthogonality, but does not necessarily lose information,
by 5555watch 1mo ago
PCA is good, but you could also try playing around with Sparse (robust) PCA. The sparsification loses orthogonality, but does not necessarily lose information, it can yield a different rotation and cleaner vectors. Now whether that matters in the context of LLMs/Embeddings - I cannot tell.