Pseudo Inverse
Reprinted and translated from this Post. All Copyrights Reserved. Many readers, like myself, may find themselves with a feeling of both familiarity and strangeness when it comes to the low-rank approximation of matrices. This familiarity stems from the fact that the concept and significance of low-rank approximation are straightforward to grasp. Moreover, with the proliferation of fine-tuning techniques built upon it, such as LoRA, the idea has become deeply ingrained through constant exposure. Nevertheless, the sheer breadth of content covered by low-rank approximation, coupled with the frequent appearance of unfamiliar yet astonishing new techniques in related papers, results in a sense of not-quite-knowing. ...