Sahil Kapoor's Playbook·May 17, 2026 LoRA (Low-Rank Adaptation)
LoRA (Low-Rank Adaptation) is a fine-tuning method introduced by Hu et al. at Microsoft in 2021. Instead of updating all billions of parameters in a large model, LoRA freezes the original weights and injects trainable low-rank matrices into each transformer layer. The insight: weight updates during fine-tuning have low "intrinsic rank", most of the useful signal lives in a much smaller subspace.
The Math
For a weight matrix W (d×k), LoRA learns two small matrices: A (d×r) and B (r×k), where rPrompt Engineering·System Prompt
![TNL The News Lens 關鍵評論網 [Unofficial]](https://cdn.bsky.app/img/avatar/plain/did:plc:a2el5kvbfktjlmztf7jss242/bafkreigdjapffjz2jsijwqfckspdgnkih7xk2biwxpiuo2lordbziv5744@jpeg)
