# LLM Finetuning ## Links All about [[LoRA (LLM)]] - [Finetuning LLMs with LoRA and QLoRA: Insights from Hundreds of Experiments - Lightning AI](https://lightning.ai/pages/community/lora-insights/) - [Parameter-Efficient LLM Finetuning With Low-Rank Adaptation (LoRA) - Lightning AI](https://lightning.ai/pages/community/tutorial/lora-llm/) [[2023-02-23]]: - Eleuther's guide: [mesh-transformer-jax/howto\_finetune.md at master · kingoflolz/mesh-transformer-jax · GitHub](https://github.com/kingoflolz/mesh-transformer-jax/blob/master/howto_finetune.md) about [[LLM Finetuning]] - Training on forefront: [How to Fine-Tune GPT-J | Forefront](https://www.forefront.ai/blog-posts/how-to-fine-tune-gpt-j) - Finetuning takeaways: [Fetching Title#scfn](https://www.reddit.com/r/LanguageTechnology/comments/pttzvk/finetuning_gptj_key_takeaways/) [[2023-04-23]]: Here's an interesting link about [[LLM Finetuning]] [[LLama]]: - [GitHub - minosvasilias/godot-dodo: Finetuning large language models for GDScript generation.](https://github.com/minosvasilias/godot-dodo) [[2023-09-13]]: - HN: [Fine-tune your own Llama 2 to replace GPT-3.5/4 | Hacker News](https://news.ycombinator.com/item?id=37484135) - [GitHub - OpenPipe/OpenPipe: Turn expensive prompts into cheap fine-tuned models](https://github.com/openpipe/openpipe) - [Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Custom Models | Hacker News](https://news.ycombinator.com/item?id=37090632) - [Fine-Tuning Llama-2: Tailoring Models to Unique Applications](https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehensive-case-study-for-tailoring-models-to-unique-applications) - [Fine Tuning Is For Form, Not Facts | Anyscale](https://www.anyscale.com/blog/fine-tuning-is-for-form-not-facts) - [Why You Need RAG, Not Finetuning for introducing new facts to an LLM](https://zzbbyy.substack.com/p/why-you-need-rag-not-finetuning) - [New Course: Finetuning Large Language Models - YouTube](https://www.youtube.com/watch?v=9PxhCekQYNI&ab_channel=DeepLearningAI) - [Finetuning Large Language Models - DeepLearning.AI](https://www.deeplearning.ai/short-courses/finetuning-large-language-models/?utm_campaign=finetuning-launch&utm_medium=video&utm_source=youtube&utm_content=teaser) ## Lakera: Ultimate Guide to LLM Fine Tuning: - [The Ultimate Guide to LLM Fine Tuning: Best Practices & Tools | Lakera – Protecting AI teams that disrupt the world.](https://www.lakera.ai/blog/llm-fine-tuning-guide) ## Sebastian Raschka: Finetuning Large Language Models > [!ai]+ ChatGPT > > 1. **In-Context Learning**: This doesn't require further training or fine-tuning of LLMs for specific tasks. By providing examples via input prompts, LLMs can perform tasks they weren't explicitly trained on. > > 2. **Feature-Based Approach**: Involves using a pre-trained LLM as a feature extractor. The output embeddings are used to train a classification model. This approach is resource-friendly but may offer limited performance. > > 3. **Finetuning I – Updating The Output Layers**: Similar to the feature-based approach, but only the output layers are trained, keeping the LLM parameters frozen. This method is slightly better in performance compared to the feature-based approach. > > 4. **Finetuning II – Updating All Layers**: This approach involves updating all layers of the LLM, not just the output layers. It's more expensive computationally but often results in superior model performance. > > 5. **Parameter-Efficient Finetuning**: This method introduces a small number of additional parameters for fine-tuning, achieving high performance while minimizing computational resources. Techniques include prefix tuning, adapters, and low-rank adaptation. > > 6. **Reinforcement Learning with Human Feedback (RLHF)**: Involves fine-tuning a pre-trained model using a combination of supervised learning and reinforcement learning, guided by human feedback to adapt the LLM to human preferences. > - [Finetuning Large Language Models](https://magazine.sebastianraschka.com/p/finetuning-large-language-models)