Local Model Training & Fine-Tuning Guide#

What Was Established#

Guide for fine-tuning local LLMs (DeepSeek) using Hugging Face transformers, with emphasis on VRAM-efficient techniques for single-GPU setups.

Key Decisions#

• Framework: Hugging Face transformers + Trainer API for fine-tuning
• Model: deepseek-ai/deepseek-llm-7b (example model)
• Efficiency: LoRA (Low-Rank Adaptation) + 4-bit quantization via bitsandbytes to fit large models on consumer GPUs

Setup#

pip install torch transformers datasets accelerate peft bitsandbytes

Verify GPU: nvidia-smi — need CUDA 11.8+.