Fine-tuning an open-weight model used to mean renting an 80GB GPU or not doing it. Two techniques - LoRA, published in 2021, and QLoRA, published in 2023 - changed the arithmetic enough that adapting a model on a single consumer GPU is the default rather than the exception. This page is what those techniques actually do, what knobs they expose, and what the honest trade-offs look like on hardware the rest of this site’s VRAM tier guides cover.
What LoRA changes about fine-tuning
The original LoRA paper introduces trainable rank-decomposition matrices injected into each Transformer layer, while the pretrained weights stay frozen. The reported wins were 10,000x fewer trainable parameters and 3x less GPU memory than full fine-tuning of GPT-3 175B, with no additional inference latency because the adapter weights can be merged back into the base model after training.
In practice, with Hugging Face’s PEFT library, the practical difference is that you freeze the base model and train only a small set of adapter weights. Multiple adapters can sit on top of one base model, each typically a few tens of megabytes, and the base stays a single download. The catch is that the trained adapter cannot invent information that was not in the base model; it can only nudge the existing behaviour.
What QLoRA adds on top
The QLoRA paper keeps the LoRA adapter idea and adds three changes to the base model: it quantises the base to 4-bit, uses a 4-bit NormalFloat (NF4) data type that the authors call “information theoretically optimal for normally distributed weights”, and adds paged optimisers to manage memory spikes during training. The headline number in the paper was fitting a 65B-parameter model on a single 48GB GPU while preserving 16-bit fine-tuning task performance, and finetuning over 1,000 models across 8 instruction datasets.
Hugging Face’s Transformers bitsandbytes integration implements this end-to-end. The four pieces that matter to set are load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, and bnb_4bit_compute_dtype=torch.bfloat16. The double-quantisation pass adds a second quantisation of the already-quantised weights to save an additional 0.4 bits per parameter; the HF docs document using nested quantisation to finetune a Llama-13b on a 16GB NVIDIA T4 with a sequence length of 1024 and a batch size of 1.
The knobs, and what they actually do
Three settings on LoraConfig do most of the work:
- rank (
r) sets the size of the low-rank update matrices. Higherrmeans more trainable parameters and more learning capacity. The PEFT conceptual guide frames it as a capacity trade: “a higher rank means the model has more parameters to train, but it also means the model has more learning capacity.” lora_alphais the scaling factor for the adapter’s contribution. With standard LoRA, the effective scaling islora_alpha / r. With rank-stabilised LoRA (use_rslora=True), the scaling switches tolora_alpha / sqrt(r), which the same guide frames as unlocking the increased performance potential at higher ranks.target_modulesnames which layers get the adapter. PEFT’s developer guide shows the common choice as["query", "value"](attention only);target_modules="all-linear"matches the QLoRA paper’s set and is the broader option.
Two further parameters decide memory versus correctness. lora_dropout adds regularisation on the adapter; bias controls whether biases are trained ("none", "all", or "lora_only"). The PEFT library also ships named initialisation strategies - init_lora_weights="pissa" for faster convergence from principal singular values, "loftq" to minimise quantisation error on top of QLoRA, and "eva" for data-driven rank allocation.
For QLoRA specifically, TRL’s SFTTrainer documentation accepts a quantization_config=BitsAndBytesConfig(...) argument directly, paired with peft_config=LoraConfig(...). The same page documents packing=True (multiple short examples packed into one sequence), assistant_only_loss=True (compute loss only on the assistant response), and notes that adapters typically use a higher learning rate - around 1e-4 - since only new parameters are being learned.
| Setting | Conservative | Common | Aggressive |
|---|---|---|---|
LoRA rank r | 8 | 16 | 64-128 |
lora_alpha | 16 | 32 | 64-128 |
| Scaling | alpha/r | alpha/r (rsLoRA off) | alpha/sqrt(r) with use_rslora=True |
target_modules | ["q_proj", "v_proj"] | "all-linear" (QLoRA paper) | "all-linear" plus modules_to_save=["lm_head"] |
| Quantisation | 8-bit base (bnb) | 4-bit NF4 + double quant | 4-bit NF4 + double quant + LoftQ init |
| Effective precision | bf16 compute | bf16 compute | bf16 compute |
| Learning rate | 1e-4 to 5e-5 | 2e-4 | 5e-5 with LoRA+ (loraplus_lr_ratio=16) |
The LoRA+ row references the PEFT developer’s create_loraplus_optimizer, which the upstream guide reports as up to 2x faster fine-tuning and 1 to 2 percent better performance.
The tools that wrap it
The same recipe shows up in three places with different ergonomics:
- PEFT + TRL + Transformers + bitsandbytes is the canonical reference stack. TRL’s SFTTrainer integrates with PEFT’s
LoraConfigdirectly, supports conversational and prompt-completion datasets, and ships packing for short-example efficiency. - Unsloth (README) ships “2x faster with 70 percent less VRAM” claims on the general path and higher numbers on MoE models, plus per-model notebooks for Qwen3.5, gpt-oss, Gemma 4 and Llama 3.1/3.2. Its integration with TRL is documented and remains Hugging Face-compatible.
- Axolotl (README) is a YAML-config front-end for the same stack. The README lists full fine-tuning, LoRA, QLoRA, GPTQ, FP8 mixed-precision and QAT (int8, int4, FP8, NVFP4, MXFP4) training paths, plus DPO, IPO, KTO, ORPO, GRPO and GDPO preference and RL methods. Hardware floor in the README is “NVIDIA GPU (Ampere or newer for bf16 and Flash Attention) or AMD GPU.”
Pick by taste: PEFT/TRL is the lowest-level and the easiest to debug; Unsloth is the path of least resistance on the named models; Axolotl is what you reach for when you want declarative configs and reproducible sweeps.
What to measure on your own workload
A fine-tuned adapter that does better than the base on one benchmark can do worse on the workload you actually run. The same point made in the quantisation article holds here: cost is not uniform across tasks, so the published numbers are a starting point, not the answer.
Two metrics the tools give you for free:
- Training loss (token-level cross-entropy) tells you whether the model is fitting the data. Use
assistant_only_loss=Truefor chat-style datasets so loss is computed only on the assistant response, not on prompts. mean_token_accuracylogged by TRL tells you the top-1 next-token hit rate on the training data. It is not a quality metric on its own, but it should rise during training; a flat number means the adapter is not learning.
For task-level measurement, the same benchmarks named in the quantisation article apply unchanged: lm-evaluation-harness for MMLU/GSM8K/HumanEval, Aider Polyglot for code, and a held-out slice of your own data for whatever the model is actually for. Run the base model and the merged adapter on the same eval and look at the delta, not the absolute number.
The honest trade-offs
- Adapters cannot invent capability. If the base model cannot do a task, an adapter trained on examples of that task will at best produce fluent failures. Fine-tuning is a behaviour-shaping tool, not a knowledge-injection tool.
- VRAM math is not weight math. A 7B base in 4-bit plus LoRA adapters, optimiser states and activations fits on a 16GB card. The same base in 16-bit plus the same adapter is a different machine class. The arithmetic lives in how much VRAM a local LLM actually needs, not in the marketing copy of the training library.
- Licence travels with the base. Apache-2.0 and most community licences let you ship a fine-tuned derivative commercially; the Gemma licence and the Llama community licence are custom and should be re-read before any commercial use.
- CPU-only machines are not in scope. The techniques on this page assume a CUDA-capable GPU. A machine without one is the no-GPU guide, not this one.
Bottom Line
LoRA freezes the base and trains only a small adapter; QLoRA additionally quantises the base to 4-bit NF4 and adds paged optimisers, which the original paper showed fit a 65B model on a single 48GB GPU. The default stack is PEFT plus TRL’s SFTTrainer plus bitsandbytes, with Unsloth and Axolotl as ergonomic alternatives. The settings that do the most work are rank, alpha, target modules and - for QLoRA - the four BitsAndBytesConfig flags. Pick rank and alpha together (alpha roughly equal to rank for standard LoRA, alpha equal to rank times sqrt(r) under rsLoRA), set target_modules="all-linear" if you want to match the QLoRA paper’s setup, and always evaluate the merged adapter against the base on a held-out slice of your real workload before you trust it.