PEFT Guide

TL;DR

• PEFT is a HuggingFace library that unifies parameter-efficient fine-tuning methods - LoRA, adapters, prefix tuning, prompt tuning, and IA³ - behind a single API.

• LoRA is the default choice for 90% of use cases - best quality-to-cost ratio, mergeable at inference, well-supported across models and serving frameworks.

• Each PEFT method trades off trainable parameters, memory, inference overhead, and task performance differently - there is no universally best method.

• PEFT adapters are small (10–500MB), swappable, and composable - enabling multi-tenant serving where one base model serves many task-specific adapters.

• PEFT integrates with Transformers, TRL, bitsandbytes (QLoRA), and serving frameworks - it's the standard abstraction layer for efficient fine-tuning in the HuggingFace ecosystem.

Why This Matters

Before PEFT libraries, every parameter-efficient method had its own implementation, config format, and save/load logic. Switching from LoRA to adapters meant rewriting your training pipeline. PEFT standardizes this: one API to configure, train, save, load, and merge any efficient fine-tuning method.

For production teams, PEFT's adapter management is the key value. You can train ten task-specific adapters, store them as small files, and hot-swap them on a single base model at serving time. This is how multi-tenant LLM serving works in practice.

If you're fine-tuning with HuggingFace tools, you're using PEFT whether you know it or not. Understanding the method options and configuration surface helps you make better decisions and debug training issues faster.

The Problem PEFT Solves

Full fine-tuning is expensive in three dimensions: compute (GPU hours), storage (full model copies), and operational complexity (separate deployments per task). Parameter-efficient methods reduce all three, but the landscape is fragmented:

Each method has different code, config, and deployment patterns. PEFT unifies them:

from peft import get_peft_model, LoraConfig, AdaLoraConfig, PrefixTuningConfig

# Same API regardless of method
model = get_peft_model(base_model, config)
model.print_trainable_parameters()
trainer.train()
model.save_pretrained("./my_adapter")

What Is PEFT?

PEFT (Parameter-Efficient Fine-Tuning) is a HuggingFace library that provides a unified interface for fine-tuning large models by updating only a small subset of parameters. The base model weights remain frozen; small trainable components are injected into the model architecture.

PEFT supports these methods:

LoRA (Low-Rank Adaptation)

Injects low-rank matrices into linear layers. The most popular method. See the dedicated LoRA guide for details.

LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])

Adapters

Inserts small bottleneck layers (down-project → activation → up-project) after attention and FFN sublayers. Original PEFT method.

AdaptionPromptConfig(adapter_layers=30, adapter_len=10)

Prefix Tuning

Prepends trainable prefix vectors to the key and value states at every transformer layer. The model attends to these learned prefixes as if they were input tokens.

PrefixTuningConfig(num_virtual_tokens=20, prefix_projection=True)

Prompt Tuning

Prepends trainable soft prompt embeddings only at the input layer (not at every layer). Fewer parameters than prefix tuning but less expressive.

PromptTuningConfig(num_virtual_tokens=20, prompt_tuning_init="RANDOM")

IA³ (Infused Adapter by Inhibiting and Amplifying Inner Activations)

Learns per-channel scaling vectors that rescale activations in attention and FFN layers. Extremely parameter-efficient (~0.01%) but less commonly used.

IA3Config(target_modules=["q_proj", "v_proj", "down_proj"])

AdaLoRA

Adaptive LoRA that dynamically allocates rank budget across layers based on importance scores. Starts with a total rank budget and prunes less important singular values during training.

AdaLoraConfig(init_r=12, target_r=8, target_modules=["q_proj", "v_proj"])

How PEFT Works

PEFT follows a consistent lifecycle regardless of method:

LoRA injects trainable low-rank matrices into attention layers while keeping the base model frozen, dramatically reducing fine-tuning memory and compute.

Source: Microsoft Research

The PEFT Model Wrapper

get_peft_model() wraps the base model and:

1. Freezes all base model parameters (requires_grad=False)
2. Injects the chosen adapter modules
3. Enables training only on adapter parameters
4. Adds save/load methods for adapter-only checkpoints

from peft import get_peft_model, LoraConfig

config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
peft_model = get_peft_model(base_model, config)
peft_model.print_trainable_parameters()
# trainable params: 4,194,304 || all params: 6,738,415,616 || trainable%: 0.0623%

Saving and Loading

PEFT saves only adapter weights and config - not the full model:

my_adapter/
├── adapter_config.json    # Method, rank, target modules, etc.
├── adapter_model.safetensors  # Adapter weights only (~10-500MB)

Loading requires the original base model plus the adapter:

from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
model = PeftModel.from_pretrained(base, "./my_adapter")

Merging

For deployment, merge adapter weights into the base model:

merged = model.merge_and_unload()
merged.save_pretrained("./merged_model")

Architecture

PEFT sits between the base model and the training/serving infrastructure:

Step-by-Step Flow

Step 1: Choose a PEFT method. Default to LoRA. Consider alternatives only with specific reasons (see Design Decisions).

Step 2: Load the base model. FP16/BF16 for standard training, 4-bit with bitsandbytes for QLoRA.

Step 3: Create the PEFT config. Set method-specific hyperparameters.

Step 4: Wrap the model. model = get_peft_model(base_model, config).

Step 5: Prepare dataset and train. Use TRL's SFTTrainer or Transformers Trainer. Only adapter parameters receive gradients.

Step 6: Evaluate. Compare against base model baseline on your eval set.

Step 7: Save adapter. model.save_pretrained("./adapter") - small file, not full model.

Step 8: Deploy. Merge for single-model serving, or load adapter dynamically for multi-tenant.

Real Production Example

A platform serving three task-specific models from one Llama 8B base - classification, summarization, and extraction - using PEFT with different methods compared:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import (
    LoraConfig,
    PrefixTuningConfig,
    PromptTuningConfig,
    get_peft_model,
    PeftModel,
)
import torch

MODEL_ID = "meta-llama/Llama-3.1-8B-Instruct"

base_model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

# Method 1: LoRA (used for classification - best accuracy)
lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_dropout=0.05,
    task_type="CAUSAL_LM",
)
lora_model = get_peft_model(base_model, lora_config)
# trainable: 41.9M params (0.52%)

# Method 2: Prefix Tuning (used for summarization - good quality, tiny adapter)
prefix_config = PrefixTuningConfig(
    num_virtual_tokens=20,
    task_type="CAUSAL_LM",
)
# prefix_model = get_peft_model(base_model, prefix_config)
# trainable: 1.6M params (0.02%)

# Method 3: Prompt Tuning (used for extraction - fastest to train)
prompt_config = PromptTuningConfig(
    num_virtual_tokens=20,
    task_type="CAUSAL_LM",
)
# prompt_model = get_peft_model(base_model, prompt_config)
# trainable: 0.16M params (0.002%)

# Training (LoRA example)
from trl import SFTTrainer
from transformers import TrainingArguments

training_args = TrainingArguments(
    output_dir="./classifier-lora",
    num_train_epochs=2,
    per_device_train_batch_size=4,
    learning_rate=2e-4,
    bf16=True,
    eval_strategy="epoch",
)

trainer = SFTTrainer(
    model=lora_model,
    args=training_args,
    train_dataset=classification_dataset,
    eval_dataset=classification_eval,
    processing_class=tokenizer,
)
trainer.train()
lora_model.save_pretrained("./adapters/classifier")

# Multi-adapter serving
class MultiAdapterServer:
    def __init__(self, base_model_id: str):
        self.base = AutoModelForCausalLM.from_pretrained(
            base_model_id, torch_dtype=torch.bfloat16, device_map="auto"
        )
        self.adapters = {}
        self.active_adapter = None

    def load_adapter(self, name: str, path: str):
        if name not in self.adapters:
            self.base = PeftModel.from_pretrained(
                self.base, path, adapter_name=name
            )
            self.adapters[name] = path
        self.base.set_adapter(name)
        self.active_adapter = name

    def generate(self, prompt: str, adapter: str, **kwargs) -> str:
        self.load_adapter(adapter, self.adapters[adapter])
        inputs = tokenizer(prompt, return_tensors="pt").to(self.base.device)
        outputs = self.base.generate(**inputs, max_new_tokens=256, **kwargs)
        return tokenizer.decode(outputs[0], skip_special_tokens=True)

server = MultiAdapterServer(MODEL_ID)
server.load_adapter("classifier", "./adapters/classifier")
server.load_adapter("summarizer", "./adapters/summarizer")
server.load_adapter("extractor", "./adapters/extractor")

result = server.generate("Classify: ...", adapter="classifier")

Their results across methods on the same classification task:

LoRA captured 99.7% of full fine-tuning quality at 0.5% of the cost. They used LoRA for all production adapters.

Design Decisions

When to Use Each Method

⚠ Common Mistakes

1. Using prompt tuning for complex tasks. Prompt tuning modifies only input embeddings - it can't change how the model processes information through layers. Use LoRA for anything beyond simple classification.

2. Not calling print_trainable_parameters(). If this shows 100% trainable, your PEFT config didn't apply correctly. Debug before wasting GPU hours.

3. Saving the full model instead of the adapter. model.save_pretrained() on a PEFT model saves only adapters. But trainer.save_model() may save the full model depending on config. Verify your saved files are small (~100MB, not ~16GB).

4. Loading adapter without the matching base model. Adapters are tied to a specific base model architecture and version. Loading a Llama 3.1 adapter on a Llama 3 base model will fail or produce garbage.

5. Mixing PEFT methods on the same model. Don't combine LoRA and prefix tuning on the same base model without understanding the interactions. Pick one method per adapter.

6. Ignoring inference overhead of non-LoRA methods. Prefix tuning adds virtual tokens to every layer's KV cache, increasing memory and latency. LoRA (merged) has zero overhead.

7. Forgetting to set task_type. PEFT configs need task_type="CAUSAL_LM" for decoder-only models or task_type="SEQ_2_SEQ_LM" for encoder-decoder. Missing this causes silent config errors.

Where It Breaks Down

Tasks requiring full model capacity. PEFT methods approximate full fine-tuning. For tasks where the base model lacks fundamental capability, no adapter method will close a large gap.

Non-HuggingFace models. PEFT integrates with Transformers. Models outside this ecosystem (some proprietary APIs, custom architectures) need manual adapter implementation.

Prefix/prompt tuning at inference. Unlike mergeable LoRA, prefix and prompt tuning require the PEFT runtime at inference. This adds complexity and latency compared to a merged model.

Composing multiple adapters. Loading multiple LoRA adapters on one model can cause unpredictable behavior. Multi-adapter serving is an active research area (LoRAHub, S-LoRA) with production limitations.

Very old or unusual model architectures. PEFT's target_modules auto-detection works for standard transformer architectures. Custom or legacy models may need manual module specification.

Running in Production

Best Practice

✅ Best Practices - Instrument every stage, version embedding models, enforce access control at retrieval time, and evaluate on a fixed golden set before shipping changes.

Important

For production, default to LoRA with merge-at-deploy. Use dynamic adapter loading only when you genuinely need multi-tenant adapter swapping and can accept the latency tradeoff.

Ecosystem

Related Technologies

• LoRA: The most important PEFT method - dedicated deep dive.

• QLoRA: PEFT + 4-bit quantization for memory-constrained training.

• Fine-Tuning: When to fine-tune and how PEFT fits in the workflow.

• RLHF: Alignment training typically uses PEFT/LoRA adapters.

• DPO: Preference optimization with PEFT reduces memory for alignment training.

Learning Path

Prerequisites: Fine-Tuning · LoRA

Next topics: QLoRA · RLHF · DPO

Estimated time: 50 min · Difficulty: Intermediate

FAQs

What is PEFT?

PEFT (Parameter-Efficient Fine-Tuning) is a HuggingFace library that lets you fine-tune large models by training only 0.01–1% of parameters using methods like LoRA, adapters, and prefix tuning.

Which PEFT method should I use?

LoRA for 90% of cases. It has the best quality-to-cost ratio, zero inference overhead when merged, and the widest ecosystem support. Consider prefix tuning only when you need hundreds of tiny adapters.

How is PEFT different from LoRA?

PEFT is the library; LoRA is one method within it. PEFT also supports adapters, prefix tuning, prompt tuning, IA³, and AdaLoRA. When people say "PEFT fine-tuning," they usually mean LoRA via the PEFT library.

Can I switch PEFT methods without changing my training pipeline?

Mostly yes. Change the config object (LoraConfig → PrefixTuningConfig) and the rest of the pipeline (dataset, trainer, save/load) stays the same. This is PEFT's main value.

How do I use PEFT with QLoRA?

Load the base model with BitsAndBytesConfig(load_in_4bit=True), call prepare_model_for_kbit_training(), then apply PEFT config as usual. See the QLoRA guide.

Can I serve multiple PEFT adapters on one model?

Yes. Use PeftModel.load_adapter() and set_adapter() to switch between adapters. For production multi-adapter serving, use LoRAX or S-LoRA. Merged LoRA does not support multi-adapter.

How do I merge a PEFT adapter?

from peft import PeftModel
model = PeftModel.from_pretrained(base_model, "./adapter")
merged = model.merge_and_unload()
merged.save_pretrained("./merged_model")

What files does PEFT save?

adapter_config.json (method config) and adapter_model.safetensors (adapter weights only). Typically 10–500MB, not the full model size.

Does PEFT work with TRL for DPO/RLHF?

Yes. Pass a PEFT-wrapped model to TRL's DPOTrainer or PPOTrainer. This is the standard way to do alignment training efficiently.

How do I find the right target_modules for my model?

from peft import LoraConfig
# Auto-detect for common architectures
config = LoraConfig(task_type="CAUSAL_LM", r=16)
# Or inspect manually:
for name, module in model.named_modules():
    if isinstance(module, torch.nn.Linear):
        print(name)

Is AdaLoRA better than LoRA?

AdaLoRA can find better rank allocation per layer, but adds training complexity. For most tasks, standard LoRA with r=16 is simpler and equally effective. Use AdaLoRA when you've tuned LoRA rank and still see underfitting.

Can I use PEFT with models not on HuggingFace Hub?

Yes, if the model is compatible with Transformers' PreTrainedModel API. Load locally with from_pretrained("/path/to/model") and apply PEFT as usual.

References

• LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
• Hugging Face PEFT Documentation
• Microsoft LoRA Paper

Further Reading

• QLoRA Paper
• bitsandbytes GitHub
• Hugging Face Fine-tuning Guide

Summary

• PEFT is the standard HuggingFace abstraction for efficient fine-tuning - one API for LoRA, adapters, prefix tuning, and more. - LoRA is the default method: best quality, mergeable, zero inference overhead. - Adapters are small (10–500MB), swappable, and enable multi-tenant serving from one base model. - PEFT integrates with TRL (SFT, DPO, RLHF), bitsandbytes (QLoRA), and serving frameworks (LoRAX, vLLM).

• For production: LoRA → train → merge → deploy. Use dynamic adapter loading only for genuine multi-tenant needs.