PEFT Explained: LoRA, QLoRA, Adapters and Prefix Tuning Compared

Parameter-Efficient Fine-Tuning (PEFT) is a collection of techniques for adapting a pretrained model. It updates only a small subset of the model's parameters, or adds a few new trainable ones.

Traditional full fine-tuning updates the model's parameters across the board. Gradients, optimiser states and checkpoints all grow with model size, so large LLMs need a lot of GPU memory, computation and storage.

LoRA, QLoRA, Adapters and Prefix Tuning are four popular PEFT methods. Each one cuts the resources needed to customise a large model while keeping much of its pretrained knowledge.

This guide explains how these techniques work and how to choose between them.

Interactive tool · Generative AI roadmap

Your roadmap to becoming a Generative AI Engineer

See what you already know, what to learn next and how long the switch to a GenAI engineering role takes. Tap any step for what to study and build.

✦ Personalise your roadmap3 quick questions · takes 30 seconds

Your name optional Current role Select your role Student / Fresher Software Engineer (Backend) Software Engineer (Frontend) Full-stack Engineer Data Analyst Data Scientist Machine Learning Engineer DevOps / Cloud Engineer QA / Test Engineer Solutions / Sales Engineer Technical Consultant Product Manager Non-technical role Years of experience Select experience 0–1 years (fresher) 1–3 years 3–5 years 5–8 years 8+ years Target companies optional

AI familiarity

NoneHaven't used AI tools for work BeginnerUse ChatGPT-style tools IntermediateBuilt side projects with LLM APIs AdvancedShipped AI features to production

Tech stack optional · pre-filled for your role

Choose your role to pre-fill a typical stack.

Python Java JavaScript / TypeScript Go C++ SQL React / Vue / Angular Node.js Django / Flask / FastAPI Spring AWS GCP Azure Docker Kubernetes Machine Learning / scikit-learn PyTorch / TensorFlow LLM APIs (OpenAI, Claude, Gemini) LangChain / LlamaIndex Vector databases No coding background + Edit tech stack

Generate my roadmap →

What's inside8 stages · 30 steps to job-ready

1. Python & engineering foundations
2. ML & deep learning essentials
3. How LLMs work
4. Prompting & building LLM apps
5. RAG & model customisation
6. Agents & tool use
7. Production, evaluation & safety
8. Portfolio & job readiness

Browse the full roadmap without personalising

Learn this Level up You have it Done

Show only what I need

Stage 1 · Start here

Python & engineering foundations

Python for AI work ~3 wks

Data handling with SQL & pandas ~2 wks

Git, APIs & JSON ~1 wk

Maths you actually need ~2 wks

Stage 2 · Core concepts

ML & deep learning essentials

Machine learning basics ~2 wks

NLP & embeddings ~2 wks

Neural networks with PyTorch ~3 wks

Stage 3 · Understand the engine

How LLMs work

Transformer architecture ~2 wks

Hugging Face & open models ~2 wks

Models, tokens & context windows ~1 wk

Small language models optional ~1 wk

Stage 4 · Start shipping

Prompting & building LLM apps

Prompt engineering ~1 wk

LLM apps with APIs ~2 wks

Structured outputs & function calling ~1 wk

LangChain & LlamaIndex ~1 wk

Stage 5 · Make models useful

RAG & model customisation

Embeddings & vector databases ~2 wks

Fine-tuning & PEFT (LoRA) ~2 wks

Multimodal & vision models optional ~2 wks

Retrieval-augmented generation ~2 wks

Preference tuning (RLHF, DPO) optional ~2 wks

Stage 6 · Automate workflows

Agents & tool use

Agent design patterns ~2 wks

Tools, MCP & system integration ~2 wks

Orchestration with LangGraph ~2 wks

Stage 7 · Make it reliable

Production, evaluation & safety

Containers & deployment ~2 wks

Evaluation (evals) ~2 wks

Monitoring, cost & latency ~1 wk

Cloud AI services ~2 wks

Guardrails & responsible AI ~1 wk

Stage 8 · Get hired

Portfolio & job readiness

Interview preparation ~1 wk

Capstone project & portfolio ~3 wks

Show the full roadmap (8 stages · 30 steps)

GenAI glossary 17 terms

LLM Transformer Token Context window Prompt engineering Embeddings Vector database RAG Fine-tuning LoRA Agent Tool calling MCP Evals Guardrails Hallucination Prompt injection

Tap a term to see what it means. You will hear these in every GenAI interview.

LLM

Large language model: a neural network trained on huge amounts of text to predict and generate language.

Transformer

The neural network architecture behind modern LLMs, built around the attention mechanism.

Token

A chunk of text (often part of a word) that a model reads and writes. Pricing and limits are counted in tokens.

Context window

The maximum number of tokens a model can consider at once, including your prompt and its answer.

Prompt engineering

Designing instructions and examples so a model reliably produces the output you need.

Embeddings

Numeric vectors that capture meaning, so similar texts end up close together.

Vector database

A database built to store embeddings and find the most similar ones quickly.

RAG

Retrieval-augmented generation: fetching relevant documents and giving them to the model so answers are grounded.

Fine-tuning

Further training a pre-trained model on your own data to specialise it.

LoRA

Low-rank adaptation: a parameter-efficient fine-tuning method that trains small add-on weights instead of the whole model.

Agent

An LLM system that plans steps, calls tools and acts towards a goal, often over several turns.

Tool calling

Letting a model call functions or APIs, such as search or a database query, with structured arguments.

MCP

Model Context Protocol: an open standard for connecting AI applications to tools and data sources.

Evals

Systematic tests that measure how well an AI system performs on representative cases.

Guardrails

Checks on inputs and outputs that keep an AI system safe, on-topic and compliant.

Hallucination

When a model states something false or unsupported with confidence.

Prompt injection

An attack where malicious text tricks a model into ignoring its instructions.

IIT Delhi CEP × Varsity

Close your gaps with a structured programme

The Certificate Programme in Generative AI from CEP, IIT Delhi covers transformers, fine-tuning, RAG, agents and responsible AI through live weekend classes and hands-on projects.

• 6 months
• Live online
• Weekend classes

Enquire about the programme View programme details

How does Parameter-Efficient Fine-Tuning Work?

PEFT adapts a pretrained model without updating every original parameter. It freezes most of the model and trains a smaller set of parameters or lightweight modules that specialise it for a task.

Most PEFT methods follow the same five-step workflow, shown below.

Step 1 : Start With a Pretrained Model

PEFT starts with a model that has already been pretrained. During pretraining, an LLM learns language patterns, facts and reasoning by predicting text across very large datasets.

That general-purpose knowledge is why PEFT works. For a specialised application, such as a legal-clause classifier, training from scratch is usually unnecessary. You only need to steer what the model already knows.

Step 2 : Freeze Most of the Model Parameters

Freezing a parameter removes it from gradient updates, so its value never changes. In PEFT, the original model weights stay exactly as they were after pretraining.

Frozen weights need no gradients or optimiser states, so the number of trainable parameters drops sharply. That lowers memory needs, although the frozen weights still have to be loaded for every training step.

Step 3 : Introduce Trainable Parameters or Modules

Next, the PEFT method adds or modifies a small set of trainable parameters. This is where the techniques differ most, because each one changes the model in a different place. The three main approaches are:

• Low-rank matrices (LoRA and QLoRA) learn a compact update to existing weight matrices.
• Adapter layers insert small trainable networks inside transformer blocks.
• Trainable prefix vectors (Prefix Tuning) add learned vectors that the attention mechanism reads as extra context.

Step 4 : Train the Model on Task-Specific Data

The model now learns from specialised examples, such as labelled support tickets or instruction–response pairs. Gradients flow back through the frozen layers, but only the selected trainable parameters change.

Dataset quality and task relevance still decide the result. In the QLoRA study, a 9,000-example dataset beat a 450,000-example dataset on chatbot performance.

Step 5 : Save and Deploy the Adapted Model

After training, only the PEFT parameters are saved, separately from the base model. The file is usually measured in megabytes rather than gigabytes.

At inference, the adapter or LoRA weights load alongside the original model. One base model can serve many tasks, each with its own small set of weights. This cuts storage when you keep several task-specific versions.

Key point: PEFT reduces the number of parameters that need updating; it does not mean the model requires no training or computational resources.

What is LoRA (Low-Rank Adaptation)?

LoRA is a PEFT technique that keeps the pretrained weights frozen. It learns small low-rank matrices that change each selected layer's effective weight transformation.

LoRA was introduced in the original LoRA paper by Hu et al.. Today it is one of the most popular ways to fine-tune large language models.

How does LoRA Work?

LoRA adds two small trainable matrices, A and B, next to a selected pretrained weight matrix W₀. W₀ stays frozen, and only A and B are trained.

Their product, BA, is a low-rank weight update (ΔW). During computation, it is added to the output of the frozen weights:

h = W₀x + (α / r) · BAx

Here, x is the layer input and α is a scaling constant. The rank r is the inner dimension of A and B, so it controls the size of the trainable update.

For example, a 4,096 × 4,096 matrix holds about 16.8 million weights. LoRA with r = 8 trains 8 × (4,096 + 4,096) = 65,536 values for it, roughly 0.4%.

B starts at zero, so training begins from exactly the base model's behaviour. The LoRA authors found that a rank of just 1 or 2 was enough for GPT-3 175B.

Why does LoRA Reduce Fine-Tuning Costs?

LoRA's savings come from how small its trainable update is. They are largest when the base model is big and you need many task-specific versions. LoRA reduces cost in three ways:

• Less gradient and optimiser-state memory. Optimisers such as Adam store extra values for every trainable parameter. On GPT-3 175B, LoRA cut GPU memory by 3× compared with full fine-tuning.
• Small, separate checkpoints. Only A and B are saved. In the paper, a GPT-3 checkpoint fell from about 350 GB to about 35 MB.
• Many adapters, one base model. Several LoRA adapters can share one frozen base model, and you can swap between them for each task.

What are the Advantages and Limitations of LoRA?

LoRA balances efficiency and flexibility, which explains why it is so widely used. Its strengths come from its compact update, and its weaknesses from the configuration choices that update needs. Its main advantages are:

• Fewer trainable parameters than full fine-tuning.
• Lower storage requirements for task-specific updates.
• Can be merged into the base weights (W₀ + BA) for deployment in supported setups, adding no latency.
• Widely supported by modern LLM fine-tuning tools, including Hugging Face's PEFT library.

LoRA is not a free win, though. Weigh these limitations against your task and hardware before you commit to it:

• Performance depends on rank, target layers and training configuration.
• It still needs suitable GPU resources, because the full base model is loaded during training.
• It may not match full fine-tuning on every task.

Key point: LoRA makes fine-tuning more efficient by learning a compact weight update instead of retraining the entire model.

What is QLoRA (Quantized LoRA)?

QLoRA extends LoRA by combining low-rank adaptation with quantisation. It fine-tunes a quantised pretrained model while the original model weights stay frozen.

The QLoRA paper by Dettmers et al. used it to fine-tune a 65B-parameter model on a single 48 GB GPU.

How does QLoRA Work?

QLoRA changes how the frozen base model is stored, not how the adapters learn. The process has four parts, and only the adapters are ever updated:

1. The pretrained model is quantised to a lower-bit format, commonly 4-bit.
2. The quantised base model stays frozen throughout fine-tuning.
3. Trainable LoRA adapters are added to selected layers. The original paper added them to every linear layer.
4. For each computation, the 4-bit weights are converted back to 16-bit (BFloat16). Gradients update only the adapters, which stay in higher precision.

What is Quantisation in QLoRA?

Quantisation means storing model weights with fewer bits. A 16-bit weight takes 2 bytes and a 4-bit weight takes half a byte, so 4-bit weights need about a quarter of the GPU memory.

Plain 4-bit rounding loses too much precision for reliable training. The original QLoRA research added three techniques to solve this:

• 4-bit NormalFloat (NF4): a data type designed for normally distributed weights, which pretrained weights usually are. Each of its 16 values covers a similar share of the weights.
• Double quantisation: quantises the scaling constants from the first quantisation step. This saves about 0.37 bits per parameter, or roughly 3 GB on a 65B model.
• Paged optimisers: use NVIDIA unified memory to move optimiser states to CPU RAM when GPU memory spikes, then move them back when they are needed.

For more on compression methods, read our guide to LLM quantisation [Internal link placeholder: add the URL of the Varsity article on LLM quantisation].

What are the Advantages and Limitations of QLoRA?

QLoRA's value is almost entirely about memory. Its trade-offs come from the extra computation and numerical approximation that quantisation adds to training. Compared with standard LoRA, its main advantages are:

• Lower memory requirements than standard full-precision LoRA setups, because frozen 4-bit weights take about a quarter of the space.
• Makes it possible to fine-tune larger models on limited hardware. The paper reports 33B models on a 24 GB GPU.
• Keeps LoRA's parameter-efficient approach, including small, separate adapter files.

These gains come with conditions that affect accuracy, speed and setup. Check the following limitations before you plan a QLoRA training run:

• Quantisation can introduce numerical approximation effects. The paper did not confirm that QLoRA matches full 16-bit fine-tuning at the 33B and 65B scales.
• Training speed and performance depend on hardware and implementation, and converting weights back to 16-bit adds computation.
• It does not remove the need for enough GPU memory or careful configuration, because activations still use memory.

Key point: LoRA reduces the number of trainable parameters, while QLoRA additionally reduces the memory required to hold the frozen base model.

What is Adapter Tuning?

Adapter tuning adds small trainable neural network modules between or within the layers of a pretrained model. Most of the original parameters stay frozen.

The approach was established for NLP by Houlsby et al.'s adapter study, which tested it with BERT on 26 text classification tasks.

How do Adapters Work?

An adapter is a bottleneck module. It compresses a hidden representation to a much smaller size, applies a non-linearity, then expands it back, with a skip connection around the whole module.

Inside a transformer block [Internal link placeholder: add the URL of the Varsity article on transformer architecture], each representation passes through the adapter before it continues through the model.

The original design placed two adapters in each layer: one after the attention sub-layer and one after the feed-forward sub-layer. Only adapter parameters (plus layer norms) are trained. Placement varies across architectures and implementations.

What are the Advantages of Adapter Tuning?

Adapters suit teams that treat each task as a separate, swappable component. In the original study, they came within 0.4% of full fine-tuning on GLUE. Their main benefits are:

• Lets multiple task-specific adapters share one pretrained model.
• Keeps most original weights unchanged, so training one task does not overwrite another.
• Gives modularity to applications with several specialised tasks, such as one adapter per language.
• Useful when task-specific modules must be versioned and maintained separately.

What are the Limitations of Adapter Tuning?

Adapters add new layers rather than changing existing ones. As a result, their trade-offs show up mainly at inference time and in day-to-day model operations. The main limitations are:

• Adds extra modules to the model architecture.
• May add inference overhead. The LoRA authors note that adapters can add latency because they make the model deeper.
• Requires decisions about adapter placement, size and architecture.
• Can make deployment more complex when you manage many adapters.

Key point: Adapter tuning specialises a model by adding small trainable modules rather than modifying its original weights.

What is Prefix Tuning?

Prefix Tuning adds trainable continuous vectors, called prefixes, to the model's attention mechanism while all pretrained parameters stay frozen. Its simpler relative, prompt tuning, adds them only at the input.

Li and Liang's prefix-tuning paper introduced the method for text generation tasks using GPT-2 and BART.

How does Prefix Tuning Work?

Prefix Tuning adds a short sequence of trainable vectors in front of the activations at every transformer layer. In practice, they act as extra key and value vectors inside attention.

Real tokens can then attend to the prefix as if it were a set of "virtual tokens". During training, gradients update only these prefix vectors. This steers the output without changing the model's weights.

Prefixes are learned numbers, not text instructions, so you cannot read or write them like a prompt. Lester et al.'s prompt-tuning paper applies the same idea at the input layer, as "soft prompts".

What are the Advantages of Prefix Tuning?

Prefix Tuning's appeal is how little it stores per task. In the original study, it trained about 0.1% of the parameters and beat fine-tuning in low-data settings. Its main benefits are:

• Keeps the original pretrained model frozen.
• Requires training a relatively small set of extra parameters.
• Supports task-specific adaptation without changing the whole model.
• Lets you keep a different learned prefix for each task.

What are the Limitations of Prefix Tuning?

The trade-offs mostly come from the prefix itself and how the attention mechanism handles it. Consider these limitations before you choose Prefix Tuning for a project:

• Performance can depend on prefix length and model architecture.
• Managing learned prefixes can require careful implementation.
• It may do worse than other PEFT approaches on some tasks. The original paper saw a small drop for summarisation.
• Extra prefix representations add to attention computation and can reduce the usable sequence length.

Key point: Prefix Tuning adapts model behaviour by learning additional context representations rather than changing the pretrained model's weights.

LoRA vs QLoRA vs Adapters vs Prefix Tuning: What Is the Difference?

All four approaches reduce the cost of adapting a pretrained model. They differ in where they add trainable parameters, how they affect model computation and which hardware limits they address.

Key Differences Between PEFT Methods

The table compares the four parameter-efficient fine-tuning methods across seven parameters. Read it row by row to see where each method adds its trainable parameters and how that affects inference.

Parameter	LoRA	QLoRA	Adapters	Prefix Tuning
Main technique	Low-rank weight updates	Quantised base model + LoRA	Additional trainable modules	Trainable prefix vectors
Base model	Usually frozen	Quantised and frozen	Usually frozen	Frozen
Trainable parameters	Low-rank matrices	LoRA matrices	Adapter modules	Prefix parameters
Quantisation	Not required	Core component of original approach	Not required	Not required
Main benefit	Efficient weight adaptation	Reduced base-model memory	Modular task-specific adaptation	Lightweight task conditioning
Inference considerations	Can often be merged	Depends on quantised deployment setup	May add module overhead	Adds prefix-related attention computation
Typical use	General PEFT fine-tuning	Memory-constrained LLM fine-tuning	Modular multi-task adaptation	Parameter-efficient task adaptation

Which PEFT Method Uses the Least Memory?

No single method always uses the least memory. Total memory depends on model size, quantisation, number of trainable parameters, optimiser, batch size, sequence length and implementation.

The trainable parameters are often the smallest part. In QLoRA's memory analysis of a 7B model, the LoRA weights took 26 MB, while the 4-bit base model took 5,048 MB.

That is why QLoRA is so useful: it shrinks the biggest fixed cost, the frozen base model. Other PEFT methods also train few parameters, but without quantisation they still hold the full base model in memory.

Which Method is Best for Production Deployment?

The best choice depends on what production means for your team: low latency, limited memory or many tasks to manage. Each method fits a different deployment priority:

• LoRA: flexible deployment and adapter management. Merge it into the base model for no added latency, or keep adapters separate and swap them per task.
• QLoRA: suits training environments with limited memory. Before serving, confirm that your inference stack supports the quantised base model.
• Adapters: suit modular, multi-task products that can accept some extra computation in each layer.
• Prefix Tuning: lightweight task conditioning when you can spare some context length.

Key point: There is no universally best PEFT technique. Choose based on model size, hardware, task requirements, training cost and deployment constraints.

PEFT vs Full Fine-Tuning: What Should Developers Choose?

Full fine-tuning updates all or most of a model's parameters during task-specific training. It gives the most freedom to change behaviour, but at the highest cost. Comparing it with PEFT shows when that cost is worth paying.

How does Full Fine-Tuning Work?

In full fine-tuning, every pretrained parameter is updated using task-specific data. Each training step calculates a gradient for every weight, and the optimiser updates them all.

Optimisers such as Adam also store two extra values for every parameter. All of this grows with model size. The QLoRA authors note that 16-bit fine-tuning of LLaMA 65B needs more than 780 GB of GPU memory.

For the end-to-end process, see our guide to fine-tuning large language models [Internal link placeholder: add the URL of the Varsity article on LLM fine-tuning].

Key Differences Between PEFT and Full Fine-Tuning

The table compares full fine-tuning vs PEFT across seven practical parameters. It focuses on resource use and deployment, the two factors that usually decide which approach a team picks.

Parameter	PEFT	Full Fine-Tuning
Trainable parameters	Small subset or additional modules	Most or all model parameters
GPU memory	Generally lower	Generally higher
Training cost	Usually lower	Usually higher
Model storage	Smaller task-specific updates may be stored separately	Separate full model checkpoints may be required
Adaptation flexibility	Depends on selected PEFT method	Direct updates across the model
Deployment	May require loading adapters with a base model	Can deploy a separately fine-tuned model
Best suited for	Resource-efficient adaptation	Tasks that benefit from broader parameter updates

When should you Choose PEFT?

PEFT is usually the practical default when resources are tight or you need flexibility across tasks. It is the better fit in the following situations:

• Limited GPU memory or computing resources.
• Several task-specific adaptations of one base model.
• A need for smaller fine-tuning checkpoints.
• Fast experiments across different datasets.

When should you Choose Full Fine-Tuning?

Full fine-tuning is worth its extra cost when the model needs deep changes or performance targets are strict. Consider it in the following cases, ideally after you have tried a PEFT baseline:

• When the task needs broad changes across the model's parameters.
• When you have enough training infrastructure.
• When experiments show PEFT does not reach the required performance.
• When the model and dataset justify the extra training cost.

Key point: PEFT is often a practical starting point for adapting LLMs, but full fine-tuning may still be appropriate when broader model adaptation is required.

How to Choose the Right PEFT Method for your LLM Project?

Choose a PEFT technique based on hardware limits, model architecture, task requirements and deployment needs. The sections below match each method to its best fit, followed by a decision table and a decision tree.

Choose LoRA When You Need Efficient Model Adaptation

LoRA is a sensible default for most LLM fine-tuning projects, and it is supported by most tools and model families. Start with LoRA when the following conditions apply:

• You want to fine-tune an LLM without updating all its parameters.
• You need reusable, compact task-specific adapters.
• Your hardware can hold the frozen base model at the precision you plan to use.

Choose QLoRA When GPU Memory Is Limited

QLoRA is the next step when the base model itself will not fit in memory. It keeps LoRA's training approach while compressing the frozen weights. Choose it when:

• You need to fine-tune a larger model on limited hardware.
• You want to reduce the memory taken by the base model's weights through quantisation.
• You can accept the implementation and numerical trade-offs of quantised training.

Choose Adapters When You Need Modular Task-Specific Components

Adapters make sense when your product is built from many distinct tasks that change on different schedules. They work best when the following conditions apply:

• You need separate modules for different tasks.
• You want to manage each task's components independently.
• Your architecture can run the extra modules efficiently.

Choose Prefix Tuning When You Need Lightweight Task Conditioning

Prefix Tuning suits tasks where a small learned context is enough to steer the model, such as generation tasks with limited data. It fits when:

• Your task can benefit from learned prefix representations.
• You want to keep the pretrained model's weights frozen.
• Your architecture and training setup support prefix-based adaptation.

The decision table sums up this guidance as one starting point for each project requirement. Treat each recommendation as a first option to benchmark, not a final answer.

Project requirement	Recommended starting point
Efficient general LLM fine-tuning	LoRA
Limited GPU memory	QLoRA
Multiple modular task-specific adaptations	Adapters
Lightweight learned task conditioning	Prefix Tuning
Broad model-wide adaptation	Full fine-tuning

What are the Limitations of PEFT?

PEFT reduces training requirements, but it comes with trade-offs. As the four limitations below show, it does not guarantee better performance, remove hardware needs or fix data-quality problems.

Performance May Differ From Full Fine-Tuning

Some tasks benefit from updating more parameters than PEFT allows, especially when the domain or behaviour must change a lot. Results depend on the method, dataset and model architecture.

The results showing parity in the LoRA and QLoRA papers come from specific benchmarks. Benchmark your adapted model against the base model, and against full fine-tuning where you can.

Hyperparameter Selection Can Be Difficult

Each PEFT method adds its own settings on top of the usual training ones. These include LoRA rank, adapter size, prefix length, learning rate and target layers.

A poor configuration can limit results without any obvious error. In QLoRA's experiments, LoRA on only the query and value layers fell short of full fine-tuning, while LoRA on all linear layers matched it. Run controlled experiments.

PEFT Still Requires Computational Resources

PEFT shrinks the set of trainable parameters, not the model. Every training step still needs a full forward pass, and the backward pass still runs through the frozen layers.

Model size, sequence length and batch size drive memory use, mostly through activations. QLoRA reduces memory requirements, but it does not make training resource-free.

Deployment and Compatibility Challenges

Every adapter is tied to the exact base model it was trained on. Teams therefore have to track which base-model version goes with each task-specific adapter.

Compatibility also varies across architectures and inference frameworks. For example, the Hugging Face PEFT quantisation guide notes that some quantisation tools limit or block adapter merging.

Key point: PEFT makes fine-tuning more resource-efficient, but successful deployment still depends on training quality, model compatibility, evaluation and infrastructure.

Conclusion

Choosing a PEFT method is less about finding the best technique and more about matching one to your constraints. Start with your model size and available hardware.

If the base model fits in memory at 16-bit precision, LoRA is a strong default. If it does not, QLoRA makes the same approach possible with far less memory.

Next, weigh the task and how you will deploy it. Adapters suit products built from many independent task modules, while Prefix Tuning suits lightweight conditioning with minimal storage.

Full fine-tuning remains the fallback when benchmarks show PEFT falls short. Whichever method you pick, leave time for evaluation, because rank, target layers and data quality affect results too.

Want to build these skills through hands-on projects? Explore Varsity's

Certificate Programme in Generative AI, offered by the Continuing Education Programme (CEP), IIT Delhi. It is designed to help you build, fine-tune and ship generative AI applications.

Frequently Asked Questions

What GPU specifications are recommended for training a 7B or 13B parameter model with PEFT?

There is no single specification, because memory depends on precision, sequence length, batch size and framework. In 16-bit, the weights alone need about 14 GB for 7B and 26 GB for 13B.

4-bit weights need roughly a quarter of that. The QLoRA paper measured about 5 GB for a 4-bit 7B model. Leave headroom for activations and adapters.

How do you select the optimal LoRA rank for a specific fine-tuning task?

Treat rank as a hyperparameter to test. Start low, with values such as 8 or 16, which are common in PEFT examples. Increase it only if validation results stop improving.

The LoRA authors found very low ranks enough for GPT-3. QLoRA saw little effect from rank once all linear layers were adapted. The best value still varies by task.

Which transformer layers should developers target when applying LoRA to an LLM?

The original LoRA paper adapted the attention query and value matrices, and Hugging Face PEFT's defaults follow the same pattern.

However, QLoRA found that targeting all linear layers was needed to match full fine-tuning. In PEFT, target_modules="all-linear" does this. Layer names vary between models, so check your architecture first.

What is DoRA, and how does it compare with traditional LoRA?

DoRA (Weight-Decomposed Low-Rank Adaptation) splits each pretrained weight into a magnitude and a direction. It trains the magnitude directly and updates the direction with LoRA.

The DoRA paper by Liu et al. reports gains over LoRA, such as +3.4 points on LLaMA-7B commonsense reasoning. It adds training overhead but can be merged for deployment.

How do you combine multiple LoRA adapters for multi-task language model applications?

To switch between tasks, load each adapter onto the same base model under its own name. Then turn on one at a time with set_adapter().

To blend tasks, PEFT's add_weighted_adapter() combines several LoRAs using methods such as linear, TIES or DARE. Only combine adapters trained on the same base model, and test each blend.

Can LoRA adapters be shared or transferred between different versions of the same base model?

Sharing works when the base model is identical: anyone with the same weights can load a published adapter. Moving an adapter to a different version is unreliable, even when the matrix sizes match.

The adapter learned an update for specific weights. PEFT records the base model's name in the adapter config, so retrain or re-test after any base-model change.

What are the most common errors developers encounter when training models with QLoRA?

Common problems include out-of-memory errors from long sequences or large batches. Others are unsupported compute data types, such as BFloat16 on older GPUs, and target layer names that do not match the model.

Skipping the recommended prepare_model_for_kbit_training() step can also break training. So can mixing incompatible versions of bitsandbytes, Transformers and PEFT.

How do you measure whether a PEFT model has overfitted its training dataset?

Track loss on a held-out validation set alongside training loss. Overfitting shows when training loss keeps falling while validation loss rises or task metrics stop improving.

Also compare the adapted model with the base model on general benchmarks to spot lost abilities. Early stopping, fewer epochs, LoRA dropout or more varied data can help.

What is the difference between LoRA merging and adapter loading during model deployment?

Merging adds the learned update into the base weights (W₀ + BA), creating one standalone model with no extra inference latency. PEFT's merge_and_unload() does this.

Adapter loading keeps the base model and the LoRA weights separate. You can then swap adapters at runtime for a small extra computing cost. A merged model serves only one task per copy.

How can developers reduce training time when fine-tuning large language models with PEFT?

Start with the data. Shorter, tightly packed sequences, and batching examples of similar length together, cut wasted computation. Use BFloat16 mixed precision where your GPU supports it, and the largest batch size memory allows.

Gradient checkpointing and QLoRA save memory but can add computing time, so use them only when memory is the limit. Test your settings on a small subset of the data first.

Which open-source tools and frameworks support parameter-efficient fine-tuning of LLMs?

The Hugging Face stack covers most needs: PEFT for LoRA, prefix tuning and prompt tuning; Transformers for models; TRL for supervised fine-tuning; and Accelerate for multi-GPU training.

bitsandbytes handles 4-bit and 8-bit quantisation. For bottleneck adapters, the AdapterHub Adapters library is a common choice. Microsoft's loralib is the LoRA authors' own reference implementation.

Sources

LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)

QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al., 2023)

Parameter-Efficient Transfer Learning for NLP (Houlsby et al., 2019)

Prefix-Tuning: Optimizing Continuous Prompts for Generation (Li and Liang, 2021)

The Power of Scale for Parameter-Efficient Prompt Tuning (Lester et al., 2021

DoRA: Weight-Decomposed Low-Rank Adaptation (Liu et al., 2024)