Can I fine-tune LLMs with FrameTrain?

Yes. FrameTrain supports fine-tuning of Large Language Models (LLMs) via LoRA and QLoRA. You can import models like Llama, Mistral, or Phi directly from HuggingFace and adapt them to your own datasets.

Which GPUs are supported?

FrameTrain runs on all NVIDIA GPUs with CUDA support (from the GTX 10 series onward) as well as on Apple Silicon (M1/M2/M3/M4) via Metal Performance Shaders. Pure CPU-based training is also possible.

What's the difference compared to Google Colab or AWS SageMaker?

FrameTrain runs entirely locally on your machine. There are no ongoing cloud costs, your data is never uploaded, and you're not dependent on an internet connection. Ideal for sensitive or proprietary datasets.

Do I need programming knowledge?

No. FrameTrain is designed as a graphical application. You configure training parameters through an intuitive interface without writing any code. Advanced users can also use configuration files.

Does FrameTrain support LoRA training?

Yes, LoRA (Low-Rank Adaptation) is fully integrated. You can configure LoRA rank, alpha, and target modules directly in the app, which also enables fine-tuning on GPUs with limited VRAM.

Why does FrameTrain only cost €4.99/month?

FrameTrain is in Early Access. The price is intentionally low so that as many developers and researchers as possible can get access early. The price increases to €9.99 after the first 100 users – sign up now to lock in the lower price permanently.

Does FrameTrain run on Apple M1/M2/M3/M4?

Yes. FrameTrain natively supports Apple Silicon via Metal Performance Shaders (MPS). Models are accelerated on the M-chip's GPU without requiring CUDA.

Which model formats are supported?

FrameTrain supports PyTorch models (.pt, .bin), SafeTensors, and the GGUF format. Models can be imported directly from the HuggingFace Hub or loaded locally.

Can I use my own datasets?

Yes. You can load your own datasets in CSV, JSON, or JSONL format. FrameTrain also supports datasets directly from the HuggingFace Datasets Hub.

Does FrameTrain support mixed precision and gradient checkpointing?

Yes. FP16 and BF16 mixed-precision training are fully supported, as is gradient checkpointing for memory-efficient training on GPUs with limited VRAM.

Docs AI Training Coach🔧 Fine-Tuning Methods

Full Fine-Tuning

Adjusting all model parameters – maximum control

In full fine-tuning, all parameters of the base model are trained and adjusted. Maximum expressiveness, but maximum memory requirements.

Pros and cons

✓ Advantages

Maximum adaptability
Best possible performance
No extra overhead from adapters
Good for fundamental domain shifts

✗ Disadvantages

Huge VRAM requirement (~28GB for 7B)
Catastrophic forgetting possible
Needs much more data
Every checkpoint = full model size

Catastrophic forgetting – the main risk

During full fine-tuning, the model can "forget" its general prior knowledge if trained too aggressively on new data. This happens when new knowledge overwrites the old weights.

Countermeasures against catastrophic forgetting:

  1. Low learning rate (1e-5 or lower)
     → Small updates preserve existing knowledge

  2. Few epochs (1–3)
     → Don't grind the model too long into new data

  3. Elastic Weight Consolidation (EWC)
     → Penalizes large changes to important parameters
     → More complex, but effective

  4. Replay (mixed training)
     → Mix of old (general) and new (specific) data
     → e.g. 80% new task + 20% general text

Full fine-tuning vs. LoRA: when to use which?

VRAM < 24 GBLoRA / QLoRAFull FT doesn't fit in memory

VRAM 24–40 GB, maximum performance desiredFull fine-tuning (bf16)Best result when memory allows

Little training data (< 1,000 examples)LoRA (small rank)Fewer trainable parameters = less overfitting

Fast iteration, many experimentsLoRASmaller checkpoints, faster training

Multiple task-specific versions of a modelLoRA adaptersLoad base model once, swap adapters

Model needs to learn fundamentally new knowledgeFull fine-tuningAdjusting all layers for deep changes

Full fine-tuning checklist

Before starting full fine-tuning: ✓ VRAM ≥ 24 GB, ✓ bf16 enabled, ✓ LR at most 5e-5, ✓ at most 3 epochs planned, ✓ early stopping enabled, ✓ best-checkpoint saving on.

LoRA In Depth

Low-Rank Adaptation – the key to efficient fine-tuning

LoRA (Low-Rank Adaptation), introduced in 2022 by Hu et al. (Microsoft), is the most important technique for efficient LLM fine-tuning. It reduces trainable parameters by 99%+ with minimal performance loss.

The mathematical core idea

Instead of changing the entire weight matrix W (d×d), LoRA freezes W and adds two small matrices A (d×r) and B (r×d) whose product represents the change:

Standard fine-tuning:
  W_new = W + ΔW    (ΔW is the same size as W!)
  7B model: ΔW ≈ 7 billion parameters to train

LoRA:
  W_new = W + B·A   (W stays frozen)
  A: d×r  (down-projection)
  B: r×d  (up-projection)

Example (d=4096, r=16):
  Original matrix:  4096 × 4096 = 16.7M parameters
  LoRA matrices:     4096×16 + 16×4096 = 131K parameters
  Reduction:          99.2% fewer trainable parameters!

  During fine-tuning: only A and B are trained
  B is initialized to 0 → at the start: BA = 0 → no effect

LoRA architecture: frozen base model + trainable adapters

LoRA hyperparameters explained

rank (r)

4, 8, 16, 32, 64

Determines the "capacity" of the LoRA adapters. Higher rank = more trainable parameters = more expressiveness, but more memory and overfitting risk.

💡 r=8 for simple tasks/little data. r=16 as a standard. r=32–64 for complex tasks with lots of data.

Memory: r=8: ~2× fewer parameters than r=16

alpha (α)

Typical: equal to r or 2×r (e.g. 16 or 32)

Scaling factor for the LoRA update: actual update = (α/r) × B·A·x. Alpha/r determines the "learning strength" of the adapters.

💡 α = r (update strength = 1) or α = 2r (stronger updates). Start with α=16 (for r=16).

Memory: No memory effect (just a scaling constant)

dropout

0.0 to 0.1

Dropout within the LoRA adapter matrices. Prevents adapter overfitting, especially on small datasets.

💡 0.05 as standard. Below 500 examples: 0.1. Above 5,000 examples: 0.0

Memory: No memory effect

target_modules

q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Which layers of the transformer are extended with LoRA. More modules = more trainable parameters = better performance, but more memory.

💡 Minimum: q_proj + v_proj. Standard: all attention layers. Maximum: all linear layers including FFN.

Memory: Each target_module adds ~2×r parameters per layer

bias

"none" (default), "all", "lora_only"

Whether bias parameters are also trained. "none" = only LoRA matrices. "all" = biases too.

💡 "none" is the default and is almost always enough.

Memory: "all" adds minimal extra parameters

LoRA for different model types

BERT / RoBERTaquery, value (attention)r=8–16LR: 2e-5

GPT-2c_attn, c_projr=8–16LR: 5e-5

LLaMA 2/3q_proj,k_proj,v_proj,o_projr=16–32LR: 2e-4

Mistral 7Bq_proj,k_proj,v_proj,o_projr=16LR: 2e-4

Phi-3 / Gemmaq_proj,v_projr=8–16LR: 1e-4

QLoRA (4-bit Fine-Tuning)

Even more efficient: quantization + LoRA combined

QLoRA (Dettmers et al., 2023) combines 4-bit quantization of the base model with LoRA adapters at full precision. Result: fine-tune a 7B model in ~4 GB VRAM!

How QLoRA works

QLoRA = quantized base model + LoRA

Step 1: Load the base model in 4-bit (NF4 quantization)
  → 7B model: 28 GB (fp32) → ~4 GB (NF4)

Step 2: Add LoRA adapters in bf16/fp16
  → Adapters stay at full precision for quality

Step 3: During training:
  → Forward pass: 4-bit weights "dequantized" to bf16 on the fly
  → Gradients only for LoRA adapters (bf16)
  → Base model stays frozen and 4-bit

Total memory for 7B:
  Base model (NF4):     ~4 GB
  LoRA adapters (bf16): ~0.3 GB
  Activations:           ~1 GB
  Optimizer states:      ~0.6 GB
  Total:                 ~6 GB → an RTX 3070 is enough!

NF4 quantization – the innovation behind QLoRA

Standard int4 quantization quantizes weights uniformly. NF4 (4-bit NormalFloat) takes into account that transformer weights are normally distributed – it assigns bit values to boundaries that optimally partition the normal distribution:

Int4 (uniform steps):
  Bit values: -8, -7, -6, ..., 7, 8  (16 levels)
  Problem: most weights lie near 0,
           extreme values are rare → poor resolution

NF4 (normally distributed steps):
  Bit values: -1, -0.69, -0.52, ..., 0.52, 0.69, 1
  (quantiles of the normal distribution)
  Advantage: better resolution where most weights lie
  Result: noticeably better quality than int4

QLoRA vs. LoRA vs. Full FT compared

Method	VRAM (7B)	Quality (rel.)	Speed	Recommendation
Full FT (fp32)	~60 GB	100%	⭐⭐⭐	A100/H100 cluster
Full FT (bf16)	~40 GB	99%	⭐⭐⭐	A100 40GB
Full FT (8-bit)	~30 GB	97%	⭐⭐	A100/2× RTX 3090
LoRA (bf16)	~16 GB	97%	⭐⭐⭐⭐	RTX 3090 / A6000
QLoRA (4-bit) ⭐	~6 GB	93–96%	⭐⭐⭐	RTX 3070/4060 Ti
QLoRA (4-bit, GC)	~4–5 GB	92–95%	⭐⭐	GTX 1660 Ti or better

GC = gradient checkpointing enabled. Quality relative to full FT fp32.

QLoRA: democratizing LLM training

Before QLoRA, 7B fine-tuning needed 28 GB+ VRAM (high-end servers only). With QLoRA it works on an RTX 3070 (8 GB VRAM). 13B models are possible with 16 GB. This makes LLM fine-tuning accessible to everyone.

Which Fine-Tuning Method When?

The complete decision tree

The choice of fine-tuning method depends on VRAM, amount of data, quality requirements, and training speed. This decision tree leads you to the optimal choice.

Step-by-step decision tree

1. Determine VRAM

How much GPU VRAM do you have?

< 6 GB→ QLoRA with gradient checkpointingThe only realistic option. RTX 3060 6GB: 7B with QLoRA is just barely possible.

6–10 GB→ QLoRA or LoRA (small model)RTX 3070/4060 Ti: QLoRA for 7B, LoRA for 1B–3B models.

12–16 GB→ LoRA (fp16/bf16) for 7BRTX 3080/4070: standard LoRA very comfortable. QLoRA for 13B.

24+ GB→ Full FT or LoRARTX 3090/4090: full FT for small models, LoRA for 7B very efficient.

2. Check the amount of data

How many training examples do you have?

< 200 examples→ LoRA rank=4–8, high dropoutTraining fewer parameters means less overfitting risk.

200–2,000 examples→ LoRA rank=8–16Standard setup. Enable early stopping.

> 2,000 examples→ LoRA rank=16–32 or full FT (if VRAM allows)Enough data for more model capacity.

3. Assess the use case

What should the model learn?

Adjust style/tone→ LoRA rank=4–8 is enoughSimple task – little LoRA capacity needed.

Learn a new domain→ LoRA rank=16–32More capacity for new domain knowledge.

Fundamentally retrain→ Full fine-tuningWhen the model needs to change its core behavior.

Multiple tasks, one model→ LoRA adapter per taskBase model once, separate adapter for each task.

Quick-reference table

Scenario	Method	Rank	LR	Epochs
7B LoRA, RTX 3070	QLoRA	16	2e-4	3–5
7B LoRA, RTX 3090	LoRA (bf16)	16	2e-4	3–5
1B full FT, RTX 3080	Full FT (bf16)	—	5e-5	2–3
BERT classification	Full FT or LoRA	8	2e-5	3
Little data (< 300)	QLoRA rank=4	4	1e-4	5–10
Lots of data (> 10K)	LoRA rank=32	32	2e-4	1–3

FrameTrain recommendation for 90% of cases

LoRA (rank=16) + QLoRA when VRAM is tight + early stopping + cosine LR decay + weight_decay=0.01 + warmup=5% of steps. This covers nearly every fine-tuning project.

⚙️ Hyperparameters

📦 Dataset Mastery