The Kaitchup – AI on a Budget

The Kaitchup – AI on a Budget

Share this post

The Kaitchup – AI on a Budget
The Kaitchup – AI on a Budget
LoRA at Scale on a Consumer GPU: Does It Work?
Copy link
Facebook
Email
Notes
More

LoRA at Scale on a Consumer GPU: Does It Work?

Reproducing TULU 3 SFT on Consumer Hardware Using LoRA and Unsloth

Benjamin Marie's avatar
Benjamin Marie
May 12, 2025
∙ Paid
6

Share this post

The Kaitchup – AI on a Budget
The Kaitchup – AI on a Budget
LoRA at Scale on a Consumer GPU: Does It Work?
Copy link
Facebook
Email
Notes
More
3
Share
Image generated with ChatGPT

LoRA is well known for drastically cutting the cost of supervised fine-tuning (SFT), and many tutorials demonstrate how to get started. However, most of these focus on narrow tasks, small datasets, or lightweight demos. What they don’t address is the more important question for real-world use cases: Can LoRA match the performance of full fine-tuning on a large-scale dataset, while costing 10 times less?

The Kaitchup – AI on a Budget is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

That’s what we’ll explore in this article. And spoiler: the answer is (almost) yes. With LoRA and tools like Unsloth, it’s possible to replicate TULU 3’s state-of-the-art SFT recipe using just a single 24 GB GPU (e.g., an RTX 4090), while the original full fine-tuning setup from AI2 required multiple GPU nodes and several hours of compute. We’ll walk through how to reproduce their results, yielding a high-quality Llama 3.1 chat model, using a far more accessible setup.

This is just the beginning: in a follow-up article, we’ll also test whether the same approach transfers well to other models, like Qwen3, or if the current recipe is uniquely tuned to Llama 3.1.

My SFT recipe using LoRA, Unsloth, and a single 24 GB GPU, can be tried with this notebook:

Get the notebook (#164)

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 The Kaitchup
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More