Launching AI Toolboxes

First release: Qwen2.5 Toolbox - Llama 3.1/3.2 and open NotebookLM toolboxes will follow

The Kaitchup now proposes specialized AI toolboxes.

Releases for November/December 2024

• Qwen2.5 Toolbox: 14 notebooks specially optimized to run, fine-tune, quantize, serve, and merge Qwen2.5 models. Already available (see below).

• Llama 3.1/3.2 Toolbox: Notebooks specially optimized to run, fine-tune, quantize, serve, and merge Llama 3.1/3.2 models. Scheduled for release by the end of November.

• NotebookLM Toolbox: Scheduled for release by mid-December, this toolbox will implement NotebookLM with state-of-the-art open LLMs and TTS.

They are regularly updated to ensure compatibility with the latest versions of frameworks like vLLM, Transformers, and TRL. I test the code once a week. Most of the code in these toolboxes requires a GPU that supports FlashAttention and bfloat16—such as those from the Ampere generation (RTX 3000 series, A series) or newer (RTX 4000 series, Ada series, H series).

How to Access The Kaitchup’s AI Toolboxes

You have two options:

1. Subscribers to The Kaitchup Pro:

   • Lifetime Access: Pro subscribers have lifelong access to all current and future toolboxes as part of their subscription. If you are a Pro subscriber, don’t purchase individual access to the toolboxes, it’s included in your subscription!

   • Access Tokens: Each Pro subscriber receives a Hugging Face access token to download the repositories. You can see your access token on this page.

   • Hugging Face User ID: In addition, Pro subscribers can optionally request access directly on the repository's main page. I check the access requests daily.

If you plan to purchase several toolboxes or if you are interested in the other perks proposed by The Kaitchup Pro, this is your best option.

1. Individual Toolbox Purchases for Non-Pro Subscribers:

   • Lifetime Access to the Purchased Toolbox

   • Purchasing: Individual toolboxes can be bought directly through Gumroad for those without a Pro subscription. There is a 30-day refund guarantee if you are not satisfied.

   • Repository Access: After purchase, all the repository content is accessible directly from Gumroad. Then, you can also request access to the repository on Hugging Face.

Once you gain access, whether through a Pro subscription or individual purchase, you will be able to open issues for discussion or improvement, fill bug reports, and suggest new notebooks, directly in the Hugging Face repository.

First Toolbox Available: Qwen2.5 Toolbox

This toolbox already includes 14 Jupyter notebooks specially optimized for Qwen2.5. The logs of successful runs are also provided. More notebooks will be regularly added.

Once you've subscribed to The Kaitchup Pro or purchased access, request repository access here.

CUDA 12.4 and PyTorch 2.4 are recommended for running the code in the toolbox. PyTorch 2.5 is not supported yet (waiting for FlashAttention updates). Note: Precisely, I use this pod template from RunPod (referral link): runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04.

Toolbox content

• Supervised Fine-Tuning with Chat Templates (5 notebooks)

  • Full fine-tuning

  • LoRA fine-tuning

  • QLoRA fine-tuning with Bitsandbytes quantization

  • QLoRA fine-tuning with AutoRound quantization

  • LoRA and QLoRA fine-tuning with Unsloth

• Preference Optimization (2 notebooks)

  • DPO training with LoRA (TRL and Transformers)

  • ORPO training (not released yet)

• Quantization (3 notebooks)

  • AWQ

  • AutoRound

  • GGUF for llama.cpp

• Inference with Qwen2.5 Instruct and Your Own Fine-tuned Qwen2.5 (4 notebooks)

  • Transformers with and without a LoRA adapter

  • vLLM offline and online inference

  • Ollama (not released yet)

  • llama.cpp

• Merging (3 notebooks)

  • Merge a LoRA adapter into the base model

  • Merge a QLoRA adapter into the base model

  • Merge several Qwen2.5 models into one with mergekit (not released yet)