Models

All the models are stored in The Kaitchup’s repository on the Hugging Face Hub.

Llama 3

Llama 3 8B quantized to 4-bit with AWQ:

• kaitchup/Llama-3-8b-awq-4bit

Fine-tune Llama 3 on Your Computer

Llama 3 8B converted to an embedding model:

• kaitchup/Llama-3-8B-llm2vec-Emb

The Mayonnaise

A collection of 4 models, made with mergekit, and performing among the best 7B models on public benchmarks.

• The Mayonnaise Collection

The recipe used to create these models is detailed here:

The Mayonnaise: Rank First on the Open LLM Leaderboard with TIES-Merging

Maixtchup: A MoE Made with 4xMistral 7B

• kaitchup/Maixtchup-4x7b

Maixtchup: Make Your Own Mixture of Experts with Mergekit

Llama 2 MT

I fine-tuned Llama 2 for the translation of several languages into English. More details in this article:

Llama 2 MT: Turn Llama 2 into a Translation System with QLoRA

You can find the translation models here:

• Llama 2 7B MT

Llama 2 Quantized for QA-LoRA

QA-LoRA fine-tunes LoRA to be “quantization-aware”. The current implementation needs LLMs to be quantized with a specific version of AutoGPTQ. More details in this article:

QA-LoRA: Quantization-Aware Fine-tuning for Large Language Models

A Llama 2 model ready for QA-LoRA is available here:

• kaitchup/Llama-2-7b-4bit-32g-autogptq

SFT, reward, and RLHF models based on OPT and trained with DeepSpeed Chat

The instructions to train and use these models are given in this article:

Train Instruct LLMs On Your GPU with DeepSpeed Chat — Step #1: Supervised Fine-tuning

• kaitchup/OPT-1.3B-SFT-DSChatLoRA: SFT model based on OPT-1.3B and trained with LoRA

• kaitchup/OPT-350M-RM-DSChat: Reward model based on OPT-350M

• kaitchup/OPT-1.3B-RLHF-DSChatLoRA: RLHF model based on OPT-1.3B trained with LoRA

Llama 2 7B quantized with GPTQ (compatible with transformers)

To quantize and run GTPQ models, follow the instructions in this tutorial:

Quantize and Fine-tune LLMs with GPTQ Using Transformers and TRL

• kaitchup/Llama-2-7b-gptq-4bit: Llama 2 7B quantized to 4-bit

• kaitchup/Llama-2-7b-gptq-3bit: Llama 2 7B quantized to 3-bit

• kaitchup/Llama-2-7b-gptq-2bit: Llama 2 7B quantized to 2-bit