blog.gopenai.com Open in urlscan Pro
162.159.152.4  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Open in app

Sign up

Sign in

Write


Sign up

Sign in



Member-only story


FINETUNING MISTRAL 7B INSTRUCT MODEL IN COLAB: A BEGINNER’S GUIDE

Qendel AI

·

Follow

Published in

GoPenAI

·
8 min read
·
Oct 4, 2023

425

9

Listen

Share


Finetuned Mistral 7B Instruct Model

Finetuning enables you to customize the Mistral 7B Instruct model to walk and
talk just as needed.

Do you have a tricky question-answering, summarization, entity-extraction, or
classification task? Regardless of your mission, finetuning can boost the
Mistral 7B Instruct model’s performance and results.

For tips on improving model responses with added context, check my article on
building a RAG pipeline with Mistral 7B Instruct model:


RAG PIPELINE WITH MISTRAL 7B INSTRUCT MODEL: A STEP-BY-STEP GUIDE


BUILDING RAG PIPELINE WITH MISTRAL 7B INSTRUCT MODEL: A BEGINNER’S GUIDE

medium.com



In this article, we’ll begin by evaluating the model’s performance on a few
examples and will guide you on finetuning the model for your use case.


FINETUNING MISTRAL 7B INSTRUCT MODEL

Step 1

Install and Import Libraries

!pip install git+https://github.com/huggingface/transformers trl accelerate torch bitsandbytes peft datasets


import torch 
from trl import SFTTrainer
from google.colab import drive
from random import randrange 
from datasets import load_dataset 
from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, BitsAndBytesConfig
from peft import AutoPeftModelForCausalLM, LoraConfig, get_peft_model, prepare_model_for_kbit_training

Step 2

Load Databricks’ Dolly 2 Dataset

# Loading the dataset 
dataset = load_dataset("databricks/databricks-dolly-15k", split="train")

# Since I will only finetune on Question-Answer pairs without context, I will filter accordingly
# Filter QA pairs without context
dataset = dataset.filter(lambda x:x['context'] == '')

# A prompting formatting function 
def create_prompt_instruction(sample):
   return f"""### Instruction: 
   Use the input below to create an instruction, which could have been used to generate the input using an LLM. 

   ### Input 
   {sample['response']}

   ### Response:
   {sample['instruction']}
   """

CREATE AN ACCOUNT TO READ THE FULL STORY.

The author made this story available to Medium members only.
If you’re new to Medium, create a new account to read this story on us.



Continue in app
Or, continue in mobile web



Sign up with Google

Sign up with Facebook

Sign up with email

Already have an account? Sign in





425

425

9


Follow



WRITTEN BY QENDEL AI

381 Followers
·Writer for

GoPenAI

Prompting | LLMs | AI Agents | Chatbots | NLP. Email: qendelai@gmail.com |
LinkedIn: https://www.linkedin.com/company/qendelai/ | Twitter: Qendel_AI

Follow




MORE FROM QENDEL AI AND GOPENAI

Qendel AI

in

GoPenAI


RAG PIPELINE WITH MISTRAL 7B INSTRUCT MODEL: A STEP-BY-STEP GUIDE


BUILDING RAG PIPELINE WITH MISTRAL 7B INSTRUCT MODEL: A BEGINNER’S GUIDE


·8 min read·Oct 4, 2023

385

4




Lucas Scott

in

GoPenAI


PYTHON IS OUT OF FAVOR?HUGGING FACE OPEN-SOURCES A NEW ML FRAMEWORK WHICH
WRITTEN IN RUST


HUGGING FACE HAS QUIETLY OPEN SOURCED AN ML FRAMEWORK — CANDLE


·5 min read·Sep 26, 2023

566

6




Sanjay Singh

in

GoPenAI


A STEP-BY-STEP GUIDE TO TRAINING YOUR OWN LARGE LANGUAGE MODELS (LLMS).


LARGE LANGUAGE MODELS (LLMS) HAVE TRULY REVOLUTIONIZED THE REALM OF ARTIFICIAL
INTELLIGENCE (AI). THESE POWERFUL AI SYSTEMS, SUCH AS GPT-3…

10 min read·Sep 30, 2023

144

1




Qendel AI

in

GoPenAI


OVER WITH LLAMA-2, MISTRAL 7B IS TAKING OVER: GET STARTED WITH MISTRAL 7B
INSTRUCT


GETTING STARTED WITH MISTRAL 7B AND LANGCHAIN INTEGRATION: A STEP-BY-STEP GUIDE.


·7 min read·Sep 30, 2023

457

5



See all from Qendel AI
See all from GoPenAI



RECOMMENDED FROM MEDIUM

Adithya S K


A BEGINNER’S GUIDE TO FINE-TUNING MISTRAL 7B INSTRUCT MODEL


FINE-TUNING FOR CODE GENERATION USING A SINGLE GOOGLE COLAB NOTEBOOK

8 min read·Oct 6, 2023

367

15




Gathnex


MISTRAL-7B FINE-TUNING: A STEP-BY-STEP GUIDE


INTRODUCING MISTRAL 7B: THE POWERHOUSE OF LANGUAGE MODELS

5 min read·Oct 4, 2023

353

12





LISTS


PREDICTIVE MODELING W/ PYTHON

20 stories·846 saves


NATURAL LANGUAGE PROCESSING

1133 stories·608 saves


PRACTICAL GUIDES TO MACHINE LEARNING

10 stories·989 saves


THE NEW CHATBOTS: CHATGPT, BARD, AND BEYOND

12 stories·286 saves


Qendel AI

in

GoPenAI


OVER WITH LLAMA-2, MISTRAL 7B IS TAKING OVER: GET STARTED WITH MISTRAL 7B
INSTRUCT


GETTING STARTED WITH MISTRAL 7B AND LANGCHAIN INTEGRATION: A STEP-BY-STEP GUIDE.


·7 min read·Sep 30, 2023

457

5




Yanli Liu

in

Level Up Coding


A STEP-BY-STEP GUIDE TO RUNING MISTRAL-7B AI ON A SINGLE GPU WITH GOOGLE COLAB


HOW TO RUN YOUR AI EFFICIENTLY THROUGH 4-BIT QUANTIZATION (WITH COLAB NOTEBOOK
INCLUDED!) .


·6 min read·Oct 8, 2023

1.1K

9




Jatin Tyagi

in

Python in Plain English


FINE-TUNING MISTRAL 7B MODEL WITH YOUR CUSTOM DATA


FINE-TUNING A LANGUAGE MODEL CAN BE A COMPLEX TASK, BUT WITH THE RIGHT DATASET,
HARDWARE, AND TRAINING SCRIPT, YOU CAN ACHIEVE IMPRESSIVE…

6 min read·Oct 17, 2023

107

2




Thomas Le Moullec


GETTING STARTED WITH MISTRAL-7B MODEL ON GCP VERTEXAI


DEPLOYING MISTRAL AI FIRST OPEN SOURCE MODEL ON GCP

11 min read·Oct 10, 2023

73

1



See more recommendations

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams