blog.gopenai.com Open in urlscan Pro
162.159.152.4  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Open in app

Sign up

Sign in

Write


Sign up

Sign in




CHAT WITH YOUR CSV FILE

Sudarshan Koirala

·

Follow

Published in

GoPenAI

·
4 min read
·
Aug 14, 2023

64

1

Listen

Share

Personal ChatBot 🤖 — Powered by Chainlit, LangChain and OpenAI.

👨🏾‍💻 GitHub ⭐️| 🐦 Twitter | 📹 YouTube | 👔LinkedIn | ☕️Ko-fi


Image by Author


INTRODUCTION

In the era of digital transformation, chatbots have become an integral part of
our daily lives, simplifying tasks and providing quick responses. However, have
you ever thought about a chatbot that can interact with Comma Separated Values
(CSV) file? In this post, I will walk you step by step on how to create a simple
ChatGPT-like UI for Chat with CSV using Chainlit, LangChain and OpenAI.

> OpenAI (Embeddings + LLM)
> LangChain (framework)
> Chainlit (creating apps)


PREREQUISITES

First make sure, you have python 3.11 installed in your system.

Run the following command in terminal to check, python3 --version

If yes, create a virtual environment using the following command.

python3 -m venv .venv && source .venv/bin/activate

If you have python 3.11, then the above command is fine. But, if you have python
version less than 3.11. Using conda is easier. First make sure that you have
conda installed. You can refer to this youtube video, I created to install conda
if you want. Then run the following command.

conda create -n .venv python=3.11 -y && source activate .venv

Once, virtual environment is created, we then install the necessary python
packages with the following command.

pip install langchain, openai, chainlit, pandas, tabulate

The brief introduction of the required python packages ,

 * langchain: A library for building applications with Large Language Models
   (LLMs).
 * openai: The OpenAI API library, used to perform question-answering on the
   csv.
 * chainlit: The Chainlit library, used to build the app's user interface.
 * Pandas: Pandas library which is a powerful Python data analysis toolkit. This
   is needed as we will be using create_pandas_dataframe_agent from LangChain.
 * Next, you need to grab the OpenAI API key from this link and place it in .env
   file. Replace the YOUR_OPENAI_API_KEY with the one you just grabbed from the
   OpenAI website.

OPENAI_API_KEY=YOUR_OPENAI_API_KEY


CREATING THE CHATBOT — HOW ?

Following are steps on how we accomplish such those things:

 * We first load the CSV file.
 * We then use create_pandas_dataframe_agent from Langchain to load the csv file
   and pass LLM model.
 * We then pass the query / question into LLM Model. We will be using OpenAI
   model.
 * The whole process is then wrapped with chainlit for creating a chatbot.


STEP BY STEP CODE EXAMPLE FOR CREATING THE CHATBOT

Import necessary modules and define env variables. In Chainlit, it is not even
necessary to load the env variables in the code. It will automatically fetch
from the .env file if provided.

from langchain.agents import create_pandas_dataframe_agent
from langchain.llms import OpenAI
import pandas as pd
import chainlit as cl
import os
import io

from dotenv import load_dotenv# Load environment variables from .env file (Optional)
load_dotenv()OPENAI_API_KEY= os.getenv("OPENAI_API_KEY")

We then create an OpenAI object.

llm = OpenAI()

Create a agent which takes in data and llm.

def create_agent(data: str, llm):
    """Create a Pandas DataFrame agent."""
    return create_pandas_dataframe_agent(llm, data, verbose=False)

The following steps are wrapped inside a function.

 1. Ask user to upload CSV file and process it.
 2. Once uploaded, pandas library is used to read the csv file.
 3. Create a agent that takes in the data and llm.
 4. After creating the agent, the data is stored in user session to store the
    data. This is all wrapped in a chainlit function as shown below.

@cl.on_chat_start
async def on_chat_start():

    # Sending an image with the local file path
    elements = [
    cl.Image(name="image1", display="inline", path="./robot.jpeg")
    ]
    await cl.Message(content="Hello there, Welcome to AskAnyQuery related to Data!", elements=elements).send()

    files = None

    # Wait for user to upload csv data
    while files is None:
        files = await cl.AskFileMessage(
            content="Please upload a csv file to begin!", 
            accept=["text/csv"],
            max_size_mb= 100,
            timeout = 180,
        ).send()

    # load the csv data and store in user_session
    file = files[0]

    msg = cl.Message(content=f"Processing `{file.name}`...")
    await msg.send()

    # Read csv file with pandas
    csv_file = io.BytesIO(file.content)
    df = pd.read_csv(csv_file, encoding="utf-8")

    # creating user session to store data
    cl.user_session.set('data', df)

    # Send response back to user
    # Let the user know that the system is ready
    msg.content = f"Processing `{file.name}` done. You can now ask questions!"
    await msg.update()

Once, the processing is done, user now asks the question related to the uploaded
csv file and gets the answer.

@cl.on_message
async def main(message: str):

    # Get data
    df = cl.user_session.get('data')

    # Agent creation
    agent = create_agent(df, llm)

    # Run model 
    response = agent.run(message)

    # Send a response back to the user
    await cl.Message(
        content=response,
    ).send()


RUNNING THE CHATBOT:

For running the chatbot, you can save the code in a python file, let’s say
csv_qa.py , then type the following command in the terminal (make sure the
virtual environment is activated). By providing -w , once the file changes, the
UI in the chatbot automatically refreshes.

chainlit run csv_qa.py -w


CONCLUSION

Creating a chatbot that can interact with csv files might seem like a daunting
task, but with Chainlit and LangChain, it becomes a manageable and exciting
project. This technology opens up new possibilities for interacting with tabular
data, making information retrieval more interactive and engaging. Stay tuned for
our next blog post, where we will explore how to create chatbots that can
interact with text files as well as text and pdf files in one go.

The GitHub code covered in this post is here . You can either create the app in
GitHub codespace or in your local computer. Instructions are provided in the
readme file of the repo.

IF you are a video person, I have covered how to chat with csv files using
langchain and chainlit. Having a video recording and blog post side-by-side
might help you understand things better.



👨🏾‍💻 GitHub ⭐️| 🐦 Twitter | 📹 YouTube | 👔LinkedIn | ☕️Ko-fi


RECOMMENDED YOUTUBE PLAYLISTS:

 1. LangChain-Framework-Build-Around-LLMs
 2. VSCode-Git-GitHub

Thank you for your time in reading this post!

Make sure to leave your feedback and comments. See you in the next blog, stay
tuned 📢



Be part of a better internet.
Get 20% off membership for a limited time.


FREE



Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.


Sign up for free


MEMBERSHIP

Get 20% off


Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app


Try for $5 $4/month
ChatGPT
Programming
Large Language Models
Artificial Intelligence
Chat With Csv


64

64

1


Follow



WRITTEN BY SUDARSHAN KOIRALA

201 Followers
·Writer for

GoPenAI

Machine Learning Engineer | Video about AI, Data Science and LLM 👉
https://www.youtube.com/@datasciencebasics

Follow




MORE FROM SUDARSHAN KOIRALA AND GOPENAI

Sudarshan Koirala


OLLAMA + HUGGINGFACE ✅🔥


CREATE CUSTOM MODELS FROM HUGGINGFACE WITH OLLAMA

Feb 25
91
3



Tarun Singh



in

GoPenAI


MASTERING RAG CHUNKING TECHNIQUES FOR ENHANCED DOCUMENT PROCESSING


DIVIDING LARGE DOCUMENTS INTO SMALLER PARTS IS A CRUCIAL YET INTRICATE TASK THAT
SIGNIFICANTLY IMPACTS THE PERFORMANCE OF…

Jun 17
149
2



Paras Madan

in

GoPenAI


BUILDING A MULTI PDF RAG CHATBOT: LANGCHAIN, STREAMLIT WITH CODE


TALKING TO BIG PDF’S IS COOL. YOU CAN CHAT WITH YOUR NOTES, BOOKS AND DOCUMENTS
ETC. THIS BLOG POST WILL HELP YOU BUILD A MULTI RAG…

Jun 6
532
2



Sudarshan Koirala


MAKE YOUR MAC TERMINAL BEAUTIFUL


HOW TO MAKE YOUR BORING MAC TERMINAL AMAZING

Jan 19
55


See all from Sudarshan Koirala
See all from GoPenAI



RECOMMENDED FROM MEDIUM

Ashish Malhotra


CONVERSATIONAL CHATBOT TRAINED ON OWN DATA: STREAMLIT AND LANGCHAIN


WHILE CREATING A CONVERSATIONAL CHATBOT, I STUMBLED UPON UMPTEEN VIDEOS AND
BLOGS. MOST OF THESE BLOGS DESCRIBE THE PROCESS AS DOCUMENT…

Apr 27
25



Eddie Otudor


BUILDING CONVERSATIONAL AI WITH CHAINLIT [CHAT WITH PDF]


CHAT WITH ANY PDF USING ANTHROPIC’S CLAUDE 3 OPUS, LANGCHAIN AND CHAINLIT. IT IS
HIGHLY CUSTOMIZABLE AND WORKS SEAMLESSLY.

Mar 26
2




LISTS


CHATGPT

21 stories·746 saves


CHATGPT PROMPTS

48 stories·1857 saves


AI REGULATION

6 stories·526 saves


WHAT IS CHATGPT?

9 stories·406 saves


Miluska Romero


CREATING CUSTOM CHATBOTS USING CSV DATA WITH PYTHON AND OPENAI API


THIS STEP-BY-STEP GUIDE IS DESIGNED TO HELP YOU CREATE A CHATBOT THAT UTILIZES
YOUR OWN CSV DATA FOR PERSONALIZED INTERACTIONS.

Mar 17
1



Thomas Hansen


CREATE AN AI CHATBOT FROM A CSV FILE


ALMOST ANY DATA CAN BE TURNED INTO A CSV FILE. SOME EXAMPLES CAN BE FOUND BELOW.

Feb 15
56



Rittika Jindal


GEN AI –PART 3: BUILDING A CHATBOT WITH LLAMA AND STREAMLIT: A BEGINNER’S GUIDE


IF YOU’RE NEW TO THE WORLD OF GENERATIVE AI AND LOOKING TO DIVE IN, YOU’VE COME
TO THE RIGHT PLACE. IN THIS SERIES, I’LL TAKE YOU THROUGH…

Apr 12
102



Rehşan Yılmaz

in

GoPenAI


CREATE YOUR CUSTOMIZED CHATBOT WITH YOUR DATA USING LANGCHAIN


HI EVERYONE! BEFORE, I STARTED EXPLORING HOW TO CREATE A CHATBOT AND SHARED MY
FIRST STEPS WITH YOU. NOW, I’M EXCITED TO TALK ABOUT MAKING…

Feb 8
20


See more recommendations

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams