www.relari.ai Open in urlscan Pro
34.234.52.18  Public Scan

Form analysis
1 forms found in the DOM

Name: wf-form-Form-newsletter — GET

<form id="wf-form-Form-newsletter" name="wf-form-Form-newsletter" data-name="Form newsletter" method="get" class="form_sub" data-wf-page-id="669f7329c898141d69e16697" data-wf-element-id="99882c17-03fd-302a-0326-b184356ccbdb"
  aria-label="Form newsletter"><input class="text_field w-input" maxlength="256" name="Email" data-name="Email" placeholder="Enter your email" type="email" id="email" required=""><input type="submit" data-wait="Please wait..."
    class="bn_form_sub w-button" value="Subscribe"></form>

Text Content

Features
Pricing
Blog
About Us
Docs

Github

Get a Demo
Sign up




SAY GOODBYE

TO GUESSWORK

with a data-driven approach
to AI development

The only data-driven toolkit to evaluate and improve your LLM application

Get Started for free
View Docs
View tutorial
pip install relari


SYSTEMATICALLY IMPROVE YOUR


Chatbo


GO FROM PROTOTYPE TO PRODUCTION FASTER

Move quickly and with confidence. Make your complex system more robust and
reliable with custom, high-quality data.

Auto Prompt
Optimizer
Custom
Evaluators
Synthetic
Golden Dataset
Systematic
Fine-tuning
Runtime
Monitor


DATA-DRIVEN DEVELOPMENT

Define

Set the expected system behavior using our custom synthetic dataset tool and/or
human annotations. Define your metrics to measure what matters.

Measure

Understand your app’s performance in every scenario - based on the standards
most important to your users.

Improve

Improve your LLM app with intention - with automated evaluation and performance
optimization






TRUSTED BY AI PIONEERS

Noam Rubin
AI Engineer at

Security Compliance AI

Before we had Relari, we relied on guesswork and instincts to select key
parameters such as similarity threshold, chunk size, embedding models, and
retrieval strategies. Using Relari’s synthetic golden datasets and tailored
evaluation metrics, we were able to easily understand trade-offs among different
parameters over large datasets, and make confident, informed decisions. This
data-driven process significantly improved our iteration speed, allowing us to
reach production-grade for multiple LLM products over a short period of time.

Read the case study
Jiang Chen
Head of Ecosystem and AI Platform


Baseline LLM-as-a-judge is expensive and unstable. In a comprehensive RAG eval
run, e spent $1,000+ bill on GPT4 tokens. It's also a challenge to collect
domain-specific datasets. Relari's synthetic dataset generation and
deterministic evaluation make it easier to develop high-quality LLM experience.

Enterprise RAG
Yuhong Sun
Co-founder


Relari's custom generated synthetic dataset is the best real world
representation we've seen! We use the data to stress test our enterprise search
engine and guide key product decisions.

Enterprise Search
Mike Sands
Senior Director of Product


Relari has helped immensely by building a set of metrics and standards that we
can use to quickly and automatically evaluate changes in our LLM pipeline.

Compliance AI
Tina Ding
Engineering Manager, AI and Enterprise Products


Generative AI is critical to Vanta’s roadmap across multiple products. Relari
plays an instrumental role in our LLM product lifecycle, helping us
systematically improve AI performance through rapid experimentation with custom
synthetic datasets and high-quality metrics.

Security Compliance AI
Nick Bradford
CTO


We iterate much faster on our coding agents thanks to the granular metrics
Relari offers! Through high-quality synthetic datasets, we can benchmark and
validate our agent performance with ease.

Coding Agent
Jiang Chen
Head of Ecosystem and AI Platform


Baseline LLM-as-a-judge is expensive and unstable. In a comprehensive RAG eval
run, e spent $1,000+ bill on GPT4 tokens. It's also a challenge to collect
domain-specific datasets. Relari's synthetic dataset generation and
deterministic evaluation make it easier to develop high-quality LLM experience.

Enterprise RAG
Yuhong Sun
Co-founder


Relari's custom generated synthetic dataset is the best real world
representation we've seen! We use the data to stress test our enterprise search
engine and guide key product decisions.

Enterprise Search
Mike Sands
Senior Director of Product


Relari has helped immensely by building a set of metrics and standards that we
can use to quickly and automatically evaluate changes in our LLM pipeline.

Compliance AI
Tina Ding
Engineering Manager, AI and Enterprise Products


Generative AI is critical to Vanta’s roadmap across multiple products. Relari
plays an instrumental role in our LLM product lifecycle, helping us
systematically improve AI performance through rapid experimentation with custom
synthetic datasets and high-quality metrics.

Security Compliance AI
Nick Bradford
CTO


We iterate much faster on our coding agents thanks to the granular metrics
Relari offers! Through high-quality synthetic datasets, we can benchmark and
validate our agent performance with ease.

Coding Agent
Jiang Chen
Head of Ecosystem and AI Platform


Baseline LLM-as-a-judge is expensive and unstable. In a comprehensive RAG eval
run, e spent $1,000+ bill on GPT4 tokens. It's also a challenge to collect
domain-specific datasets. Relari's synthetic dataset generation and
deterministic evaluation make it easier to develop high-quality LLM experience.

Enterprise RAG
Yuhong Sun
Co-founder


Relari's custom generated synthetic dataset is the best real world
representation we've seen! We use the data to stress test our enterprise search
engine and guide key product decisions.

Enterprise Search
Mike Sands
Senior Director of Product


Relari has helped immensely by building a set of metrics and standards that we
can use to quickly and automatically evaluate changes in our LLM pipeline.

Compliance AI
Tina Ding
Engineering Manager, AI and Enterprise Products


Generative AI is critical to Vanta’s roadmap across multiple products. Relari
plays an instrumental role in our LLM product lifecycle, helping us
systematically improve AI performance through rapid experimentation with custom
synthetic datasets and high-quality metrics.

Security Compliance AI
Nick Bradford
CTO


We iterate much faster on our coding agents thanks to the granular metrics
Relari offers! Through high-quality synthetic datasets, we can benchmark and
validate our agent performance with ease.

Coding Agent



LEARN MORE ABOUT LLM DEVELOPMENT ON OUR BLOG


VANTA AI CASE STUDY

July 24, 2024
Case study


GENERATE SYNTHETIC DATA TO TEST LLM APPLICATIONS

May 7, 2024
Technical guide


MAKE THE MOST OUT OF LLM PRODUCTION DATA: SIMULATED USER FEEDBACK

April 10, 2024
Technical guide

Read more


PRICING

How to get started
Community


Ideal for individual developers, researchers interested in the running
open-source metrics locally.

Free
Get Started
30+ open-source metrics
Open-source evaluation framework
Community support (discord, github)
Starter


Ideal for individual developers who want to leverage the entire suite of
data-driven toolkits via Cloud API & UI.

Free
Get Started
1 seat
1 synthetic dataset
1,000 usage credits / month*
Standard evaluation metrics
Auto prompt optimizer
Community Support
Team


Best for AI teams who want to deploy reliable LLM applications at scale with
dedicated support from Relari.

$1,000 / mo
Schedule a Demo
Up to 5 seats
Unlimited synthetic datasets
10,000 usage credits / month*
Standard evaluation metrics
Custom evaluation metrics
Auto prompt optimizer
Dedicated support (joint slack channel)
Enterprise


Best for AI teams who want to deploy reliable LLM applications at scale with
enterprise feature requirements.

Custom pricing
Schedule a Demo
Custom number of seats
Unlimited synthetic datasets
Custom usage credits / month*
Standard evaluation metrics
Custom evaluation metrics
Auto prompt optimizer
CI/CD integration
Virtual private cloud deployment
On-prem deployment
Dedicated support with SLA
*credits can be used towards dataset generation, evaluation runs, and prompt
optimization runs



GOT A QUESTION?

Why data-driven development?


Data is our secret weapon to make your application stand out. Leveraging data in
your AI development journey from early experimentation to post-launch can help
you make confident decisions and supercharge your user satisfaction.

What's a golden dataset?


LLM systems are inherently non-deterministic, making it difficult to diagnose
issues and track performance. Golden datasets provide a reliable benchmark for
evaluation, enabling you to consistently measure and improve your system's
performance.

Can I use my own datasets with Relari?


Absolutely! You can use your own labeled datasets with inputs and expected
outputs for evaluation, prompt optimization, and fine-tuning features within
Relari. Additionally, you can augment your datasets with new synthetic data.

What do I need to provide to generate synthetic datasets?


You simply need to specify your application architecture (such as RAG, agents,
etc.) and provide seed examples for your use cases (sample inputs and outputs).
To enhance the dataset quality, you can also supply sample data, like examples
of documents your AI accesses.

How do I create custom evaluation metrics?


With our Team or Enterprise plan, you can train custom metrics based on tailored
rubrics and/or user feedback.

How does auto prompt optimizer work?


Relari's prompt optimizer simplifies the tedious prompt tweaking process. It
automatically improves prompts to perform better on custom metrics using a
golden dataset, which can be customer-provided or synthetically generated.

My application isn’t written in Python or TypeScript. Will Relari be helpful?


Yes, definitely. We offer an API that allows you to interact programmatically
with all of Relari's features, including dataset generation, evaluation, and
prompt optimization. Please refer to our documentation for more details.

I can’t have data leave my environment. Can I self-host Relari?


Yes, on our enterprise plan, we offer the option for customers to self-host
Relari, ensuring your data remains within your environment.


STOP THE GUESSWORK AND SHIP FASTER!

Get started with Relari’s data-driven development platform to supercharge your
LLM product

Get Started for free
View Docs



SIGN UP FOR OUR NEWSLETTER

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Features
Pricing
Blog
About us
Docs

Github


© 2024 Relari, inc. All rights reserved.


TermsPrivacy PolicySecurity