venturebeat.com Open in urlscan Pro
192.0.66.2  Public Scan

Form analysis
1 forms found in the DOM

GET https://venturebeat.com/

<form method="get" action="https://venturebeat.com/" class="search-form" id="nav-search-form">
  <input id="mobile-search-input" class="" type="text" placeholder="Search" name="s" aria-label="Search" required="">
  <button type="submit" class="">
    <svg width="24" height="24" viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg">
      <g>
        <path fill-rule="evenodd" clip-rule="evenodd"
          d="M14.965 14.255H15.755L20.745 19.255L19.255 20.745L14.255 15.755V14.965L13.985 14.685C12.845 15.665 11.365 16.255 9.755 16.255C6.16504 16.255 3.255 13.345 3.255 9.755C3.255 6.16501 6.16504 3.255 9.755 3.255C13.345 3.255 16.255 6.16501 16.255 9.755C16.255 11.365 15.665 12.845 14.6851 13.985L14.965 14.255ZM5.255 9.755C5.255 12.245 7.26501 14.255 9.755 14.255C12.245 14.255 14.255 12.245 14.255 9.755C14.255 7.26501 12.245 5.255 9.755 5.255C7.26501 5.255 5.255 7.26501 5.255 9.755Z">
        </path>
      </g>
    </svg>
  </button>
</form>

Text Content

WE VALUE YOUR PRIVACY

We and our partners store and/or access information on a device, such as cookies
and process personal data, such as unique identifiers and standard information
sent by a device for personalised ads and content, ad and content measurement,
and audience insights, as well as to develop and improve products. With your
permission we and our partners may use precise geolocation data and
identification through device scanning. You may click to consent to our and our
760 partners’ processing as described above. Alternatively you may access more
detailed information and change your preferences before consenting or to refuse
consenting. Please note that some processing of your personal data may not
require your consent, but you have a right to object to such processing. Your
preferences will apply to this website only. You can change your preferences at
any time by returning to this site or visit our privacy policy.
MORE OPTIONSAGREE

Skip to main content
Events Video Special Issues Jobs
VentureBeat Homepage

Subscribe

 * Artificial Intelligence
   * View All
   * AI, ML and Deep Learning
   * Auto ML
   * Data Labelling
   * Synthetic Data
   * Conversational AI
   * NLP
   * Text-to-Speech
 * Security
   * View All
   * Data Security and Privacy
   * Network Security and Privacy
   * Software Security
   * Computer Hardware Security
   * Cloud and Data Storage Security
 * Data Infrastructure
   * View All
   * Data Science
   * Data Management
   * Data Storage and Cloud
   * Big Data and Analytics
   * Data Networks
 * Automation
   * View All
   * Industrial Automation
   * Business Process Automation
   * Development Automation
   * Robotic Process Automation
   * Test Automation
 * Enterprise Analytics
   * View All
   * Business Intelligence
   * Disaster Recovery Business Continuity
   * Statistical Analysis
   * Predictive Analysis
 * More
   * Data Decision Makers
   * Virtual Communication
     * Team Collaboration
     * UCaaS
     * Virtual Reality Collaboration
     * Virtual Employee Experience
   * Programming & Development
     * Product Development
     * Application Development
     * Test Management
     * Development Languages


Subscribe Events Video Special Issues Jobs



MISTRAL CEO CONFIRMS ‘LEAK’ OF NEW OPEN SOURCE AI MODEL NEARING GPT-4
PERFORMANCE

Carl Franzen@carlfranzen
January 31, 2024 10:44 AM
 * Share on Facebook
 * Share on X
 * Share on LinkedIn

Credit: VentureBeat made with Midjourney V6

The past few days have been a wild ride for the growing open source AI community
— even by its fast-moving and freewheeling standards.

Here’s the quick chronology: on or about January 28, a user with the handle
“Miqu Dev” posted a set of files on HuggingFace, the leading open-source AI
model and code-sharing platform, that together comprised a seemingly new
open-source large language model (LLM) labeled “miqu-1-70b.”

1
/
4
Moving responsible AI forward as fast as AI
Read More

741.1K
3.1K
2



Video Player is loading.
Play Video
Unmute

Duration 0:00
/
Current Time 0:00
Playback Speed Settings
1x
Loaded: 0%

0:00

Remaining Time -0:00
 
FullscreenPlayRewind 10 SecondsUp Next

This is a modal window.



Beginning of dialog window. Escape will cancel and close the window.

TextColorWhiteBlackRedGreenBlueYellowMagentaCyanTransparencyOpaqueSemi-TransparentBackgroundColorBlackWhiteRedGreenBlueYellowMagentaCyanTransparencyOpaqueSemi-TransparentTransparentWindowColorBlackWhiteRedGreenBlueYellowMagentaCyanTransparencyTransparentSemi-TransparentOpaque
Font Size50%75%100%125%150%175%200%300%400%Text Edge
StyleNoneRaisedDepressedUniformDropshadowFont FamilyProportional
Sans-SerifMonospace Sans-SerifProportional SerifMonospace SerifCasualScriptSmall
Caps
Reset restore all settings to the default valuesDone
Close Modal Dialog

End of dialog window.

Share
Playback Speed

0.25x
0.5x
1x Normal
1.5x
2x
Replay the list

TOP ARTICLES






 * Powered by AnyClip
 * Privacy Policy




Moving responsible AI forward as fast as AI


The HuggingFace entry, which is still up at the time of this article’s posting,
noted that the new LLM’s “Prompt format,” how users interact with it, was the
same as Mistral, the well-funded open source Parisian AI company behind Mixtral
8x7b, viewed by many to be the top performing open source LLM presently
available, a fine-tuned and retrained version of Meta’s Llama 2.


POSTED ON 4CHAN

The same day, an anonymous user on 4chan (possibly “Miqu Dev”) posted a link to
the miqu-1-70b files on 4chan, the notoriously longstanding haven of online
memes and toxicity, where users began to notice it.


VB EVENT

The AI Impact Tour – NYC

We’ll be in New York on February 29 in partnership with Microsoft to discuss how
to balance risks and rewards of AI applications. Request an invite to the
exclusive event below.

 


Request an invite

Some took to X, Elon Musk’s social network formerly known as Twitter, to share
the discovery of the model and what appeared to be its exceptionally high
performance at common LLM tasks (measured by tests known as benchmarks),
approaching the previous leader, OpenAI’s GPT-4 on the EQ-Bench.





MISTRAL QUANTIZED?

Machine learning (ML) researchers took notice on LinkedIn, as well.

advertisement


“Does ‘miqu’ stand for MIstral QUantized? We don’t know for sure, but this
quickly became one of, if not the best open-source LLM,” wrote Maxime Labonne,
an ML scientist at JP Morgan & Chase, one of the world’s largest banking and
financial companies. “Thanks to @152334H, we also now have a good unquantized
version of miqu here: https://lnkd.in/g8XzhGSM

The investigation continues. Meanwhile, we might see fine-tuned versions of miqu
outperforming GPT-4 pretty soon.“

Quantization in ML refers to a technique used to make it possible to run certain
AI models on less powerful computers and chips by replacing specific long
numeric sequences in a model’s architecture with shorter ones.

Users speculated “Miqu” might be a new Mistral model being covertly “leaked” by
the company itself into the world — especially since Mistral is known for
dropping new models and updates without fanfare through esoteric and technical
means — or perhaps an employee or customer gone rouge.


CONFIRMATION FROM THE TOP

Well, today it appears we finally have confirmation of the latter of those
possibilities: Mistral co-founder and CEO Arthur Mensch took to X to clarify:
“An over-enthusiastic employee of one of our early access customers leaked a
quantised (and watermarked) version of an old model we trained and distributed
quite openly…

To quickly start working with a few selected customers, we retrained this model
from Llama 2 the minute we got access to our entire cluster — the pretraining
finished on the day of Mistral 7B release. We’ve made good progress since — stay
tuned!“

advertisement



Hilariously, Mensch also appears to have taken to the illicit HuggingFace post
not to demand a takedown, but to leave a comment that the poster “might consider
attribution.”



Still, with Mensch’s note to “stay tuned!” it appears that not only is Mistral
training a version of this so-called “Miqu” model that approaches GPT-4 level
performance, but it may, in fact, match or exceed it, if his comments are to be
interpreted generously.


A PIVOTAL MOMENT IN OPEN SOURCE AI AND BEYOND?

advertisement


That would be a watershed moment not just for open-source generative AI but the
entire field of AI and computer science: since its release back in March 2023,
GPT-4 has remained the most powerful and highest-performing LLM in the world by
most benchmarks. Not even any of Google’s presently available, long-rumored
Gemini models have been able to eclipse it — yet (according to some measures,
the current Gemini models are actually worse than the older OpenAI GPT-3.5
model).

The release of an open source GPT-4 class model, which would presumably be
functionally free to use, would likely place enormous competitive pressure on
OpenAI and its subscription tiers, especially as more enterprises look to open
source models or a mixture of open source and closed source, to power their
applications, as VentureBeat’s founder and CEO Matt Marshall recently reported.
OpenAI may retain the edge with its faster GPT-4 Turbo and GPT-4V (vision), but
the writing on the wall is pretty clear: the open source AI community is
catching up fast. Will OpenAI have enough of a head start, and a metaphorical
“moat” with its GPT Store and other features, to remain in the top spot for
LLMs?

VentureBeat's mission is to be a digital town square for technical
decision-makers to gain knowledge about transformative enterprise technology and
transact. Discover our Briefings.




THE AI IMPACT TOUR NYC

Join us in New York for an invitation-only evening of networking and insights at
our exclusive event: "How to balance risks and rewards of AI applications."

Request an Invite


 * VentureBeat Homepage
 * Follow us on Facebook
 * Follow us on X
 * Follow us on LinkedIn
 * Follow us on RSS

 * Press Releases
 * Contact Us
 * Advertise
 * Share a News Tip
 * Contribute to DataDecisionMakers

 * Privacy Policy
 * Terms of Service
 * Do Not Sell My Personal Information

© 2024 VentureBeat. All rights reserved.