www.confluent.io
Open in
urlscan Pro
2a05:d014:275:cb02:295:4d7d:ac7b:7207
Public Scan
URL:
https://www.confluent.io/what-is-apache-kafka/
Submission: On January 25 via api from ZA — Scanned from DE
Submission: On January 25 via api from ZA — Scanned from DE
Form analysis 1 forms found in the DOM
<form role="search"><svg class="cfHeaderNav-style-module--searchBarIcon--ZSA0y" aria-hidden="true" width="18px" height="18px" focusable="false" data-prefix="fas" data-icon="search" role="img" viewBox="0 0 512 512">
<path fill="currentColor"
d="M505 442.7L405.3 343c-4.5-4.5-10.6-7-17-7H372c27.6-35.3 44-79.7 44-128C416 93.1 322.9 0 208 0S0 93.1 0 208s93.1 208 208 208c48.3 0 92.7-16.4 128-44v16.3c0 6.4 2.5 12.5 7 17l99.7 99.7c9.4 9.4 24.6 9.4 33.9 0l28.3-28.3c9.4-9.4 9.4-24.6.1-34zM208 336c-70.7 0-128-57.2-128-128 0-70.7 57.2-128 128-128 70.7 0 128 57.2 128 128 0 70.7-57.2 128-128 128z">
</path>
</svg><input type="search" placeholder="Search" autocomplete="off" autocorrect="off" autocapitalize="off" name="s" value="" class="style-module--input--fZLFL st-default-search-input" id="nav-search-input"><svg
class="cfHeaderNav-style-module--inputBarResetIcon--JFpIe" aria-hidden="true" width="18px" height="18px" focusable="false" data-prefix="fas" data-icon="times" role="img" viewBox="0 0 352 512">
<path fill="currentColor"
d="M242.72 256l100.07-100.07c12.28-12.28 12.28-32.19 0-44.48l-22.24-22.24c-12.28-12.28-32.19-12.28-44.48 0L176 189.28 75.93 89.21c-12.28-12.28-32.19-12.28-44.48 0L9.21 111.45c-12.28 12.28-12.28 32.19 0 44.48L109.28 256 9.21 356.07c-12.28 12.28-12.28 32.19 0 44.48l22.24 22.24c12.28 12.28 32.2 12.28 44.48 0L176 322.72l100.07 100.07c12.28 12.28 32.2 12.28 44.48 0l22.24-22.24c12.28-12.28 12.28-32.19 0-44.48L242.72 256z">
</path>
</svg></form>Text Content
Kafka In the Cloud: Why It’s 10x Better With Confluent | Get free eBook
Login Contact Us
* Why Confluent
*
Confluent vs. OSS Kafka®
Streaming Data Pipelines
Our Customers
Our Partners
* Products
* Choose Your deployment
Confluent Cloud
* Pricing
* Login
Software: Confluent Platform
* Subscription
*
Stream Designer
Connectors
ksqlDB
Stream Governance
Confluent vs. Kafka: Why you need Confluent
Confluent is 10x Kafka
Streaming Data Pipelines
* Solutions
*
By Customer
By Industry
By Use Case
By Architecture
All Solutions
*
Hybrid and Multicloud Modernization
Event-driven Microservices
Use Case Showcase
Streaming Use Cases to transform your business
* Learn
*
Blog
Resources
Training
Professional Services
*
Careers
Events
* Meetups
* Kafka Summit
* Current: Data Streaming Event
* Webinars
Streaming ETL cloud demo
Mastering Kafka Streams and ksqlDB
Microservices with Confluent
* Developers
*
Confluent Developer
Docs
* Apache Kafka Quick Start
Streaming Audio Podcast
Ask the Community
* Watch DemoGet Started Free
*
* US English
Watch DemoGet Started Free
* Watch DemoGet Started Free
*
* Why Confluent
*
Confluent vs. OSS Kafka®
Streaming Data Pipelines
Our Customers
Our Partners
* Products
* Choose Your deployment
Confluent Cloud
* Pricing
* Login
Software: Confluent Platform
* Subscription
*
Stream Designer
Connectors
ksqlDB
Stream Governance
Confluent vs. Kafka: Why you need Confluent
Confluent is 10x Kafka
Streaming Data Pipelines
* Solutions
*
By Customer
By Industry
By Use Case
By Architecture
All Solutions
*
Hybrid and Multicloud Modernization
Event-driven Microservices
Use Case Showcase
Streaming Use Cases to transform your business
* Learn
*
Blog
Resources
Training
Professional Services
*
Careers
Events
* Meetups
* Kafka Summit
* Current: Data Streaming Event
* Webinars
Streaming ETL cloud demo
Mastering Kafka Streams and ksqlDB
Microservices with Confluent
* Developers
*
Confluent Developer
Docs
* Apache Kafka Quick Start
Streaming Audio Podcast
Ask the Community
*
WHAT IS KAFKA?
Apache Kafka is an open-source distributed streaming system used for stream
processing, real-time data pipelines, and data integration at scale. Originally
created to handle real-time data feeds at LinkedIn in 2011, Kafka quickly
evolved from messaging queue to a full-fledged event streaming platform capable
of handling over 1 million messages per second, or trillions of messages per
day.
Get Started for FreeGet the Free Kafka E-Book
WHY KAFKA?
Kafka has numerous advantages. Today, Kafka is used by over 80% of the Fortune
100 across virtually every industry, for countless use cases big and small. It
is the de facto technology developers and architects use to build the newest
generation of scalable, real-time data streaming applications. While these can
be achieved with a range of technologies available in the market, below are the
main reasons Kafka is so popular.
HIGH THROUGHPUT
Capable of handling high-velocity and high-volume data, Kafka can handle
millions of messages per second.
HIGH SCALABILITY
Scale Kafka clusters up to a thousand brokers, trillions of messages per day,
petabytes of data, hundreds of thousands of partitions. Elastically expand and
contract storage and processing.
LOW LATENCY
Can deliver these high volume of messages using a cluster of machines with
latencies as low as 2ms
PERMANENT STORAGE
Safely, securely store streams of data in a distributed, durable, reliable,
fault-tolerant cluster
HIGH AVAILABILITY
Extend clusters efficiently over availability zones or connect clusters across
geographic regions, making Kafka highly available and fault tolerant with no
risk of data loss.
Benchmarking RabbitMQ vs Kafka vs Pulsar
HOW KAFKA WORKS
Apache Kafka consists of a storage layer and a compute layer that combines
efficient, real-time data ingestion, streaming data pipelines, and storage
across distributed systems. In short, this enables simplified, data streaming
between Kafka and external systems, so you can easily manage real-time data and
scale within any type of infrastructure.
REAL-TIME PROCESSING AT SCALE
An data streaming platform would not be complete without the ability to process
and analyze data as soon as it's generated. The Kafka Streams API is a powerful,
lightweight library that allows for on-the-fly processing, letting you
aggregate, create windowing parameters, perform joins of data within a stream,
and more. Perhaps best of all, it is built as a Java application on top of
Kafka, keeping your workflow intact with no extra clusters to maintain.
DURABLE, PERSISTENT STORAGE
An abstraction of a distributed commit log commonly found in distributed
databases, Apache Kafka provides durable storage. Kafka can act as a 'source of
truth', being able to distribute data across multiple nodes for a highly
available deployment within a single data center or across multiple availability
zones.
PUBLISH + SUBSCRIBE
At its heart lies the humble, immutable commit log, and from there you can
subscribe to it, and publish data to any number of systems or real-time
applications. Unlike messaging queues, Kafka is a highly scalable, fault
tolerant distributed system, allowing it to be deployed for applications like
managing passenger and driver matching at Uber, providing real-time analytics
and predictive maintenance for British Gas' smart home, and performing numerous
real-time services across all of LinkedIn. This unique performance makes it
perfect to scale from one app to company-wide use.
WHAT IS KAFKA USED FOR?
Commonly used to build real-time streaming data pipelines and real-time
streaming applications, today, there are hundreds of Kafka use cases. Any
company that relies on, or works with data can find numerous benefits.
DATA PIPELINES
In the context of Apache Kafka, a streaming data pipeline means ingesting the
data from sources into Kafka as it's created and then streaming that data from
Kafka to one or more targets.
STREAM PROCESSING
Stream processing includes operations like filters, joins, maps, aggregations,
and other transformations which enterprises leverage to power many use-cases.
Kafka Streams is a stream processing library built for Apache Kafka enabling
enterprises to process data in real-time.Learn more
STREAMING ANALYTICS
Kafka provides high throughput event delivery, and when combined with
open-source technologies such as Druid can form a powerful Streaming Analytics
Manager (SAM). Druid consumes streaming data from Kafka to enable analytical
queries. Events are first loaded in Kafka, where they are buffered in Kafka
brokers before they are consumed by Druid real-time workers.
STREAMING ETL
Real-time ETL with Kafka combines different components and features such as
Kafka Connect source and sink connectors to consume and produce data from/to any
other database, application, or API, Single Message Transform (SMT) – an
optional Kafka Connect feature, Kafka Streams for continuous data processing in
real-time at scale.
EVENT-DRIVEN MICROSERVICES
Apache Kafka is the most popular tool for microservices because it solves many
of the issues of microservices orchestration while enabling attributes that
microservices aim to achieve, such as scalability, efficiency, and speed. It
also facilitates inter-service communication while preserving ultra-low latency
and fault tolerance.
APACHE KAFKA IN ACTION
WHO USES KAFKA?
TO MAXIMIZE KAFKA, YOU NEED CONFLUENT
Founded by the original developers of Kafka, Confluent delivers the most
complete distribution of Kafka with Confluent, improving Kafka with additional
community and commercial features designed to enhance the streaming experience
of both operators and developers in production, at massive scale.
You love Apache Kafka®, but not managing it. Confluent's cloud-native, complete,
and fully managed service goes above & beyond Kafka so your best people can
focus on what they do best - delivering value to your business.
Cloud Kafka
CLOUD-NATIVE
We’ve re-engineered Kafka to provide a best-in-class cloud experience, for any
scale, without the operational overhead of infrastructure management. Confluent
offers the only truly cloud-native experience for Kafka—delivering the
serverless, elastic, cost-effective, highly available, and self-serve experience
that developers expect.
Complete Kafka
COMPLETE
Creating and maintaining real-time applications requires more than just open
source software and access to scalable cloud infrastructure. Confluent makes
Kafka enterprise ready and provides customers with the complete set of tools
they need to build apps quickly, reliably, and securely. Our fully managed
features come ready out of the box, for every use case from POC to production.
Kafka Everywhere
EVERYWHERE
Distributed, complex data architectures can deliver the scale, reliability, and
performance that unlocks use cases previously unthinkable, but they're
incredibly complex to run. Confluent's complete, multi-cloud data streaming
platform makes it easy to get data in and out of Kafka Connect, manage the
structure of data using Confluent Schema Registry, and process it in real time
using ksqlDB. Confluent meets our customers everywhere they need to be —
powering and uniting real-time data across regions, clouds, and on-premises
environments.
GET STARTED IN MINUTES
By integrating historical and real-time data into a single source of truth,
Confluent makes it easy to build an entirely new category of modern,
event-driven applications, gain a universal data pipeline, and unlock powerful
new use cases with full scalability, security, and performance.
Try free with $400 in free credits to spend during your first four months. No
credit card required.
Try FreeSee All Solutions
* Product
* Confluent Platform
* Connectors
* ksqlDB
* Stream Governance
* Confluent Hub
* Subscription
* Professional Services
* Training
* Customers
* Cloud
* Confluent Cloud
* Support
* Sign Up
* Log In
* Cloud FAQ
* Solutions
* Financial Services
* Insurance
* Retail and eCommerce
* Automotive
* Government
* Gaming
* Communication Service Providers
* Technology
* Manufacturing
* Fraud Detection
* Customer 360
* Messaging Modernization
* Streaming ETL
* Event-driven Microservices
* Mainframe Offload
* SIEM Optimization
* Hybrid and Multicloud
* Internet of Things
* Data Warehouse
* Developers
* Confluent Developer
* What is Kafka?
* Resources
* Events
* Online Talks
* Meetups
* Current: Data Streaming Event
* Tutorials
* Docs
* Blog
* About
* Investor Relations
* Startups
* Company
* Careers
* Partners
* News
* Contact
* Shop Merchandise
* Trust and Security
*
*
*
*
*
*
*
*
*
*
*
*
*
*
Terms & Conditions | Privacy Policy | Do Not Sell My Information | Modern
Slavery Policy | Cookie Settings
Copyright © Confluent, Inc. 2014-2022. Apache, Apache Kafka, Kafka, and
associated open source project names are trademarks of the Apache Software
Foundation
By clicking “Accept All Cookies”, you agree to the storing of cookies on your
device to enhance site navigation, analyze site usage, and assist in our
marketing efforts. Cookie Notice
Cookies Settings Reject All Accept All Cookies