aiven.io Open in urlscan Pro
2a09:8280:1::37:4b0  Public Scan

Form analysis
1 forms found in the DOM

POST /community/forum/login

<form id="hidden-login-form" method="post" action="/community/forum/login" style="display: none;">
  <input name="username" type="text" id="signin_username">
  <input name="password" type="password" id="signin_password">
  <input name="redirect" type="hidden">
  <input type="submit" id="signin-button" value="Log In">
</form>

Text Content

Skip to main content


 * Platform
   * Event streaming services
     
     * Aiven for Apache Kafka® 
       
       Distributed event streaming platform for high-throughput data pipelines
     
     * Aiven for Apache Flink® 
       
       Event stream processing platform for real-time ETL and analytics needs
   * Analytics database services
     
     * Aiven for OpenSearch® 
       
       Distributed search and analytics suite with a rich set of extensions
     
     * Aiven for ClickHouse® 
       
       Fast, resource effective cloud data warehouse for analytical workloads
     
     * Aiven for M3 
       
       Distributed time series database for monitoring in scale
     
     * Aiven for InfluxDB® 
       
       Time series database designed for variable data sets
   * Operational database services
     
     * Aiven for PostgreSQL® 
       
       High performance relational database with advanced extensions
     
     * Aiven for MySQL 
       
       Popular relational database for a wide range of applications
     
     * Aiven for Redis® 
       
       In-memory, key-value NoSQL database with a small footprint
     
     * Aiven for Apache Cassandra® 
       
       Distributed, wide-column NoSQL database designed to handle large amounts
       of data
   * Data visualization services
     
     * Aiven for Grafana® 
       
       Data visualization suite for monitoring, analytics and alerting
   * Platform
     
     * Aiven Platform 
       
       One platform for streaming, storing and analyzing data on any cloud
     
     * Integrations and connectors 
       
       Integrate Aiven services with your existing monitoring, orchestration and
       data infrastructure
     
     * Security and compliance 
       
       We provide a premium level of security on all accounts, regardless of
       size
     
     * Support services 
       
       We proactively monitor all services 24/7/365, with Basic support always
       included, and three additional tiers available
     
     * View our DevOps tools 
       
       
 * Solutions
   * By use case
     
     * Event streaming 
       
       Event streaming for the continuous movement and processing of real time
       data
     
     * Monitoring and observability 
       
       Open source observability tools for large scale monitoring
     
     * Time Series Databases 
       
       Everything you always wanted to know about the next data dimension
     
     * Aiven for IoT 
       
       A robust data pipeline build for ever-increasing data loads
   * By industry
     
     * Aiven for retail 
       
       Flexible retail data solutions for any situation and budget
     
     * Aiven for energy 
       
       Distributed and scalable solutions for the smart grid
     
     * View all solutions 
       
       
   * Why Aiven
     
     * Download the IDC white paper 
       
       
     
     * Optimize cloud costs with Aiven's data platform 
       
       
 * Customers
   * Aiven customers
     
     * Customer spotlights 
       
       See how Aiven helps customers achieve success
   * Case studies
     
     * Conrad Electronic expands e-commerce platform with Aiven 
       
       
     
     * Fast-growing startup Hookdeck powers cloud data infrastructure with
       Aiven 
       
       
     
     * Digital Asset Research boosts resiliency and reduces downtime with Aiven 
       
       
     
     * Explore all stories 
       
       
   * Featured
     
     * Optimizing Cloud Infrastructure: Talon.One's 250 Databases and One-Third
       Cost Reduction 
       
       
 * Resources
   * Using Aiven
     
     * Developer center 
       
       Tutorials, deep dives and code examples with open source projects
     
     * Documentation 
       
       Product guides, tutorials and FAQs
     
     * Aiven Ideas 
       
       Submit your product ideas and explore our public roadmap
     
     * Aiven API 
       
       Integrate Aiven calls to your own processes
     
     * Changelog 
       
       Latest product updates and improvements
     
     * Incident status updates 
       
       Check current operational system status
   * Community
     
     * Aiven Community 
       
       Network and share ideas
     
     * Aiven Community Forum 
       
       Join the discussion
     
     * Open source at Aiven 
       
       Aiven and open source
     
     * Open Source Program Office 
       
       Contributing to a better future
     
     * Events calendar 
       
       Worldwide conferences, meetups, events and webinars
   * Learn
     
     * Blog 
       
       Aiven's resource for everything open source
     
     * Resource library 
       
       Ebooks, white papers, research and more from the Aiven archives.
     
     * Newsletter 
       
       Get open source and Aiven news direct to your inbox
     
     * Read our latest blog post
 * Company
   * About
     
     * About Aiven 
       
       Our vision, mission and history
     
     * Careers 
       
       Learn more about working at Aiven
     
     * Cluster startup program 
       
       Scale up your startup with Aiven
     
     * Sustainability 
       
       Building sustainable data infrastructure for everyone
     
     * Partners 
       
       Partner with Aiven and help drive innovation
     
     * Press room 
       
       Press releases and media resources
     
     * Contact us 
       
       Talk to us – we’d love to hear from you!
     
     * 
   * Careers
     
     * See our open positions 
       
       
   * Sustainability
     
     * Sustainable open source data infrastructure 
       
       
 * Pricing


Sign UpLog In
 * 



 * Topics
   
   
   
   
 * More
   

Categories
 * Aiven Community
 * All categories

Tags
 * klaw
 * free_plan
 * faq
 * cluster
 * startups
 * All tags



CHICAGO OSDI MEETUP RECAP - FUN & LEARNING PARTNERED WITH DISCOVER

Meetups & Events


You have selected 0 posts.

select all

cancel selecting

Aug 20
1 / 1
Aug 20

Aug 20

jennjunod
1
Aug 20


What happened at the Open Source Data Infrastructure Chicago meetup?

August 9 the folks at Discover hosted the Chicago Open Source Data
Infrastructure meetup - the first in Chicago from the meetups we’re organizing
globally - and co-located with the Devopsdays Chicago conference which I and a
few colleagues were in town for.

It was a fun evening, with talks by my colleague Dewan Ahmed, and Ehfaj Khan,
Expert Application Engineer at Discover Financial Services (DFS).



Open Source Infrastructure Data Meetup - Presented by Discover & Aiven



Ehfaj compared batch processing, a technique for processing large amounts of
data offline, with event (or: real-time) processing.

Key challenges with batch processing are that while the system can identify data
sets that are not fit for processing (yet) or even corrupted, sending them back
to the source system for correction causes delays in business processes.
Unstructured data (and data inconsistency and data integrity issues) and batch
processing are no friends. And: debugging can be painful and time-consuming when
processing large amounts of data.

Data is becoming the backbone and nervous system of any organization.
Organizations are pushing to modernize their data infrastructure at an
increasing rate. Speed as a key element of business success, that’s where
real-time data comes in!

Event processing provides you with a live feedback loop, so that you can act on
your data as it comes in. Real-time visibility in your business processing and
better response to your target customers makes for better customer experience.
Event processing also significantly enhances the organization’s visibility of
its operations and enables the organization to make faster and better decisions.

But whether event processing is the answer for your business, that (of course)
depends. Like the successful adoption of any new technology, tools or framework
depends on your use case and the business problem you’re trying to solve.

Dewan talked about Infrastructure-as-Code (IaC) for data. Or, like he said: “we
build apps to move data” - of course IaC is for data as well.

Your streaming platform, relational database, NoSQL database, networking,
monitoring and security need the same “-ilities” that the rest of your system
needs:

 * Reproducibility
 * Repeatability
 * Disposability (pets vs cattle)
 * Consistency
 * Ability to incorporate design changes

Here too it’s important to investigate first internally whether IaC is right for
you, if you’re set up right to create a pilot project, to ultimately (maybe)
roll it out wherever it makes sense.

But to showcase how orchestration with Terraform would work, Dewan shared a demo
in which he uses Apache Kafka Mirrormaker to replicate data over different data
centers, from (Kafka) source to target cluster.

Data center 1 AKMM Data center 2 topic-a Replication flow topic-b; dc1.topic-a
Apache Kafka Source Apache Kafka Target

You can check out his demo here.

If you’re in the Chicago area, make sure to join the Chicago OSDI Meetup Group

2

 * 2023-08 Edition: What's Up With DevRel at Aiven?




Reply



NEW & UNREAD TOPICS

Topic Replies Views Activity Open Source Data Infrastructure Meetup Milan 21st
June 2023
Meetups & Events
1 215 Jun 23 I have a few doubts about free plan condition
MySQL
free_plan
1 527 May 2 Aiven Operator for Kubernetes version 0.13.0 now available
News & Announcements
0 109 6d Announcing database migration support for PostgreSQL, MySQL, and Redis
in Aiven Console
News & Announcements
free_plan
0 104 21d Jakarta Open Source Data Infrastructure meetup - July
APAC Meetups & Events
0 164 Jul 24


WANT TO READ MORE? BROWSE OTHER TOPICS IN MEETUPS & EVENTS OR VIEW LATEST
TOPICS.

Share








   Platform
 * Aiven Platform
 * Integrations and connectors
 * Security and compliance
 * Support services
 * DevOps tools
 * Pricing
 * Optimize cloud costs with Aiven

   Company
 * About
 * Careers
 * Cluster startup program
 * Sustainability
 * Partners
 * Press room
 * Aiven invoicing addresses

   Resources
 * Developer center
 * Blog
 * Events calendar
 * Resource library
 * Customer spotlights
 * Aiven community
 * Aiven community forum
 * Open source at Aiven
 * Open Source Program Office
 * Documentation
 * Aiven Ideas
 * Aiven API
 * Changelog
 * Incident status updates

   Solutions
 * Event-driven architecture
 * Monitoring and observability
 * Time Series Databases
 * Aiven for IoT
 * Aiven for retail
 * Aiven for energy
 * All solutions

   Services
 * Aiven for Apache Kafka®
 * Aiven for Apache Kafka® Connect
 * Aiven for Apache Flink®
 * Aiven for ClickHouse®
 * Aiven for M3
 * Aiven for Apache Cassandra®
 * Aiven for OpenSearch®
 * Aiven for PostgreSQL®
 * Aiven for MySQL®
 * Aiven for Redis®
 * Aiven for InfluxDB®
 * Aiven for Grafana®

Let's connect
 * Book a demo
 * Contact us
 * Subscribe to newsletter
 * Community forum
 * Events calendar

 * 
 * 
 * 
 * 
 * 
 * 

 * Terms of services
 * SLA
 * Data processing agreement
 * Privacy policy
 * Cookie policy
 * Subprocessors & subcontractors
 * Responsibility matrix
 * Do not sell or share my personal information

Apache, Apache Kafka, Kafka, Apache Flink, Flink, Apache Cassandra, and
Cassandra are either registered trademarks or trademarks of the Apache Software
Foundation in the United States and/or other countries. ClickHouse is a
registered trademark of ClickHouse, Inc. https://clickhouse.com. M3, M3
Aggregator, M3 Coordinator, OpenSearch, PostgreSQL, MySQL, InfluxDB, Grafana,
Terraform, and Kubernetes are trademarks and property of their respective
owners. *Redis is a registered trademark of Redis Ltd. and the Redis box logo is
a mark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by
Aiven is for referential purposes only and does not indicate any sponsorship,
endorsement or affiliation between Redis and Aiven. All product and service
names used in this website are for identification purposes only and do not imply
endorsement.




Invalid date Invalid date



Select “Allow all” to let us store some tasty performance cookies on your device
and deliver an awesome experience. Enjoy ourcookie policyand cookie recipe

Customize settings Allow all



PRIVACY PREFERENCE CENTER

When you visit any website, it may store or retrieve information on your
browser, mostly in the form of cookies. This information might be about you,
your preferences or your device and is mostly used to make the site work as you
expect it to. The information does not usually directly identify you, but it can
give you a more personalized web experience. Because we respect your right to
privacy, you can choose not to allow some types of cookies. Click on the
different category headings to find out more and change our default settings.
However, blocking some types of cookies may impact your experience of the site
and the services we are able to offer.
Cookie Policy
Allow all


MANAGE CONSENT PREFERENCES

PERFORMANCE COOKIES

Performance cookies


These cookies allow us to count visits and traffic sources so we can measure and
improve the performance of our site. They help us to know which pages are the
most and least popular and see how visitors move around the site. All
information these cookies collect is aggregated and therefore anonymous. If you
do not allow these cookies we will not know when you have visited our site, and
will not be able to monitor its performance.

TARGETING COOKIES

Targeting cookies


These cookies may be set through our site by our advertising partners. They may
be used by those companies to build a profile of your interests and show you
relevant adverts on other sites. They do not store directly personal
information, but are based on uniquely identifying your browser and internet
device. If you do not allow these cookies, you will experience less targeted
advertising.

NECESSARY COOKIES

Always active


These cookies are necessary for the website to function and cannot be switched
off in our systems. They are usually only set in response to actions made by you
which amount to a request for services, such as setting your privacy
preferences, logging in or filling in forms. You can set your browser to block
or alert you about these cookies, but some parts of the site will not then work.
These cookies do not store any personally identifiable information.

SOCIAL MEDIA COOKIES

Social media cookies


These cookies are set by a range of social media services that we have added to
the site to enable you to share our content with your friends and networks. They
are capable of tracking your browser across other sites and building up a
profile of your interests. This may impact the content and messages you see on
other websites you visit. If you do not allow these cookies you may not be able
to use or see these sharing tools.

FUNCTIONAL COOKIES

Functional cookies


These cookies enable the website to provide enhanced functionality and
personalization. They may be set by us or by third party providers whose
services we have added to our pages. If you do not allow these cookies then some
or all of these services may not function properly.

UX COOKIES

UX cookies


These cookies help us inform our choices on user experience for the marketing
website and Aiven Console.

Back Button


PERFORMANCE COOKIES



Search Icon
Filter Icon

Clear
checkbox label label
Apply Cancel
Consent Leg.Interest
checkbox label label
checkbox label label
checkbox label label

Save settings