www.ssp.sh Open in urlscan Pro
2a00:d70:0:b:2002:0:d91a:342b  Public Scan

Form analysis
0 forms found in the DOM

Text Content

🧠 SECOND BRAIN



Search

Search IconIcon to open search
Light Mode Dark Mode
newsletter
newsletter


MY GIGS (PODCASTS, VIDEOS, SLIDES, ETC.)

Last updated Mar 30, 2024

Table of Contents
 1. 5. Bedag Podcast: Open Source Meets Data Engineering
    1. Key Insights
    2. Open Source: The Core of Innovation
    3. Data Engineering: The Future of Data Processing
 2. 4. Open-source data engineering, productivity, writing, and philosophy
    1. Key Insights
 3. 3. TechTalk with Databricks Denny Lee: Data Integration with Airbyte and
    Delta Lake
    1. Key Insights
    2. Personal Reflection
 4. 2. Data Engineer Turns His Hobby Into a Career
    1. Key Insights
    2. Personal Reflection
 5. 1. Dagster at Rohde and Schwarz Mobile Network Testing
    1. Key Insights
    2. Personal Reflection

Here I have a list of my appearance online or slides I shared during an event.


# 5. BEDAG PODCAST: OPEN SOURCE MEETS DATA ENGINEERING

This podcast episode Matthias Stürmer and I dive into the topic of Open Source
and Data. Matthias is a professor at the Bern University of Applied Sciences
(BFH) and Head of the Institute for Public Sector Transformation. I’m grateful
for his time and the insightful discussion we had.

🎧 Listen on Spotify - 🚨 Attention: This podcast is in Swiss German


# KEY INSIGHTS


# OPEN SOURCE: THE CORE OF INNOVATION

 * What is Open Source? A behind-the-scenes look: Open Source stands for access
   and freedom. Unlike Closed Source software like CRM or ERP, Open Source
   offers the opportunity to customize and further develop the code.
 * The Open Model: Why it represents the perfect balance between Open Source
   flexibility and commercial success. No lock-in, more flexibility, and
   innovation, but with the awareness that more work and maintenance are
   required.
 * Open Core or Closed Source? The choice between Buy vs. Build depends heavily
   on technical capabilities and long-term goals. A Managed Service supporting
   the Open Core model can be an attractive solution.


# DATA ENGINEERING: THE FUTURE OF DATA PROCESSING

 * HelloDATA: An open enterprise data platform, developed by Bedag and adopted
   by the Canton of Bern. We discuss the advantages of choosing Open Source for
   data platforms and why it can be a superior alternative to solutions like
   Azure/Power BI.
 * Why not Azure/Power BI? A critical look at the lock-in risks and the
   importance of Open Standards and Open Source for a sustainable data strategy.

Join us as we explore these topics with industry experts and gain deep insights
into the world of Open Source and Data Engineering. Your opinions and questions
are, as always, welcome – let’s shape the future of technology together!

Podcast Bedag - OSS Data Engineering


# 4. OPEN-SOURCE DATA ENGINEERING, PRODUCTIVITY, WRITING, AND PHILOSOPHY

In this episode, I had the pleasure of talking to Thomas Bustos on “Let’s Talk
AI.” We explored topics close to my heart, ranging from open-source data
engineering to the nuances of my productivity workflow. We even touched upon the
philosophies I’ve embraced through life’s journey. As a Swiss data engineer who
transitioned into a writer and author, this episode encapsulates my two decades
of experience and my unbridled passion for productivity and knowledge sharing.

📺 Watch on YouTube
🎧 Listen on Spotify




# KEY INSIGHTS

Some sneak peeks from it:

Reflecting on the Modern Data Stack: The future of data engineering, I see a
landscape where tools consolidate, integrating more seamlessly, with Kubernetes
steering the ship. My fascination lies in the continuous evolution of the
Semantic Layer and the untapped potential of Rust in the realm of data
engineering. These advancements might redefine how we approach data engineering
in the future.

Career Insights: I advocate for a “pathless path” approach, one that’s guided by
curiosity and intuition. Engage in personal projects; they are fun and offer
deep, practical learning. Remember, learning is a journey, not a race, so take
your time to absorb and understand.

Merging Programming, Writing, and Note-Taking: In pursuing productivity, I’ve
found a unique harmony in combining my love for programming with my passion for
writing and note-taking. Tools like Obsidian and Vim motions are integral to my
workflow, enhancing efficiency and organization. Embracing the Second Brain
concept, I’ve created a comprehensive knowledge repository, uniting all these
facets seamlessly.

Embracing Minimalism in Work and Writing: My work, both as a data engineer and
writer, is positively influenced by minimalism. This approach helps me focus on
the essentials, strip away the superfluous, and truly listen to my intuition.
This minimalist philosophy aids in making more meaningful decisions in my
professional and personal life.

Stoic Principles: Adopting Stoicism has been transformative, helping me manage
my thoughts and emotions effectively. The stoic principle of focusing on what I
can control and accepting what I can’t has been a guiding light in my work and
personal life.

Advice on DE Programming Languages: In the ever-evolving landscape of data
engineering, Python and SQL remain the cornerstone, with Rust emerging as a
powerful tool for performant applications.

Ending Message: Stay curious, trust yourself, and embrace the journey. I’m a
proponent of “Slow Productivity” – a mindful, deliberate approach to work and
life.

Date: 2023-11-28


# 3. TECHTALK WITH DATABRICKS DENNY LEE: DATA INTEGRATION WITH AIRBYTE AND DELTA
LAKE

In this engaging session with Denny Lee from Databricks, I explored the
intricate world of data integration using Airbyte and Delta Lake. We delved into
the challenges and solutions of integrating diverse data sources into a unified
data lake. This discussion was particularly enriching as it combined practical
insights with theoretical understanding, bridging the gap between concept and
application in the realm of data engineering.

📺 Watch on YouTube
🎧 Listen on Spotify




# KEY INSIGHTS

 * Integration Challenges: We discussed the complexity of integrating multiple
   data sources, emphasizing the importance of robust and scalable solutions
   like Airbyte for efficient data ingestion.
 * The Power of Delta Lake: The conversation highlighted Delta Lake’s role in
   simplifying data management, offering a structured yet flexible approach to
   handling vast datasets.
 * Open Source Synergy: A significant focus was on the synergy between
   open-source technologies like Airbyte and Delta Lake, showcasing how they
   empower data engineers to build more dynamic and resilient data pipelines.
 * Future of Data Engineering: The discussion ventured into the future trends in
   data engineering, touching on the evolving landscape and the role of emerging
   technologies in shaping it.


# PERSONAL REFLECTION

The interaction with Denny not only allowed for a deep dive into technical
nuances but also offered a chance to reflect on the broader implications and
future direction of data engineering. It served as a reminder of the continuous
learning and adaptation required in our ever-evolving field.

Date: 2022-12-06


# 2. DATA ENGINEER TURNS HIS HOBBY INTO A CAREER

In this heartening podcast episode, I delved into my personal journey from a
business intelligence engineer to a data engineer and eventually embraced my
passion for writing as a career. The pivot from a conventional BI engineering
role to the more dynamic field of data engineering was driven by a desire for
automation and a deeper engagement with coding, moving away from the
drag-and-drop tools of traditional BI platforms.

📺 Watch on YouTube




# KEY INSIGHTS

 * Transition from BI to Data Engineering: This episode highlighted my shift
   from business intelligence, where I worked with tools like SSIS and Oracle,
   to embracing the challenges and creativity of data engineering. It
   underscored the evolution of the field and how I adapted to these changes.
 * Embracing Writing as a Career: Discussing my side hobby of writing about
   data, productivity, and personal knowledge management, I shared how this
   hobby transformed into a professional pursuit. This shift was marked by a
   transition from writing as a personal interest to taking up technical writing
   as part of my professional role.
 * The Power of Continuous Learning: The conversation also focused on the
   importance of continuous learning and staying updated with the latest trends
   in technology. This aspect has been crucial in both my technical and writing
   endeavors.
 * Innovation and Automation: I shared insights into my drive for innovation and
   automation, a core aspect of my work in data engineering. This reflects my
   approach to work, where I constantly seek to streamline processes and enhance
   efficiency.


# PERSONAL REFLECTION

Reflecting on my journey, this podcast with Chris was a testament to the power
of pursuing one’s passions and interests. My story illustrates that career paths
are not linear and that embracing change and new opportunities can lead to
fulfilling and dynamic career trajectories. This episode serves as an
encouragement to others to explore and integrate their hobbies and passions into
their professional lives.

Date: 2022-08-23


# 1. DAGSTER AT ROHDE AND SCHWARZ MOBILE NETWORK TESTING

In this podcast episode, I shared the innovative use of Dagster


DAGSTER

Dagster is one of many . It operates in a and manner, offering unique
capabilities in data...

15.8.2024

at Rohde & Schwarz Mobile Network Testing. My focus was on detailing the
transformation in data architecture at Rohde & Schwarz, showcasing how we
harnessed the power of open-source data engineering tools, orchestrated with
Dagster.

📺 Watch on YouTube
📄 View Slides




# KEY INSIGHTS

 * Rohde & Schwarz’s Journey: I described Rohde & Schwarz’s journey in
   developing smart analytics for mobile network testing, highlighting the
   company’s focus on electronic test equipment, broadcast and media,
   cybersecurity, and radio communications.
 * Innovative Data Architecture: The core of my talk revolved around the
   innovative data architecture we implemented at Rohde & Schwarz. This included
   the transition of our data warehouse to the cloud, aiming for big data
   readiness and the integration of various open-source tools.
 * Dagster’s Role: I explained how Dagster played a pivotal role in managing and
   orchestrating our ETL (Extract, Transform, Load) logic in a centralized
   manner. The move to Dagster was motivated by the need for a more scalable,
   cloud-compatible tool that could handle increasing data volumes due to 5G
   advancements.
 * Custom ETL and Challenges: The custom ETL processes were a significant part
   of our architecture, dealing with complex data transformation and
   aggregation. Challenges included handling large data sets, ensuring fast
   query responses, and maintaining an efficient web UI for user interactions.
 * Dagster Advantages: I elaborated on the advantages of using Dagster, such as
   its out-of-the-box features like restart capabilities, backfilling,
   dependency management, and a user-friendly UI. Dagster’s functional design
   and the reusability of code significantly streamlined our development
   process.
 * Future Directions: Looking forward, I highlighted our plans to enhance our
   use of Dagster with unit testing, smoke tests, more intensive use of assets,
   integration of data lineage features, and the exploration of dynamic
   orchestration and partitioning.


# PERSONAL REFLECTION

This presentation was a deep dive into how data engineering and architecture can
be revolutionized in a corporate setting. It emphasized the importance of
embracing open-source tools and technologies to stay ahead in the rapidly
evolving field of data engineering. My journey with Dagster at Rohde & Schwarz
serves as a case study for others in the industry, demonstrating how adapting to
new tools and methodologies can lead to significant improvements in data
processing and analysis.

Date: 2021-05-11

--------------------------------------------------------------------------------

Origin:
References: Data Engineering Podcasts


DATA ENGINEERING PODCASTS

Best Data Thoughtleader Interview Podcasts The Analytics Engineering
Podcast (site, RSS) The Data Engineering Podcast (site, twitter, RSS) The Data
Stack Show (site, twitter, RSS) Intricity101 Data Sharks
Podcast (site, twitter, RSS) Drill to Detail with...

15.8.2024


Created 2023-11-27


INTERACTIVE GRAPH

My Gigs (Podcasts, Videos, Slides, etc.)/matthias st%c3%bcrmer/hellodata/podcast
bedag-oss-data-engineering/thomas bustos/2023 11-28/delta lake/2022 12-06/2022
08-23Dagster/2021 05-11Data Engineering Podcasts/2023 11-27
 * Home
 * Blog
 * Twitter
 * LinkedIn
 * Github


BACKLINKS

 * No backlinks found