Niels Berglund

One Billion Row Challenge & Azure Data Explorer

Gunnar Morling initiated the One Billion Row Challenge on his blog, aiming to load and aggregate one billion rows of temperature data using Java. In this blog post, I show how I did the challenge using Azure Data Explorer. It was not without challenges (pun intended), but I got there in the end.

Posted by nielsb on Sunday, January 28, 2024

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II

In this post, I do not cover that much about leaderboards. I instead look at ingesting large volumes of data into ADX and how to transform that data using Azure Data Explorer Update Policies.

Posted by nielsb on Monday, April 3, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I

This post continues the journey towards creating a leaderboard system. In this post we learn more about ADX and why it is an excellent choice for near real-time applications, how to 'hook up' Kafka to ADX and how to ingest data into an ADX database.

Posted by nielsb on Sunday, March 19, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - Prelude

This post is the first in a series about developing online gaming leaderboards. This post sets the foundation and you use Docker to run Kafka as well as Python in a container generating and publishing events.

Posted by nielsb on Sunday, February 26, 2023

Interesting Stuff - Week 15, 2022

Kafka internals, Kafka & Trino @ Uber, Feathr feature store, IoT & Azure Data Explorer, and other interesting topics.

Posted by nielsb on Sunday, April 17, 2022

Interesting Stuff - Week 9, 2022

Kafka at Linkedin, Free clusters for KQL and Azure Data Explorer, Delta Lake & Synapse, SQLBits 2022, and other interesting topics.

Posted by nielsb on Saturday, March 5, 2022

How to Ingest Into Azure Data Explorer From Apache Kafka using Kafka Connect

We look at how to configure and set up Kafka Connect to allow ingestion into Azure Data Explorer.

Posted by nielsb on Wednesday, October 27, 2021

Interesting Stuff - Week 41, 2021

Kafka Stream Governance, VSCode & Python, timeseries in ADX, and other interesting topics.

Posted by nielsb on Sunday, October 10, 2021

Technology musings about coding and data. Some topics: .NET, SQL Server, AI/ML, Big Data, Streaming, Kafka, Azure Data Explorer, and a lot more.