Niels Berglund

One Billion Row Challenge & Azure Data Explorer

Gunnar Morling initiated the One Billion Row Challenge on his blog, aiming to load and aggregate one billion rows of temperature data using Java. In this blog post, I show how I did the challenge using Azure Data Explorer. It was not without challenges (pun intended), but I got there in the end.

Posted by nielsb on Sunday, January 28, 2024

Cross-Platform Compatibility Made Easy with Multi-Platform Docker Images: Azure Data Explorer Sink Connector & Kafka Connect

In this blog post, we'll explore the steps required to build and deploy multi-platform Docker images, without having to apply platform specific tags.

Posted by nielsb on Monday, April 17, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II

In this post, I do not cover that much about leaderboards. I instead look at ingesting large volumes of data into ADX and how to transform that data using Azure Data Explorer Update Policies.

Posted by nielsb on Monday, April 3, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I

This post continues the journey towards creating a leaderboard system. In this post we learn more about ADX and why it is an excellent choice for near real-time applications, how to 'hook up' Kafka to ADX and how to ingest data into an ADX database.

Posted by nielsb on Sunday, March 19, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - Prelude

This post is the first in a series about developing online gaming leaderboards. This post sets the foundation and you use Docker to run Kafka as well as Python in a container generating and publishing events.

Posted by nielsb on Sunday, February 26, 2023

How to Ingest Into Azure Data Explorer From Apache Kafka using Kafka Connect

We look at how to configure and set up Kafka Connect to allow ingestion into Azure Data Explorer.

Posted by nielsb on Wednesday, October 27, 2021

Technology musings about coding and data. Some topics: .NET, SQL Server, AI/ML, Big Data, Streaming, Kafka, Azure Data Explorer, and a lot more.