Niels Berglund

Interesting Stuff - Week 17, 2023

The man that beat roulette, hooking up Azure Data Explorer and Kafka, real-time machine learning pipelines. Spark loader for Hugging Face datasets, processing data from multiple streaming platforms using Delta Live Tables and the real-time streaming ecosystem.

Posted by nielsb on Sunday, April 30, 2023

Interesting Stuff - Week 16, 2023

In this week's roundup, I cover some 'cool stuff': Azure Databricks Delta Engine, multi-platform Docker image for Kafka Connect and Kusto Sink. Generative AI for Synthetic Data creation. Using Databricks SQL together with LLMs.

Posted by nielsb on Sunday, April 23, 2023

Cross-Platform Compatibility Made Easy with Multi-Platform Docker Images: Azure Data Explorer Sink Connector & Kafka Connect

In this blog post, we'll explore the steps required to build and deploy multi-platform Docker images, without having to apply platform specific tags.

Posted by nielsb on Monday, April 17, 2023

Interesting Stuff - Week 14, 2023

This week: **Azure Data Explorer** (ADX) workshop, the third post in the series of building a real-time leaderboard using Kafka and ADX. Also links to building a GPT-powered application and a really interesting intro to reinforcement learning.

Posted by nielsb on Sunday, April 9, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - II

In this post, I do not cover that much about leaderboards. I instead look at ingesting large volumes of data into ADX and how to transform that data using Azure Data Explorer Update Policies.

Posted by nielsb on Monday, April 3, 2023

Interesting Stuff - Week 13, 2023

This week: be cool, read about Polar vs Pandas. How to ingest data into ADX. On the streaming side really interesting about how to do Canary releases for streaming platforms. Also something extremely interesting: Machine Learning and Data Streaming!

Posted by nielsb on Sunday, April 2, 2023

Interesting Stuff - Week 11, 2023

This week I link to a learning path for Azure Data Explorer and Kusto Query Language. I finally published the second post about Leaderboards using Kafka and ADX. In the post you also find OpenAI "stuff".

Posted by nielsb on Sunday, March 19, 2023

Develop a Real-Time Leaderboard Using Kafka and Azure Data Explorer - I

This post continues the journey towards creating a leaderboard system. In this post we learn more about ADX and why it is an excellent choice for near real-time applications, how to 'hook up' Kafka to ADX and how to ingest data into an ADX database.

Posted by nielsb on Sunday, March 19, 2023

Interesting Stuff - Week 9, 2023

This weekly roundup covers prompt engineering for Generative AI. I also cover stream processing vs real-time OLAP DBs; using Kafka and other open-source technologies to build AWS data lakes. To finish off: Spark Structured Streaming and REST APIs.

Posted by nielsb on Sunday, March 5, 2023

Interesting Stuff - Week 8, 2023

This week I cover Linux containers on Apple Silicon. There is also how to use Mastodon, Kafka and Open Search together. Let us not forget my post about leaderboards and Azure Data Explorer.

Posted by nielsb on Sunday, February 26, 2023

Technology musings about coding and data. Some topics: .NET, SQL Server, AI/ML, Big Data, Streaming, Kafka, Azure Data Explorer, and a lot more.