Interesting Stuff - Week 36, 2022

Photo by DeepMind on Unsplash

Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:

  • AI/data science
  • data in general
  • data architecture
  • streaming
  • distributed computing
  • SQL Server
  • transactions (both db as well as non db)
  • and other “stuff”

This blog post is the “roundup” of the things that have been most interesting to me for the week just ending.

Machine Learning / AI

Azure Data Explorer

  • Generally available: Kusto Emulator. This is awesome! Microsoft has released a Docker image exposing the Kusto Query Engine endpoint in Azure Data Explorer. This means we can now create databases, and ingest and query data in a local environment!

Streaming

  • IoT Real-Time Analytics — WAGO PLC with Databricks Auto Loader. Databricks Auto Loader allows you to stream or incrementally ingest and transform files from a data lake. This post looks at a solution whereby IoT messages are sent to an Azure IoT hub and ingested in Azure Data Lake Storage (ADLS). From ADLS, the data is streamed into Azure Databricks Delta Tables using Databricks Auto Loader. Very interesting, this may be something for us at Derivco!
  • Welcome to Streamiz.Kafka.Net. If you have used Kafka, you are probably aware of KStreams, the library to create stream processing applications. You are also most likely aware that KStreams is Java/Scala only. That is about to change now with Streamiz, a .NET stream processing library for Kafka, i.e. similar to KStreams. It is in active development, and if you are into Kafka, stream processing and .NET, I urge you to look at it.

WIND (What Is Niels Doing)

I am prepping for this:

Yes, on September 22 at 12:30 UTC, I deliver the session ksqlDB - The Real Time Streaming Database at Data Platform Summit 2022.

The conference is free; you register here.

~ Finally

That’s all for this week. I hope you enjoy what I did put together. Please comment on this post or ping me if you have ideas for what to cover.


Blog Feed:

To automatically receive more posts like this, please subscribe to my RSS/Atom feed in your feed reader!

Interesting Stuff - Week 35, 2022 Interesting Stuff - Week 37, 2022

Related Articles

comments powered by Disqus