Throughout the week, I read a lot of blog-posts, articles, and so forth that has to do with things that interest me:
- AI/data science
- data in general
- data architecture
- streaming
- distributed computing
- SQL Server
- transactions (both db as well as non db)
- and other “stuff”
This blog-post is the “roundup” of the things that have been most interesting to me for the week just ending.
Azure Data Explorer
- (Preview) Configure ADX table retention, caching and batching policies via one-click. This post looks at something I just came across the other day when I edited the lab exercises for my one day Azure Data Explorer training class A Day of Azure Data Explorer. So what the post covers and I found out was how you can configure retention, batching & caching policies for tables & materialized views via one click. Quite cool!
Big Data
- BUILDING THE LAKEHOUSE ARCHITECTURE WITH AZURE SYNAPSE ANALYTICS. As the title says, this post looks at using the full range of Synapse services to build a lake house architecture on top of Azure Databricks Delta Lake tables.
- Orchestrate Operations, Validations, and Approvals on Data Entities with Azure Purview Workflows. Azure Purview is a data governance service from Microsoft that automates the discovery and cataloging of data. It went into General Availability (GA) in October 2021. Recently Microsoft released as a preview Azure Purview Workflows. This article looks into the Purview Workflows and what you can do with them.
Streaming
- Build a real-time data analytics pipeline with Airbyte, Kafka, and Pinot. This post looks at using Airbyte, Kafka and Pinot for building analytics dash boards for user-facing analysis. Very interesting!
- Why ZooKeeper Was Replaced with KRaft – The Log of All Logs. Anyone interested in Kafka probably now knows how ZooKeeper is being replaced by a Raft based quorum protocol. This post explores the rationale behind the replacement, examines why Raft was utilized and altered to become KRaft and describes the new Quorum Controller built on top of KRaft protocols.
~ Finally
That’s all for this week. I hope you enjoy what I did put together. Please comment on this post or ping me if you have ideas for what to cover.
comments powered by Disqus