This week: Dive into the dynamic world of AI/ML and streaming technology. Explore Ray, the groundbreaking compute runtime, and the rise of RAG-Fusion in information retrieval.
Delve into the pursuit of Artificial General Intelligence (AGI) in 2023 and discover how data streaming empowers Elemental Cognition’s AI platform.
Plus, get ready to master Apache Flink with an essential resource. Stay updated and informed with our insights into the tech landscape.
AI/ML
- Ray: The Next Generation Compute Runtime for ML Applications. Ray is an open-source unified compute framework that makes scaling AI and Python workloads easy — from reinforcement learning to deep learning to tuning and model serving. This InfoQ presentation introduces Ray’s basic API and architectural concepts and dives deeper into some of its innovative ML use cases.
- Forget RAG, the Future is RAG-Fusion. This blog-post discusses the emergence of Retrieval Augmented Generation (RAG) technology and its impact on information retrieval. The post acknowledges the benefits of RAG but also points out some challenges: constraints with current search technologies, human search inefficiencies, and oversimplification of search. To address these issues, the blog-post introduces a concept called RAG-Fusion. RAG-Fusion aims to bridge the gap between user queries and their actual intent. It does this by generating multiple user queries and reranking the search results using techniques like Reciprocal Rank Fusion (RRF).
- How Close Are We to AGI?. The blog-post linked to discusses the rapid pace of technological advancements in 2023 and the increasing interest in Artificial General Intelligence (AGI). AGI is defined as a system that can perform intellectual tasks as humans do without assistance. The post highlights research and development efforts to progress toward AGI, including deep learning and reinforcement learning. The blog-post concludes by emphasizing the need to wait for future technological advancements to gauge our proximity to AGI.
Streaming
- How Data Streaming Powers an AI Platform for When You Can’t Afford to Be Wrong. This blog-post discusses how Elemental Cognition (EC) uses data streaming powered by Confluent Cloud to enhance its AI platform for critical decision-making. EC combines large language models (LLMs) with custom domain models to deliver provably correct results in complex scenarios, overcoming the shortcomings of LLMs alone, such as hallucinations and limited problem-solving capabilities. The platform, represented by products like Cogent and Cora, ensures that customers can rely on accurate and optimal solutions by exposing its reasoning and data sources. Confluent Cloud helps EC maintain data provenance, scale, and resilience, enabling real-time AI reasoning across various industries, including life sciences and travel planning, while ensuring reliability and performance.
- Apache Flink. For those interested in Kafka and following what happens in the Kafka ecosystem, you know that Confluent is going all in on Flink. That means that now is a good time to learn Flink. The link here will help you with that. It is not a blog-post as such, but it links to a series of blog-posts by Robin Moffatt on Apache Flink. The series is a great introduction to Apache Flink and covers topics like: Learning Apache Flink S01E01: Where Do I Start?, Learning Apache Flink S01E02: What is Flink?, Learning Apache Flink S01E03: Running my First Flink Cluster and Application. This is a must read if you want to learn Flink.
~ Finally
That’s all for this week. I hope you enjoy what I did put together. Please comment on this post or ping me if you have ideas for what to cover.
comments powered by Disqus