RavenDB can now seamlessly integrate with existing ecosystems, pushing events into Kafka topics, and talking to the RabbitMQ message broker. In addition, those using Grafana can now pull and visualize data from their RavenDB database.
ETL to Kafka
Kafka is one of the standard solutions for event streaming, used by thousands of companies around the world to build high-performance data pipelines.
With RavenDB 5.4, instead of implementing application code pushing messages and events to Kafka, users can now offload this task to database. Kafka ETL support allows RavenDB to take the role of message producer in a Kafka architecture. This ongoing task will extract data from the database, transform it with a custom script, and load the resulting JSON object to a Kafka destination. Not only will this shorten development time, but users will also be able to rely on this persistent mechanism that implements a retry process to resiliently push their messages to Kafka.
ETL to RabbitMQ
Further supporting the implementation of distributed systems, 5.4 has added Ongoing Tasks that can transform and enqueue documents from RavenDB into RabbitMQ’s queueing infrastructure. Instead of implementing code that would push messages to RabbitMQ, users can rely on a robust and reliable ETL mechanism that will extract documents from their database, transform them with a custom script, and hand them over to RabbitMQ. Not only will this shorten development time, but users will also be able to rely on this persistent mechanism that implements a retry process to resiliently enqueue their messages
With these ETL features, RavenDB opens up a whole new world of architectural patterns and dramatically reduces complexity.
Grafana Data Source Plugin
Grafana is one of the leading open-source solutions for data visualization and observability. Until this version, RavenDB was offering exposure of telemetry data to Grafana via the Telegraf plugin. With RavenDB 5.4, users can now use their RavenDB database as a Data Source for Grafana. With Grafana’s analytics and interactive visualization capabilities, users can understand their data quickly, observe trends and patterns, and make informed decisions.