Date: 05/01/2022

RabbitMQ & Dataflows

A RabbitMQ library of .NET tools to help rapidly develop well performant services or to just help manage durable connectivity with the RabbitMQ.Client!


Updated: 05/03/2024
Test Server: Windows 11 Pro (v10.0.22631 [Build 22631])
RabbitMQ Server: v3.13.2
Erlang: v26.2.5

Release build
Codacy Badge1
Codacy Badge2

Why Make A RabbitMQ Powered Dataflow?

Dataflows have configurable concurrency, serialization, monitoring, compression, and encryption all as first class citizens. This paradigm allows developers to just focus on the important stuff - getting work done!

Here are some features ready with RabbitMQ.Dataflows.


  • Pipelines (v1) are light weight workflow-esque function-as-step executor.
    • All steps process in the order provided allowing you to still control order of execution.
    • Configurable parallelism and concurrency built-in.
    • No automatic OpenTelemetry.
    • No automatic exception catching/async error handling (thrown exceptions kill Pipelines).
  • Dataflows (v2) are the more complex, yet complete, workflow class.
    • All steps process in the order provided allowing you to still control order of execution.
    • Configurable parallelism and concurrency built-in.
    • Async processing, batch processing, consumer cloning and more!
    • Async error handling (simplify functional error handling by allowing functions to throw).
    • AutoPublish (to the next Queue) functionality built-in.
    • OpenTelemetry with native distributed tracing for Publish/Consumer.


  • A RabbitMQ IConnectionPool and IChannelPool (connection durability).
  • Supports ILogger<T> via LogHelpers static singleton.
  • Contracted IWorkState simplifies functional generic returns and integration.
  • Has ISerializationProvider support for System.Text.Json and MessagePack.
    • Easy to write your own providers.
  • Has ICompressionProvider built-in support for Gzip, Deflate, Brotli, and LZ4.
  • Has IEncryptionProvider built-in support for AesGcm and BouncyCastle AesGcm.
  • RecyclableMemoryStream options for ICompressionProvider and IEncryptionProvider.
  • Publishers AutoPublish seamless encryption/decryption steps (adds headers too).
  • Consumers allow seamless compression/decompression steps (uses headers).
  • Publisher/Consumers have built-in OpenTelemetry support.


  • All built-in steps will have integration tests that should remove concerns from end-user developer.
  • Future case will include much more complex abstract UnitTesting as time allows.
  • The developer should only need to unit test their functional business code.

Guides, Code, and Examples

More to come and visual aids can be found towards the bottom!

You can also find various library examples inside the tests/UnitTests or the tests/RabbitMQ.Console.Test project.

Main RabbitMQ Library



A library that focuses on RabbitMQ connection and channel management to create fault tolerant Publishers and Consumers.
Formerly called CookedRabbit.Core/Tesseract.

DataFlow Library



A library that provides the base magic Dataflows for RabbitMQ.Dataflows.

  • Custom TPL Block - ChannelBock used as a Channel-based BufferBlock<TIn>
  • Has DataFlowEngine and ChannelBlockEngine.
  • Has Pipelines (Dataflow alternative).

Core Productivity Libraries

These libraries are here to help you build powerful Dataflows for your messages.



A library that has a collection of .NET ISerializationProvider or the interface to make your own.

  • Supports MessagePack and System.Text.Json and Newtonsoft.Json.



A library that has a collection of .NET ICompressionProvider or the interface to make your own.

  • Supports LZ4, Gzip, Brotli, and Deflate.
  • Supports RecyclableMemoryStream variants.



A library that focuses on implementing hashing.

  • Supports Argon2.



A library that provides encryption contracts and the base AesGCM/AesCBC .NET IEncryptionProvider as well as the interface to make your own.

  • Supports AesCbc via CryptoStream (good for encrypted file/memorystreams).
  • Supports .NET AesGcm 128, 192, 256 (non-streams).
  • Supports BouncyCastle AesGcm 128/192/256.
  • Supports RecyclableMemoryStream variants.



A library that focuses on general purpose re-usable functions and extensions that simplify the coding experience.

Non-Critical Library Integrations



A library that provides the provides helper classes for data manipulation and transformation.

Also provides Database abstractions, a simple Dapper integration, and SqlKata integration SQL query generation.

Database Connection Factory Support

  • System.Data.SqlClient
  • Microsoft.Data.SqlClient
  • MySql.Data.MySqlClient
  • Npgsq
  • MySql.Data
  • Oracle
  • SQLite

Visual Aids

I have included a high level Excalidraw file with screenshots below. Be sure to open the file though for better quality and more up-to-date information as I will continue enhancing it over time. The goal isn't to have everything in here, but I myself understand the flow of a library with the addition of a visual aid. There is nothing truly special about this library, it's just the basics, but there are a lot of moving parts when you step back.

RabbitService - Start Life Cycle

Rabbit Service Start

Consumer - Start Life Cycle

Rabbit Service Consumer Start

ConsumerDataflow - Data Flow

Consumer Dataflow Flow


Have an issue? Make an Edit and Pull Request here :)