Microsoft patterns & practices

This project focuses on using Apache Storm/Trident with Java. For guidance on using .NET without Storm, see the companion Data Pipeline Guidance.

The two primary concerns of this project are:

• Facilitating cold storage of data for later analytics. That is, translating the chatty stream of events into chunky blobs.

• Demonstrate how to use OpaqueTridentEventHubSpout and Apache Storm/Trident to store Microsoft Azure Eventhub messages to Microsoft Azure Blob exactly-once.

• Architecture Overview
• Getting Started
• Create Java Topology Project eventhub-blobwriter from Scratch
• Design Considerations and Technical How-To

• Performance Result: The performance result will be published once we finishes the performance test.

• Using Zookeeper to store the state: The current sample stores state in Redis Cache. We plan to replace that with Zookeeper.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.