◎
Cookie3 · Big Data Solana Transaction Engine
A high-throughput engine that processed transactions from Solana's major decentralized exchanges (DEXs) in near real-time and turned them into usable market signals.
What I used
.NETRustClickHouseRabbitMQDocker
01
What I did
- Designed and maintained a pipeline for on-chain transactions pulled from Solana nodes.
- Parsed and normalized data from different DEXs into a single coherent model.
- Stored and aggregated data in ClickHouse for large-scale analytics.
- Ensured queuing and fault tolerance with RabbitMQ.
02
What I learned
- Designing high-throughput systems and working in a Big Data environment.
- CQRS and Clean Architecture in practice, not just in theory.
- Optimizing writes and queries in the columnar ClickHouse database.
03
Challenges
- A huge, irregular transaction volume required buffering and backpressure.
- Every DEX returned data in a slightly different format — many parsing edge cases.
- Keeping latency low while preserving data completeness.