Understanding Storage I/O Under Load
Download as PPTX, PDF•0 likes•1,038 views
The document presents insights into ScyllaDB, a high-performance NoSQL database designed for distributed, data-intensive applications, emphasizing its capabilities in reducing latency and improving throughput for various companies. It discusses challenges related to storage under load, specifically mixed workloads, and highlights the need for efficient disk management through a tool called Diskplorer. The document concludes with an overview of disk performance analysis and scheduling algorithms relevant to achieving optimal I/O operations.
Understanding Storage I/O Under Load
- 1. Understanding Storage I/O Under Load Avi Kivity, CTO Pavel Emelyanov, Principal Engineer
- 2. Presenter 2 Avi Kivity ● Original maintainer of Linux KVM - Kernel-based Virtual Machine ● Co-maintainer of Seastar, ScyllaDB ● Co-founder of ScyllaDB
- 3. Presenter 3 Pavel Emelyanov ● Ex Linux kernel hacker ● Founder of the CRIU project ● Co-maintainer of Seastar & ScyllaDB
- 4. 4 + For distributed, data-intensive apps that require high performance and low latency + 400+ users worldwide + Results + Comcast: Reduced P99 latencies by 95% + FireEye: 1500% improvement in throughput + Discord: Reduced C* nodes from ~140 to 6 + iFood: 9X cost reduction vs. DynamoDB + Open Source, Enterprise and Cloud options + Fully compatible with Apache Cassandra and Amazon DynamoDB About ScyllaDB 1ms <1ms 10ms 1M 10M ScyllaDB Universe of 400+ Users
- 5. 400+ Companies Use ScyllaDB Seamless experiences across content + devices Fast computation of flight pricing Corporate fleet management Real-time analytics 2,000,000 SKU -commerce management Real-time location tracking for friends/family Video recommendation management IoT for industrial machines Synchronize browser properties for millions Threat intelligence service using JanusGraph Real time fraud detection across 6M transactions/day Uber scale, mission critical chat & messaging app 5 Network security threat detection Power ~50M X1 DVRs with billions of reqs/day Precision healthcare via Edison AI Inventory hub for retail operations Property listings and updates Unified ML feature store across the business Cryptocurrency exchange app Geography-based recommendations Distributed storage for distributed ledger tech Global operations- Avon, Body Shop + more Predictable performance for on sale surges GPS-based exercise tracking
- 7. Current SSDs are Amazing 7
- 8. SSDs are Amazing + 6.4 GB/s read + 3.3 GB/s write + 1M read IOPS + 200k write IOPS + Often, several disks in a single server! 8
- 9. SSDs are Amazing, but not Magic + 6.4 GB/s read + OR 3.3 GB/s write + OR 1M read IOPS + OR 200k write IOPS + Or some kind of mix + But what kind of mix?! 9
- 10. + Online transaction processing (OLTP) + There’s a real user at the other end + Maintenance workloads + Scaling out + Compaction + Backup + Analytics (OLAP) + Want to soak up free bandwidth, but not under a tight deadline + Multi-tenancy + Several OLTP and OLAP workloads on the same disk/data Why mixed workloads? 10
- 11. Prevent one workload from dominating and hurting another workload The Challenge 11
- 13. Introducing Diskplorer + Tool to test disks at a variety of mixed workloads + Open source: https://github.com/scylladb/diskplorer + Python, fio, matplotlib + Fancy graphs + Hours of fun! 13
- 17. GCP n2-standard-16, 8 load SSDs 17
- 18. GCP n2-standard-8, 2TB Persistent Disk 18
- 20. Part III Working with Disks 20
- 21. 21 + Equation for the “safety area” can be complex + Bandwidth and IOPS are not instantly measurable Disk math troubles
- 22. Disk math (step 1) 22
- 23. 23 + Equation for the “safety area” can be complex + Bandwidth and IOPS are not instantly measurable Disk math troubles
- 24. Token bucket algo (and its math) 24
- 25. Disk math (step 2) 25
- 28. 28 + runtime – total CPU time consumed + virtual runtimes – total time normalized by entity “weight” + algorithm selects entity with smallest virtual runtime + long inactive entity “forgets” idle virtual runtime CPU scheduler classes
- 31. United States 2445 Faber St, Suite #200 Palo Alto, CA USA 94303 Israel Maskit 4 Herzliya, Israel 4673304 www.scylladb.com @scylladb Thank You! avi@scylladb.com @avikivity