TLDR
Assumptions​
-
- Every Kafka config is set to its default, unless mentioned explicitly in these pages
- All traffic is assumed to be evenly distributed
- Replication Factor is set to 3
- Brokers are evenly split across availability zones
-
- AKalculator uses the US East region in all clouds
- AWS:
us-east
(N. Virginia) - GCP:
us-east1
- Azure:
East US
(not East US 2)
- AWS:
- AKalculator uses the US East region in all clouds
-
- 1 vCPU allows a broker to handle 5 MiB/s Produce and 15 MiB/s Consume (at the same time)
-
- 75% of consume traffic reads from the tail of the log
- all tail reads are served from pagecache and do not touch the disk
- 25% of consume traffic is historical
- historical reads hit the disk when tiered storage is disabled, and do not touch the disk when tiered storage is enabled
- 75% of consume traffic reads from the tail of the log
-
- Write/Read IOPS are all assumed as 128 KiB per IO
-
- configures a 60 minute
local.retention.ms
- uses SSDs
- assumes zero compacted topics
- configures a 60 minute
-
- The calculator assumes that the cluster has 1000 partitions
-
- max disk size is 16 TiB
- max allowed disk usage is 50% (i.e, 50% free space capacity is kept)
-
- clusters are between
3<->200
brokers in size
- clusters are between