Development of a standardized approach for evaluating business insights in stream processing systems based on technical metrics - Bashtovyi, Artem; Fechan, Andrii (Technology audit and production reserves, 2025)
The object of research is the benchmarking process of stream processing frameworks, specifically evaluating the impact of Service Level Objectives (SLOs) in real-time data processing systems. One of the most problematic aspects is the lack of standardization in SLO definitions, which leads to inconsistencies between technical performance indicators (latency, throughput) and business objectives. Additionally, existing benchmarking methodologies primarily assess technical metrics without considering their business relevance. In the course of the study, experimental methods were used to analyze the relationship between latency and throughput under varying load conditions. A series of experiments were conducted with a Kafka Streams-based stream processing setup, modifying workload parameters and resource constraints. The results obtained demonstrate the nonlinear relationship between latency and throughput. Increasing event rates can either enhance or degrade performance depending on resource constraints and Kafka Streams' commit interval settings. The findings demonstrate that under stable conditions, latency decreases from 21 s to 6.2 s while throughput increases from 0.6 ops/sec to 72 ops/sec. When computational bottlenecks are introduced, latency spikes to 349 s and throughput drops to 32 ops/sec, highlighting performance degradation. Conversely, distributed processing reduces latency to 11 s and increases throughput to 169.9 ops/sec. While higher loads generally improve throughput, excessive processing delays can unexpectedly reduce it due to resource contention. These insights provide a foundation for dynamic SLO adjustments to optimize real-time data processing efficiency. The presented approach helps to avoid generalized and inefficient methods for measuring the performance of stream processing frameworks.