Skip to main content

Job Performance

Value and Problem Solved​

The "Viewing Performance Metrics" feature provides users with comprehensive insights into the performance of their Flink jobs. It helps users:

  • Identify bottlenecks, data skew, and resource utilization issues.
  • Understand the flow and behavior of data across a jobs tasks and operators.
  • Clorrelate performance metrics to pinpoint issues.

This feature simplifies debugging and optimization by presenting detailed performance metrics, enabling users to react to issues and optimize their pipelines effectively.

Functionality and How It Works​

Overview Boxes​

Summarize key metrics, such as:

  • Records in and Records out of a job.
  • The number of discarded records within the filtered timeframe.

Operator Performance metrics​

Per task / operator, View the different metrics such as:

  • Records In
  • Records Out
  • Discarded Records

Key Features:​

  • Data Chunking: Divides data into 50 equal buckets based on time intervals, ensuring efficient representation of large datasets.
  • Height Scaling: Each bucket's height corresponds to the record count relative to the maximum bucket, while maintaining minimum visibility for smaller buckets.
  • Interaction: Enables users to select, hover over, and explore specific buckets to view records or events related to windows and highlights.

This section enhances the user's ability to analyze record distributions effectively and interactively in complex pipelines. It improves the clarity and interactivity of exploring large-scale stateful operations, particularly in Flink pipelines.

Interaction with the Timeline​

  • Users can drag and drop the timeline touchpoint using a brush to navigate to any time frame within the session.
  • The metrics and overview wboxes will updates accordingly.