System types · Built-in All levels

Confluent Kafka

The built-in Confluent Kafka system type — scraping the Confluent Cloud Metrics API export endpoint, detection prefixes, starter checks for consumer lag, cluster load and hot partitions, and tuning notes.

SLSluicio team 5 min read Updated Jul 2026

Confluent Cloud is managed Kafka — the brokers are Confluent’s problem, but the usage is still yours: consumers lagging, a Dedicated cluster running out of headroom, one partition taking disproportionate ingress. Confluent publishes exactly these signals through its Metrics API, and this system type turns them into starter checks.

Key: confluent-kafka · applied automatically to services emitting metrics prefixed confluent_kafka_ or confluent.kafka..

Getting the telemetry

The Confluent Cloud Metrics API has a Prometheus-format export endpoint — scrape it with the Collector’s prometheus receiver using a Cloud API key:

receivers:
  prometheus/confluent:
    config:
      scrape_configs:
        - job_name: confluent-cloud
          scrape_interval: 60s
          scheme: https
          metrics_path: /v2/metrics/cloud/export
          params:
            resource.kafka.id: ["lkc-xxxxxx"]   # your cluster id(s)
          static_configs:
            - targets: ["api.telemetry.confluent.cloud"]
          basic_auth:
            username: ${env:CONFLUENT_API_KEY}
            password: ${env:CONFLUENT_API_SECRET}

Wire the receiver into a metrics pipeline with a resource processor setting a service.name, exactly as in Monitoring with the OpenTelemetry Collector.

What Sluicio detects

Metrics prefixed confluent_kafka_ or confluent.kafka.. Scraped from the export endpoint, names arrive in the underscored Prometheus form — confluent_kafka_server_consumer_lag_offsets, confluent_kafka_server_cluster_load_percent, confluent_kafka_server_hot_partition_ingress, and so on; the dotted prefix covers setups that translate the names to OTel conventions.

Starter checks

Check	Condition	Severity	Why it matters
Consumer lag	`confluent_kafka_server_consumer_lag_offsets` > 1,000, split by `consumer_group_id`	Warning	A group is falling behind its topics — same signal as self-hosted Kafka lag, served by Confluent.
Cluster load	Cluster load ratio > 0.8	Warning	On Dedicated clusters, load approaching 1.0 means you’re out of CKU headroom — expect throttling; time to expand.
Hot partition ingress	Confluent reports a partition as hot on ingress	Warning	One partition absorbing outsized traffic — usually a skewed partition key — throttles that partition while the rest idle.

Tuning notes

Cluster load only exists on Dedicated clusters. On Basic and Standard clusters the metric isn’t reported and the check simply never fires — disable it there to keep the check list honest. 0.8 is a reasonable “start planning” line; how close to 1.0 you can cruise depends on how bursty your traffic is.
Consumer lag carries the same advice as the Apache Kafka type: tune per group against real throughput, and add a duration. Note the export endpoint’s granularity — with one sample per minute, short spikes are invisible anyway.
Hot partition alerts point at your partition keys, not at Confluent. The fix is usually rekeying or increasing partition count on the affected topic.
The export endpoint returns the latest complete minute per metric; keep scrape_interval at 60s — scraping faster only re-reads the same sample.