Monitoring with the OpenTelemetry Collector

The OpenTelemetry Collector is the one piece of infrastructure that turns “some instrumented apps” into a monitored landscape. It receives OTLP from your services, scrapes the systems that can’t instrument themselves — brokers, gateways, connectors — and exports everything to Sluicio through a single authenticated pipe:

your apps ────OTLP────▶ ┌──────────────────┐
                        │  OTel Collector  │ ──OTLP/HTTP──▶  Sluicio cell ingest
Kafka / RabbitMQ /      │  receive · scrape │                 (with ingest key)
NATS / Debezium … ──▶   │  batch · export  │
                        └──────────────────┘

This guide takes you from nothing to telemetry in Sluicio: install a Collector, configure receivers, processors and the Sluicio exporter, wire the pipelines per signal, and verify data arrives. If you only need the exporter settings for a Collector you already run, the shorter Point the Collector at Sluicio reference covers just that.

Prerequisites

• A running Sluicio cell and an admin account (to mint an ingest key).
• A host (VM, container platform, or Kubernetes cluster) that can reach both your infrastructure and your cell’s ingest URL.
• Optionally, one or more apps already emitting OTLP — see Instrument a service if you’re starting from zero. The Collector setup below works the same either way; app telemetry just gives the traces and logs pipelines something to carry.

1 · Mint an ingest key

In Sluicio, open Settings → Ingestion, create a key, and copy it — the full value is shown once. The same screen shows your cell’s ingest URL (e.g. https://<your-cell>-ingest.example.com) and a ready-to-paste exporter snippet with both filled in, so you can lift the exporters: block below straight from the app.

Treat the key like a password

Anyone holding it can write telemetry into your organisation. Keep it out of config files and git — the config below reads it from the SLUICIO_INGEST_KEY environment variable. Revoke and reissue from the same screen if it leaks.

2 · Install a Collector

Any of the usual distributions works; what matters is that you run otelcol-contrib — the infra receivers used below (kafkametrics, rabbitmq, prometheus, jmx) ship in the contrib distribution, not the core one.

Container — the quickest start, and what the run step below uses:

docker pull otel/opentelemetry-collector-contrib:latest

Binary / package — grab otelcol-contrib for your platform from the releases page (tarballs plus .deb/.rpm packages that install a systemd service):

curl -sSLO https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.121.0/otelcol-contrib_0.121.0_linux_amd64.tar.gz
tar -xzf otelcol-contrib_0.121.0_linux_amd64.tar.gz otelcol-contrib

Helm — for Kubernetes, the opentelemetry-collector chart; pass the config from step 3 as the chart’s config: value:

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm install otel-collector open-telemetry/opentelemetry-collector \
  --set mode=deployment \
  --set image.repository=otel/opentelemetry-collector-contrib \
  -f values.yaml   # your config + SLUICIO_INGEST_KEY from a Secret

Whichever route you pick, pin a version in production rather than tracking latest.

3 · Configure the Collector

One file describes the whole flow: receivers take telemetry in, processors shape it, the exporter sends it to Sluicio, and pipelines wire the three together per signal. This is a complete, working config — OTLP in from your apps, plus one concrete infra example (an Apache Kafka cluster):

otel-collector-config.yaml

receivers:
  # Your instrumented apps send OTLP here — SDKs default to
  # localhost:4317 (gRPC) / 4318 (HTTP).
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

  # Infra example: scrape a Kafka cluster — brokers, topics and
  # consumer groups (including consumer-group lag).
  kafkametrics:
    brokers: ["kafka-1:9092", "kafka-2:9092"]
    protocol_version: "3.7.0"
    scrapers: [brokers, topics, consumers]
    collection_interval: 60s

processors:
  # First in every pipeline: start refusing data before the
  # Collector runs out of memory, instead of crashing with it.
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20

  # Last in every pipeline: batch before export — easier on the
  # network and on Sluicio's ingest.
  batch:
    send_batch_size: 512
    timeout: 5s

  # Collector-scraped sources carry no service.name of their own —
  # set one so Kafka shows up as a distinct service in Sluicio.
  # (App telemetry already has it from the SDK; don't touch those.)
  resource/kafka:
    attributes:
      - key: service.name
        value: kafka
        action: upsert

exporters:
  otlphttp/sluicio:
    # Your cell's ingest base URL (Settings → Ingestion). The exporter
    # appends /v1/traces, /v1/metrics and /v1/logs itself.
    endpoint: https://<your-cell>-ingest.example.com
    compression: gzip
    headers:
      Authorization: "Bearer ${env:SLUICIO_INGEST_KEY}"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/sluicio]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/sluicio]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/sluicio]
    # Infra sources get their own pipeline so the resource
    # processor names them without touching app telemetry.
    metrics/kafka:
      receivers: [kafkametrics]
      processors: [memory_limiter, resource/kafka, batch]
      exporters: [otlphttp/sluicio]

How the pieces fit:

• One exporter, many pipelines. Every signal — app traces, app metrics, app logs, scraped Kafka metrics — converges on the single otlphttp/sluicio exporter. Sluicio ingests OTLP over HTTP/protobuf only, so the exporter must be otlphttp, not otlp (gRPC). Your apps can still talk gRPC to the Collector; it translates on the way out.
• A pipeline per signal. traces, metrics and logs are separate pipelines by design — a receiver only feeds pipelines of the signals it produces. kafkametrics produces metrics only, hence just a metrics/kafka pipeline.
• A pipeline per infra source. Giving each scraped system its own pipeline (metrics/kafka, later metrics/rabbitmq, …) lets each one get its own resource/<name> processor, so each lands in Sluicio as a distinct service instead of collapsing into one.
• Processor order matters. memory_limiter goes first (protect the Collector before doing any work), batch goes last (batch what’s about to leave).

4 · Run it and send traffic

export SLUICIO_INGEST_KEY="<the key you copied>"

docker run -d --name otel-collector \
  -v "$(pwd)/otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml" \
  -e SLUICIO_INGEST_KEY \
  -p 4317:4317 -p 4318:4318 \
  otel/opentelemetry-collector-contrib:latest

Point your apps’ OTLP exporters at the Collector (http://<collector-host>:4318, or :4317 for gRPC) and generate some traffic. The Kafka scrape starts on its own — one batch of kafka.* metrics per collection_interval.

5 · Verify in Sluicio

Within a few seconds of traffic (and one scrape interval for Kafka):

• Services — each app appears under its service.name, and kafka appears alongside them as its own service.
• Metrics — the catalog lists the scraped metrics: kafka.consumer_group.lag, kafka.partition.current_offset, kafka.topic.partitions, and friends.

Say yes to the monitoring template

Sluicio recognises known systems by their metric-name prefixes — kafka.* here — and offers to apply that system’s monitoring template: starter health checks like consumer-group lag staying under a threshold. Accepting it is the fastest route from “data arrives” to “someone gets paged when it matters”. The same happens for RabbitMQ, Confluent Cloud, NATS, Debezium and the other built-in system types below.

If nothing shows up, check the Collector’s logs (docker logs otel-collector) for the exporter’s HTTP responses — the response-code table in the Collector reference decodes them (401 bad key, 415 wrong encoding, and so on).

6 · Add the rest of your infrastructure

The Kafka block is a pattern, not a special case: add a receiver, a resource/<name> processor, and a metrics/<name> pipeline for each system. Receivers to reach for, per system:

System	Receiver	Metrics that arrive
Apache Kafka	kafkametrics (shown above)	kafka.* — broker count, partition offsets, consumer-group lag
RabbitMQ	rabbitmq — see the full guide	rabbitmq.* — queue depth, consumer counts, message rates
Confluent Kafka / Cloud	prometheus scraping the Confluent Cloud Metrics API export endpoint	confluent_kafka_server_* — retained bytes, consumer lag offsets
NATS	prometheus scraping prometheus-nats-exporter	gnatsd_varz_* — connections, in/out msgs, slow consumers
Debezium	jmx, or prometheus + the JMX exporter	CDC lag, connector connectivity, snapshot progress
Azure Service Bus	community azureservicebus receiver (custom build) — see the full guide	servicebus.* — queue/topic depth, dead-letters, sizes
KrakenD	none needed — KrakenD emits OTLP itself; point it at this Collector — see the full guide	gateway request rates, latencies, error counts

All seven are built-in system types in Sluicio, so the auto-detect-and-template offer from step 5 applies to each.

7 · Monitor the Collector itself

Once the Collector is the artery for all your telemetry, its health is worth a service of its own. The Collector reports internal otelcol_* metrics — expose them on a local Prometheus endpoint and loop them back through a scrape pipeline:

otel-collector-config.yaml (additions)

receivers:
  prometheus/self:
    config:
      scrape_configs:
        - job_name: otel-collector
          scrape_interval: 60s
          static_configs:
            - targets: ["127.0.0.1:8888"]

processors:
  resource/collector:
    attributes:
      - key: service.name
        value: otel-collector
        action: upsert

service:
  telemetry:
    metrics:
      level: normal
      readers:
        - pull:
            exporter:
              prometheus:
                host: 127.0.0.1
                port: 8888
  pipelines:
    metrics/collector:
      receivers: [prometheus/self]
      processors: [memory_limiter, resource/collector, batch]
      exporters: [otlphttp/sluicio]

Sluicio ships an OpenTelemetry Collector system type with starter checks for exactly these metrics: export failures (otelcol_exporter_send_failed_*), sending-queue backlog (otelcol_exporter_queue_size), and dropped spans (otelcol_processor_dropped_spans). Apply the template when Sluicio offers it and the pipe watches itself.

Once you can see what you’re ingesting, you may find you’re ingesting more than you use — an admin can generate a ready-made filter processor snippet from the Reports page (Trim ingestion) to drop unneeded metrics, logs or spans at the Collector, before they ever reach the cell.

Where to go next

• Instrument a service — emit OTLP from your apps (Java, .NET, JavaScript) so the traces and logs pipelines have something to carry.
• Point the Collector at Sluicio — the compact exporter reference: auth headers, signal paths, service.name rules, troubleshooting.
• Track RabbitMQ queue depth — the same receiver-per-system pattern, worked end to end for RabbitMQ.
• What you get from your telemetry — what Sluicio builds from all of this once it arrives.