Jaeger

Overview

Jaeger is a CNCF graduated tracing backend: it receives spans, stores them and serves a UI and query API. Jaeger v2 (current line, 2.21.0 in September 2026) is built on the OpenTelemetry Collector: one binary, one image (jaegertracing/jaeger), configured only through a YAML file, and it receives OTLP natively. The v1 images (all-in-one, jaeger-collector, jaeger-query) and their SPAN_STORAGE_TYPE / COLLECTOR_OTLP_ENABLED environment variables are gone from the current docs (v1.76 is the archived last line), and the old Jaeger client libraries are deprecated: instrument with OpenTelemetry SDKs. Docs: https://www.jaegertracing.io/docs/latest/.

Instructions

Task A: Deploy

All-in-one with in-memory storage (data is lost on restart), good for a laptop:

docker run --rm --name jaeger -p 16686:16686 -p 4317:4317 -p 4318:4318 -p 5778:5778 -p 9411:9411 \
  cr.jaegertracing.io/jaegertracing/jaeger:2.21.0

UI at http://localhost:16686, OTLP gRPC on 4317, OTLP HTTP on 4318, remote sampling on 5778. Pin a version tag instead of latest. Any other role or storage needs --config /path/config.yaml mounted into the container. The config uses the OpenTelemetry Collector layout: receivers, processors, exporters, service.pipelines, plus Jaeger extensions jaeger_storage (backends), jaeger_query (UI and API) and remote_sampling. Values can use ${env:NAME:-default}; --set receivers.otlp.protocols.grpc.endpoint=0.0.0.0:4317 overrides one key. Inside a container, receivers must listen on 0.0.0.0.

Persistent single-node setup with Badger (development or small installs):

# config.yaml
service:
  extensions: [jaeger_storage, jaeger_query, remote_sampling]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger_storage_exporter]
extensions:
  jaeger_query:
    storage:
      traces: main_store
  jaeger_storage:
    backends:
      main_store:
        badger:
          directories: { keys: /badger/key, values: /badger/data }
          ephemeral: false
          ttl: { spans: 168h }
  remote_sampling:
    file: { path: /etc/jaeger/sampling.json, reload_interval: 30s }
    http: { endpoint: 0.0.0.0:5778 }
receivers:
  otlp:
    protocols:
      grpc: { endpoint: 0.0.0.0:4317 }
      http: { endpoint: 0.0.0.0:4318 }
processors:
  batch:
exporters:
  jaeger_storage_exporter:
    trace_storage: main_store

# docker-compose.yml
services:
  jaeger:
    image: jaegertracing/jaeger:2.21.0
    command: ["--config", "/etc/jaeger/config.yaml"]
    ports: ["16686:16686", "4317:4317", "4318:4318", "5778:5778"]
    volumes:
      - ./config.yaml:/etc/jaeger/config.yaml:ro
      - ./sampling.json:/etc/jaeger/sampling.json:ro
      - jaeger-data:/badger
volumes:
  jaeger-data:

The image runs as uid 10001: a fresh named volume is owned by root and Jaeger fails with mkdir /badger/key: permission denied. Fix once with docker run --rm -v <project>_jaeger-data:/badger alpine:3 chown -R 10001 /badger.

Production storage: Cassandra, Elasticsearch and OpenSearch are the supported distributed backends (the Jaeger team recommends OpenSearch over Cassandra at scale); ClickHouse is experimental behind a feature gate; Kafka is for buffering. Swap the backend block for Elasticsearch:

  jaeger_storage:
    backends:
      main_store:
        elasticsearch:
          server_urls: ["http://elasticsearch:9200"]
          indices:
            index_prefix: "jaeger-main"
            spans: { date_layout: "2006-01-02", rollover_frequency: "day", shards: 3, replicas: 1 }

A second backend under jaeger_query.storage.traces_archive gives an archive store for traces saved manually from the UI. Collector and query are stateless and can be scaled out; for separate roles reuse one config per role.

Task B: Instrument with OpenTelemetry (Python)

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpc opentelemetry-instrumentation-flask opentelemetry-instrumentation-requests

# tracing.py
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.instrumentation.requests import RequestsInstrumentor

def init_tracing(service_name: str):
    resource = Resource.create({
        "service.name": service_name,
        "service.version": "1.2.0",
        "deployment.environment.name": "production",
    })
    provider = TracerProvider(resource=resource)
    provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint="http://jaeger:4317", insecure=True)))
    trace.set_tracer_provider(provider)
    RequestsInstrumentor().instrument()
    return trace.get_tracer(service_name)

# after creating the Flask app: FlaskInstrumentor().instrument_app(app)

Add custom spans with with tracer.start_as_current_span("save_order") as span: span.set_attribute("order.id", order_id). Without code changes, opentelemetry-instrument python app.py with OTEL_SERVICE_NAME=order-service OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger:4317 also works.

Task C: Instrument with OpenTelemetry (Node.js)

npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node @opentelemetry/exporter-trace-otlp-grpc @opentelemetry/resources @opentelemetry/semantic-conventions

// tracing.js: load first, e.g. node --require ./tracing.js server.js
const { NodeSDK } = require('@opentelemetry/sdk-node')
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc')
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node')
const { resourceFromAttributes } = require('@opentelemetry/resources')
const { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION } = require('@opentelemetry/semantic-conventions')

const sdk = new NodeSDK({
  resource: resourceFromAttributes({ [ATTR_SERVICE_NAME]: 'api-gateway', [ATTR_SERVICE_VERSION]: '3.1.0' }),
  traceExporter: new OTLPTraceExporter({ url: 'http://jaeger:4317' }),
  instrumentations: [getNodeAutoInstrumentations({ '@opentelemetry/instrumentation-fs': { enabled: false } })],
})
sdk.start()
process.on('SIGTERM', () => sdk.shutdown())

In OpenTelemetry JS 2.x the Resource class is gone; new Resource(...) throws, use resourceFromAttributes.

Task D: Sampling

Jaeger's remote_sampling extension serves per-service strategies from a file (reloaded on change). With no file, every service gets probabilistic 0.001 (0.1%). ratelimiting works per service but not per operation.

{
  "service_strategies": [
    { "service": "payment-service", "type": "probabilistic", "param": 1.0 },
    { "service": "api-gateway", "type": "ratelimiting", "param": 5 }
  ],
  "default_strategy": {
    "type": "probabilistic",
    "param": 0.1,
    "operation_strategies": [{ "operation": "/health", "type": "probabilistic", "param": 0.001 }]
  }
}

Check what an SDK would receive: curl -s "http://localhost:5778/api/sampling?service=payment-service". SDKs only follow these strategies when configured with a Jaeger remote sampler (for example OTEL_TRACES_SAMPLER=jaeger_remote with OTEL_TRACES_SAMPLER_ARG=endpoint=http://jaeger:5778,pollingIntervalMs=60000,initialSamplingRate=0.25 in SDKs that support it); otherwise set head sampling in the SDK. Tail-based sampling is available through a tail sampling processor in the pipeline, at the cost of memory in Jaeger. An adaptive mode computes rates from traffic.

Task E: Query traces via the API

The stable HTTP API is /api/v3 (OTLP JSON); gRPC jaeger.api_v3.QueryService is on 16685. The /api/* JSON that the UI uses is internal and may change.

curl -s http://localhost:16686/api/v3/services
curl -s "http://localhost:16686/api/v3/operations?service=order-service"
# traces slower than 500 ms in the last hour; the response is OTLP resourceSpans
curl -s "http://localhost:16686/api/v3/traces?query.service_name=order-service&query.start_time_min=$(date -u -d '-1 hour' +%FT%TZ)&query.start_time_max=$(date -u +%FT%TZ)&query.duration_min=500ms" \
  | jq -c '.result.resourceSpans[].scopeSpans[].spans[] | {traceId, name}'
# one trace by its 32-hex id
curl -s http://localhost:16686/api/v3/traces/5b8efff798038103d269b633813fc60c

query.attributes takes URL-encoded JSON such as {"http.status_code":"500"}; query.start_time_min and query.start_time_max are required, and an empty result is HTTP 404 with No traces found.

Examples

Example 1: Try tracing locally in two minutes

User request: "Run Jaeger on my laptop and send one test span so I can see the UI"

docker run -d --rm --name jaeger -p 16686:16686 -p 4318:4318 cr.jaegertracing.io/jaegertracing/jaeger:2.21.0
curl -s -X POST http://localhost:4318/v1/traces -H 'Content-Type: application/json' -d '{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"order-service"}}]},"scopeSpans":[{"spans":[{"traceId":"5b8efff798038103d269b633813fc60c","spanId":"eee19b7ec3c1b174","name":"POST /api/orders","kind":2,"startTimeUnixNano":"1791028939000000000","endTimeUnixNano":"1791028940000000000"}]}]}]}'

Result: the curl prints {"partialSuccess":{}}; http://localhost:16686 lists order-service and the 1-second POST /api/orders trace (pick a recent timestamp, or widen the UI time range). docker stop jaeger removes the container and its data.

Example 2: Find slow checkouts in a Flask service

User request: "Trace my Flask order service and show me which step makes /api/orders slow"

Add Task B's init_tracing("order-service"), wrap validate_order, save_to_database and notify_payment_service in start_as_current_span, call the endpoint, then search order-service with Min Duration 500ms in the UI or use the Task E query.

Result: each trace shows one POST /api/orders root span with three child spans; the widest bar names the slow step (for example notify_payment_service at 480 ms of 520 ms), with order.id attributes to search by.

Guidelines

• Do not copy v1 compose files: jaegertracing/all-in-one, jaeger-collector and jaeger-query images and SPAN_STORAGE_TYPE are v1. v2 reads no storage env vars; use the YAML config.
• Memory storage and Badger are for development; use Elasticsearch/OpenSearch or Cassandra in production and set a retention (7 to 14 days is typical).
• Ports 4317/4318 (OTLP) and 16686 have no authentication by default: bind to localhost or put them behind a proxy or private network.
• Always sample payment and auth paths at 100% and health checks near zero; 0.1% default means a quiet service may show no traces.
• Legacy ports (14250 gRPC, 14268 Thrift HTTP, 6831/6832 UDP) exist only for old Jaeger clients; new code uses OTLP.
• Traces exported through a collector in front of Jaeger work the same way: point the collector's OTLP exporter at jaeger:4317.