Introduction
Grafana Tempo is an open-source distributed tracing system designed to store and query traces at scale without sampling or expensive indexing. Unlike Jaeger or Zipkin, Tempo uses object storage (S3, GCS) for minimal costs and infinite scalability.
Why use it in 2026? In a world dominated by microservices and Kubernetes, tracing is essential for debugging distributed latencies and errors. Tempo integrates natively with Grafana for intuitive visual dashboards and supports OpenTelemetry, the universal standard. This beginner tutorial guides you step by step: from Docker Compose installation to generating and visualizing real traces with the HotROD demo app. By the end, you'll master the basics for monitoring your applications. Estimated time: 15 minutes.
Prerequisites
- Docker and Docker Compose installed (version 20+).
- Ports 3000 (Grafana) and 8080 (Tempo) available.
- Basic knowledge of YAML and command line.
- A web browser.
- Optional: MinIO for persistent storage (included in the example).
Create the Docker Compose file
version: "3"
services:
tempo:
image: grafana/tempo:latest
command: [ "-config.file=/etc/tempo.yaml" ]
volumes:
- ./tempo.yaml:/etc/tempo.yaml
- tempo-data:/tmp/tempo
ports:
- "3200:3200" # tempo
- "3200:3200/udp" # tempo gRPC
- "4317:4317" # OTLP gRPC
- "4318:4318" # OTLP HTTP
- "9411:9411" # Zipkin
- "14268:14268" # Jaeger gRPC
- "14250:14250" # Jaeger thrift HTTP
- "8080:8080"
networks:
- tempo
grafana:
image: grafana/grafana:latest
volumes:
- grafana-data:/var/lib/grafana
ports:
- "3000:3000"
networks:
- tempo
environment:
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
- GF_USERS_ALLOW_SIGN_UP=false
- GF_EXPLORE_ENABLED=true
hotrod:
image: grafana/hotrod:latest
ports:
- "8080:8080"
environment:
- OTEL_SERVICE_NAME=hotrod
- OTEL_EXPORTER_OTLP_ENDPOINT=http://tempo:4318
- OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
networks:
- tempo
depends_on:
- tempo
minio:
image: minio/minio:latest
volumes:
- minio-data:/data
- ./minio.yaml:/etc/minio/config.yaml
command: server /data --console-address ":9001"
environment:
- MINIO_ROOT_USER=minioadmin
- MINIO_ROOT_PASSWORD=minioadmin
ports:
- "9000:9000"
- "9001:9001"
networks:
- tempo
volumes:
grafana-data: {}
minio-data: {}
tempo-data: {}
networks:
tempo:
driver: bridgeThis Docker Compose file deploys Tempo, Grafana, the HotROD demo app (instrumented with OTEL), and MinIO for backend storage. HotROD simulates an e-commerce site with automatic tracing. Exposed ports allow direct access. Note: Also create the tempo.yaml file before running.
Launch the stack
Create a project folder, paste the docker-compose.yml above, then run docker compose up -d. Check logs with docker compose logs tempo. The stack is ready in 1-2 minutes: Grafana at http://localhost:3000, Tempo API at http://localhost:8080/ready (should return 'ready'). HotROD at http://localhost:8080.
Analogy: It's like assembling an observability Lego kit—each container is an interconnected brick via the 'tempo' network.
Configure Tempo (YAML file)
server:
http_listen_port: 8080
distributor:
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
jaeger:
protocols:
grpc:
endpoint: 0.0.0.0:14250
thrift_http:
endpoint: 0.0.0.0:14268
zipkin:
endpoint: 0.0.0.0:9411
storage:
trace:
backend: s3
wal:
path: /tmp/tempo/wal
s3:
bucket: tempo
endpoint: minio:9000
access_key: minioadmin
secret_key: minioadmin
insecure: true
minio:
endpoint: "minio:9000"
bucket_name: "tempo"
access_key: "minioadmin"
secret_key: "minioadmin"
insecure: true
querier:
frontend_worker:
frontend_address: 127.0.0.1:9095
log_level: errorThis config enables OTLP/Zipkin/Jaeger receivers for ingesting traces. WAL + S3 storage (via MinIO) ensures persistence without indexing. 'Insecure: true' is for development; in production, use HTTPS and real keys. The querier optimizes queries.
Generate traces with HotROD
Go to http://localhost:8080. Click buttons like 'Checkout', 'Log in', etc., to generate traffic. Each action creates a distributed trace (frontend → DB → payment).
Open Grafana (admin/admin or anonymous) > Explore > Select Tempo as the source (add it via Configuration > Data sources > Tempo, URL: http://tempo:3200). Search by service='hotrod' or traceID. Tip: Traces appear in <1s.
Bash script to generate traffic
#!/bin/bash
for i in {1..10}; do
curl "http://localhost:8080/checkout?item=widget-$i"
curl "http://localhost:8080/login"
echo "Generated batch $i"
done
# Check Tempo ready
curl http://localhost:8080/readyThis script simulates 10 checkouts and logins, generating rich traces. Run chmod +x generate-traces.sh && ./generate-traces.sh. Check 'ready' to confirm Tempo is ingesting. Great for load testing.
Explore traces in Grafana
Steps in Grafana Explore:
- Data source: Tempo.
- Query:
{ .service.name = "hotrod" } | unparam(TraceQL). - Filter by 'duration > 100ms'.
- Click a trace to see the waterfall (span timeline).
Analogy: Like a slow-motion movie of a transaction—each span is an acted-out scene.
Python OTEL instrumentation example
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.resources import Resource
# Config exporter vers Tempo
trace.set_tracer_provider(TracerProvider(resource=Resource.create({"service.name": "python-app"}))
provider = trace.get_tracer_provider()
processor = BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces"))
provider.add_span_processor(processor)
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("main"):
with tracer.start_as_current_span("slow-operation"):
import time
time.sleep(0.5)
print("Trace envoyée à Tempo!")
# pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp-proto-grpcThis Python script creates two spans (main + slow-op) and exports them via OTLP HTTP to Tempo. Run it after pip install .... Check in Grafana with { .service.name = "python-app" }. Perfect for integrating your existing apps.
Advanced TraceQL query
{ .service.name = "hotrod" } | spanKind = "server" and .name =~ "payment" and duration > 100ms | select(span)This TraceQL query filters slow (>100ms) server spans from 'payment' in HotROD. Paste it into Grafana Explore. TraceQL is as powerful as PromQL for traces: regex, math, aggregations.
Best practices
- Always use OpenTelemetry: Vendor-neutral standard, avoids lock-in.
- Persistent storage: Replace MinIO with S3/GCS in production with
insecure: false. - Security: Enable Grafana auth, don't expose all ports.
- Scalability: Deploy in a cluster with multiple ingesters.
- Logs + Traces: Integrate Loki for full observability correlation.
Common errors to avoid
- Forgetting WAL: Without
wal.path, traces are lost on restart. - Misconfigured ports: Check firewall; blocked OTLP 4318 = ghost traces.
- Empty queries: Explicitly add Tempo as a datasource in Grafana.
- No backend: Tempo rejects traces without S3/MinIO configured.
Next steps
- Official docs: Grafana Tempo.
- OpenTelemetry: Get Started with OTEL.
- Kubernetes: Tempo Helm chart.
- Expert Training: Master observability with our Learni courses.