Introduction
CockroachDB is a distributed SQL database designed for resilience and horizontal scalability. This advanced tutorial covers deploying a multi-region cluster with fine-grained replication, zone configuration, and features like changefeeds. You will learn to ensure high availability while optimizing performance for critical transactional workloads.
Prerequisites
- Docker and Docker Compose v2.20+
- CockroachDB v24.3+
- Strong knowledge of SQL and system administration
- Access to 3 machines or VMs in distinct regions
- TLS certificates ready to use
Create the Docker Compose File
version: '3.8'
services:
roach1:
image: cockroachdb/cockroach:v24.3.0
command: start --certs-dir=/certs --join=roach1,roach2,roach3 --advertise-addr=roach1:26257
volumes:
- ./certs:/certs
- roach1-data:/cockroach/cockroach-data
roach2:
image: cockroachdb/cockroach:v24.3.0
command: start --certs-dir=/certs --join=roach1,roach2,roach3 --advertise-addr=roach2:26257
volumes:
- ./certs:/certs
- roach2-data:/cockroach/cockroach-data
roach3:
image: cockroachdb/cockroach:v24.3.0
command: start --certs-dir=/certs --join=roach1,roach2,roach3 --advertise-addr=roach3:26257
volumes:
- ./certs:/certs
- roach3-data:/cockroach/cockroach-data
volumes:
roach1-data:
roach2-data:
roach3-data:This Compose file defines a three-node cluster with persistent volumes and certificates. Each node uses --join to form the cluster and --advertise-addr for inter-node communication.
Initialize the Cluster
#!/bin/bash
cockroach init --certs-dir=certs --host=roach1:26257
cockroach sql --certs-dir=certs --host=roach1:26257 \
--execute="SET CLUSTER SETTING cluster.organization = 'Learni';
SET CLUSTER SETTING enterprise.license = 'your-license-key';"The init command starts the cluster. The enterprise settings enable advanced features such as changefeeds and encrypted backups.
Configure Replication Zones
ALTER DATABASE defaultdb CONFIGURE ZONE USING
num_replicas = 5,
constraints = '+region=us-east1:2,+region=eu-west1:2,+region=ap-south1:1',
lease_preferences = '[[+region=us-east1], [+region=eu-west1]]';
CREATE TABLE orders (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL,
amount DECIMAL(12,2) NOT NULL
) WITH (zone = 'orders-zone');Zones allow control over replication by region. Here we enforce 5 replicas with lease preference to minimize latency on transactional queries.
Set Up a Changefeed
CREATE CHANGEFEED FOR TABLE orders
INTO 'kafka://kafka:9092'
WITH
updated, resolved = '10s',
format = json,
envelope = row,
diff;
-- Verification
SELECT * FROM [SHOW CHANGEFEED JOBS];The changefeed streams changes in real time to Kafka. The resolved and diff options ensure consistency and allow detection of deletes.
Incremental Backup Script
#!/bin/bash
BACKUP DATABASE defaultdb
TO 's3://cockroach-backups/$(date +%Y%m%d)'
AS OF SYSTEM TIME '-10m'
WITH revision_history, encryption_passphrase = 'strong-passphrase';This incremental backup uses AS OF SYSTEM TIME for consistency and revision_history to enable point-in-time restores.
TLS and Authentication Configuration
#!/bin/bash
mkdir -p certs my-safe-directory
cockroach cert create-ca --certs-dir=certs --ca-key=my-safe-directory/ca.key
cockroach cert create-node --certs-dir=certs --ca-key=my-safe-directory/ca.key roach1 roach2 roach3 localhost 127.0.0.1
cockroach cert create-client root --certs-dir=certs --ca-key=my-safe-directory/ca.keyGenerating node and client certificates is mandatory for a secure cluster. Never expose the CA key.
Best Practices
- Always define lease_preferences to minimize latency
- Use per-table zones rather than database-wide
- Monitor replication metrics with the crdb_internal.zones table
- Enable automatic statistics collection
- Test region-loss scenarios before going to production
Common Mistakes to Avoid
- Forgetting to include certificates in Docker volumes
- Setting an odd num_replicas without region constraints
- Running changefeeds without resolved timestamps
- Ignoring under-replication alerts in the dashboard
Further Reading
Deepen your knowledge of distributed workloads with our advanced CockroachDB training.