Quick Start

This guide will help you understand Kasho’s architecture and get it running in your environment. While we use Docker Compose as an example, the concepts apply to any production orchestration system like Kubernetes.

Production Deployment Overview

Kasho consists of four required containers that work together:

1. Redis - Used for distributed state management and caching
2. licensing - Validates and manages Kasho licenses
3. pg-change-stream - Captures changes from your source PostgreSQL database
4. pg-translicator - Applies transformations and writes to your target database

Container Images

All Kasho components are packaged in a single Docker image available from Docker Hub:

• Image: kashoio/kasho:latest (or specific version like v0.3.0)
• Commands: ./pg-change-stream and ./pg-translicator

Prerequisites

Before you begin, ensure you have:

• PostgreSQL 15+ databases (source and target)
• Databases configured with wal_level = logical (see below)
• A Redis instance (or Redis-compatible service)
• Container orchestration system (Kubernetes, ECS, Docker, etc.)
• Valid Kasho license file

SHOW wal_level;

wal_level = logical

Docker Compose Example

Step 1: Example Directory Structure

For this example, create a directory structure:

mkdir kasho-example
cd kasho-example

Step 2: Example Docker Compose Configuration

Create a docker-compose.yml file:

services:
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis-data:/data
 
  licensing:
    image: kashoio/kasho:latest
    command: /app/bin/licensing
    ports:
      - "50052:50052"
    environment:
      GRPC_PORT: 50052
    volumes:
      - ./config:/app/config:ro  # Contains license.jwt
 
  pg-change-stream:
    image: kashoio/kasho:latest
    command: /app/bin/pg-change-stream
    environment:
      KV_URL: redis://redis:6379
      PRIMARY_DATABASE_URL: ${PRIMARY_DATABASE_URL}
      LICENSING_SERVICE_ADDR: ${LICENSING_SERVICE_ADDR}
    volumes:
      - ./config:/app/config:ro  # Contains license.jwt
    ports:
      - "50051:50051"
    depends_on:
      - redis
      - licensing
 
  pg-translicator:
    image: kashoio/kasho:latest
    command: /app/bin/pg-translicator
    environment:
      CHANGE_STREAM_SERVICE_ADDR: pg-change-stream:50051
      REPLICA_DATABASE_URL: ${REPLICA_DATABASE_URL}
      LICENSING_SERVICE_ADDR: ${LICENSING_SERVICE_ADDR}
    volumes:
      - ./config:/app/config:ro  # Contains transforms.yml and license.jwt
    depends_on:
      - pg-change-stream
      - licensing
 
volumes:
  redis-data:

Step 3: Configuration Files

Create a config directory for your transforms and license:

mkdir config

Place your license.jwt file in the config directory:

cp /path/to/your/license.jwt config/

Create config/transforms.yml with your transformation rules:

version: v1
tables:
  # Example: Transform user emails in the public.users table
  public.users:
    email: FakeEmail
    phone: FakePhone

See the Transform Configuration guide for all available transforms.

Step 4: Environment Variables

Create a .env file with your database connections and service configuration.

# Source database (read-only access needed)
# Replace with your actual source database connection string
PRIMARY_DATABASE_URL=postgresql://kasho:password@source-host:5432/source_db?sslmode=disable
 
# Target database (write access needed)  
# Replace with your actual target database connection string
REPLICA_DATABASE_URL=postgresql://kasho:password@target-host:5432/target_db?sslmode=disable
 
# Redis connection for pg-change-stream
KV_URL=redis://redis:6379
 
# Change stream service for pg-translicator
CHANGE_STREAM_SERVICE_ADDR=pg-change-stream:50051
 
# License service address
LICENSING_SERVICE_ADDR=licensing:50052

Step 5: Run the Example

docker-compose up -d

Verify all services are running:

docker-compose ps

You should see:

• redis - Running
• licensing - Running on port 50052
• pg-change-stream - Running on port 50051
• pg-translicator - Running

Production Considerations

When deploying to production:

1. Container Orchestration: Translate the Docker Compose configuration to your platform:

   • Kubernetes: Create Deployments, Services, and ConfigMaps
   • ECS: Define Task Definitions and Services
   • Other platforms: Follow your standard container deployment practices

2. Redis: Use a managed Redis service or ensure high availability

3. Networking: Ensure containers can communicate:

   • pg-translicator must reach pg-change-stream service
   • Both services need access to their respective databases
   • Consider service discovery mechanisms in your environment

4. Configuration Management: Mount transform configurations using your platform’s methods:

   • Kubernetes: ConfigMaps or Secrets
   • ECS: Parameter Store or S3
   • Docker: Volume mounts or config management tools

Bootstrap Existing Data

If you have existing data in your source database, you’ll need to run the bootstrap process:

1. Execute the bootstrap process in the pg-change-stream container:

# Run bootstrap process (will prompt for confirmation)
docker exec -it kasho-example-pg-change-stream-1 /app/scripts/bootstrap-kasho.sh

2. The bootstrap process will:
   • Create a consistent snapshot of your source database
   • Start capturing changes from that point
   • Load the snapshot data
   • Transition to streaming mode

If you prefer, you can manually bootstrap things with a bit more effort. See the Bootstrap Process guide for detailed information.

Verify Installation

Check that replication is working:

1. Check pg-change-stream status (using gRPC):

   # Using grpcurl (install with: brew install grpcurl)
   grpcurl -plaintext localhost:50051 kasho.ChangeStreamService/GetStatus

2. View logs:

   docker-compose logs -f pg-change-stream
   docker-compose logs -f pg-translicator

3. Make a test change in your source database:

   -- In your source database
   UPDATE users SET updated_at = NOW() WHERE id = 1;

4. Verify the change appears in your target database

Next Steps

• Configure your databases for production use
• Understand configuration options
• Learn about transforms

Troubleshooting

Services won’t start:

• Ensure your environment variables are set correctly, especially the database URLs.
• Verify transforms.yml exists in the config directory
• Check logs: docker-compose logs [service-name]

Connection errors:

• Ensure databases are accessible from Docker containers
• Verify PostgreSQL has wal_level = logical
• Check firewall rules allow connections