# Docker Deployment Guide

The persistence system consists of two separate Docker containers:

1. Engine container (writes command logs)
2. Snapshotter container (generates snapshots and manages archival)

These containers must share access to the persistence volume, but can handle archived data separately.

/app/persistence/           # Main persistence directory
└── archived/              # Archive subdirectory

• Engine requires read/write access to /app/persistence
• Snapshotter requires read/write access to both /app/persistence and /app/persistence/archived
• Archived directory can optionally be mounted as a separate volume

• Main persistence volume:

  • Requires high IOPS
  • Should use SSD-backed storage
  • Example: AWS EBS gp3 or io2
  • Size based on expected daily command log generation

• Archive volume:

  • Can use slower, cheaper storage
  • Consider automated lifecycle policies
  • Example: AWS EBS st1 or sc1
  • Size based on retention requirements

volumes:
  persistence:
    driver: ebs  # Example for AWS
    driver_opts:
      type: gp3
      size: 100
      iops: 3000
  archive:
    driver: ebs
    driver_opts:
      type: st1
      size: 500

1. Engine writes command logs to /app/persistence
2. Engine rotates to new command log periodically
3. Snapshotter detects completed command logs
4. Snapshotter generates new snapshot
5. Snapshotter moves older files to /app/persistence/archived

1. Engine implements write-ahead, persisting all commands before processing
2. Persistence volume iops setting will impact latency.
3. Higher iops will enable lower latency through the engine.

Monitor:

• Available space in both volumes
• Rate of command log generation
• Snapshot generation frequency
• Archive directory growth

• Archive volume can be backed up independently
• Consider point-in-time snapshots of the persistence volume
• Implement retention policies for archived data