Quick Start & Reference

Configuration, deployment, commands, alerting, and log integration. Get your first backup running in under 10 minutes.

⚡  Quick Start

Get your first backup running in under 10 minutes.

📄 Source-available: the source code is provided after contract signing. Contact us to receive your copy.

1. Install

# Extract the archive received after contract signing
cd backup-framework
pip install -r requirements.txt

2. Configure

Copy the example environment file and fill in your credentials:

cp .env.example .env
# Edit .env with your S3 and MongoDB credentials

3. Register a bucket

Register your S3 bucket credentials once. The name is a local alias you choose.

python3 -m backup --register-bucket \
  --name my-bucket \
  --bucket prod-backups \
  --access-key <ACCESS_KEY> \
  --secret-key <SECRET_KEY> \
  --region us-east-1

For a non-AWS endpoint (MinIO, Cloudflare R2, Wasabi…) add --endpoint https://your.endpoint.

4. Register a job

Register a named job that binds a directory to a bucket.

python3 -m backup --register \
  --name daily-backup \
  --dir /path/to/your/data \
  --bucket my-bucket

5. Run your first backup

python3 -m backup --run --job daily-backup

6. Verify

# Check what was backed up
python3 -m backup status --job daily-backup

# Run a full audit (3 checks)
python3 -m backup audit --job daily-backup
💡 Use --dry-run on the first scan to preview what would be uploaded without making any changes.

📋  Requirements

DependencyVersionNotes
Python3.10+3.12+ recommended
MongoDB4.0+Atlas or local; optional for S3-only restore
S3-compatible storageAnyAWS S3, MinIO, Cloudflare R2, Wasabi…
RAM2 GB min4 GB+ recommended for large jobs
CPU2+ coresFor multi-threaded uploads

⚙  Configuration

All configuration is via environment variables (or a .env file). No credentials are ever hardcoded.

VariableDescription
S3_BUCKETTarget S3 bucket name
S3_ACCESS_KEYS3 access key (Fernet-encrypted at rest)
S3_SECRET_KEYS3 secret key (Fernet-encrypted at rest)
S3_ENDPOINT_URLOptional — for non-AWS S3-compatible endpoints
MONGO_URIMongoDB connection string
MONGO_DBMongoDB database name
FERNET_KEYKey used to encrypt stored S3 credentials
LOG_FILEOptional — write JSON logs to a rotating file
BANDWIDTH_LIMITOptional — e.g. 10MB/s, 500KB/s
WORKERSNumber of upload worker threads (default: 4)
COMPRESSION_LEVELgzip level 1–9, or 0 to disable
SECRETS_PROVIDEROptional — aws, azure, gcp, ibm, vault, openbao

Fernet credential encryption

S3 credentials are never stored in plaintext. Generate a key once and store it securely (or pull it from your secrets provider):

python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"

Key rotation is supported via MultiFernet — see the full setup guide for the rotation procedure.

🖱  Deployment

Docker Compose (recommended for production)

# From the project directory received after contract signing
cp .env.example .env  # fill in credentials
docker compose up -d
# Web dashboard at https://localhost
# HTTP automatically redirects to HTTPS

systemd timer

[Unit]
Description=Arkhivio daily backup

[Service]
Type=oneshot
EnvironmentFile=/etc/arkhivio/.env
ExecStart=/usr/bin/python3 -m backup scan upload audit

[Install]
WantedBy=multi-user.target

Cron

# /etc/cron.d/arkhivio  — daily at 02:00
0 2 * * * backup-user /usr/bin/python3 -m backup scan upload audit --job daily

📄  Command Reference

CommandDescription
scanWalk directories, compute CRC32, detect changed files
uploadUpload new/modified files to S3
auditRun 3-check integrity verification
restoreRestore files from S3 (MongoDB-assisted)
restore-s3onlyRestore directly from S3, no MongoDB required
statusShow last run status and metrics
historyList historical runs for a job
reconcileDetect drift between MongoDB and S3
deleteRemove a file from S3 and MongoDB
listList backed-up files for a host/job

Common flags

FlagDescription
--job NAMEJob name (used as MongoDB collection key)
--dir PATH [PATH…]One or more directories to back up
--host NAMEOverride hostname (useful for Kubernetes pods)
--dry-runPreview changes without uploading
--workers NNumber of parallel upload threads
--throttle RATEBandwidth limit, e.g. 10MB/s

📁  Multiple Directories

A single job can back up multiple directories. All paths are tracked under one job, use one set of metrics, and upload in parallel.

python3 -m backup scan upload \
  --job full-server-backup \
  --dir /etc /var/www /home/deploy /opt/app

Global exclude patterns apply across all directories. Existing single-directory jobs require no changes — fully backward compatible.

⚖  Kubernetes

Pod names are ephemeral and change on every restart. Use --host to set a stable identity so all runs for a workload are grouped correctly in MongoDB and S3:

python3 -m backup scan upload audit \
  --job k8s-backup \
  --host my-deployment \
  --dir /data
💡 Set --host to the Deployment or StatefulSet name, not the pod name, so S3 paths stay stable across pod restarts.

🔔  Alerting

Because all events are JSON, any log aggregator can generate alerts by filtering on event field values. No special agent or plugin required.

Critical alerts (page immediately)

EventCondition
upload_errorAny file backup failure
delete_s3_errorS3 delete failure
reconcile_missing_s3> 5 missing objects in 1 hour

Warning alerts

EventCondition
upload_slowdownS3 rate-limiting > 10 times in 1 hour
scan_stat_errorstat() failures > 100 in 1 hour
secrets_load_failedCloud secrets provider unreachable
mongo_tls_disabledTLS disabled for non-localhost MongoDB
(absence)No upload_complete event in > 25 hours

Grafana Loki example rule

- alert: ArkhivioUploadError
  expr: |
    count_over_time(
      {app="arkhivio"} | json | event="upload_error"
      [5m]) > 0
  for: 0m
  labels:
    severity: critical
  annotations:
    summary: Arkhivio upload failure detected

Notification channels

Route alerts to: Slack, Microsoft Teams, PagerDuty, OpsGenie, email, or SMS — using your existing alerting pipeline.

📊  Log Integration

Set LOG_FILE=/var/log/arkhivio/backup.log to write to a rotating file (50 MB per file, 10 files retained) in addition to stdout. Both handlers emit identical newline-delimited JSON.

Supported log destinations

📊  Grafana Loki (Promtail)
⚡  Elastic Stack (Filebeat)
☕  Splunk Universal Forwarder
☁  AWS CloudWatch Logs
⚙  systemd journal
🐙  rsyslog / syslog
💻  Windows Event Log (NSSM)
📈  Prometheus (mtail)

📝  Event Reference

Key events emitted by the framework (all include ts, event, level, job, host):

EventLevelDescription
job_startedinfoJob begun; includes step, host
file_scannedinfoFile fingerprinted; path, size, crc32, status
scan_completeinfoSummary: new, modified, unchanged, deleted counts
file_uploadedinfoS3 PUT succeeded; key, bytes, crc32
upload_errorerrorS3 PUT failed; path, error message
upload_slowdownwarnS3 rate-limit received; will retry after backoff
upload_completeinfoJob summary: uploaded, skipped, errors, bytes
audit_checkinfoSingle check result: check name, status, details
audit_completeinfoAll checks done: passed, total
file_restoredinfoFile downloaded and CRC32 verified
restore_completeinfoSummary: files, bytes, errors, crc32_verified
reconcile_missing_s3errorFile marked synced but absent from S3
secrets_load_failederrorCloud secrets provider unreachable

One-time purchase. No recurring license fees.

No per-seat fees. No per-server royalties. No usage-based billing.

The source code is delivered when you sign the contract.