Shadow Replica API

The shadow-replica.js module provides redundant backup distribution for sealed identity backups. It uploads encrypted backups to external storage and generates recovery manifests.

Overview

┌─────────────┐     ┌──────────────────┐     ┌─────────────────┐
│   Sealed    │     │  Shadow Replica  │     │ Storage Provider│
│   Backup    │────►│                  │────►│                 │
│   (.json)   │     │  • Validation    │     │ • Catbox (now)  │
└─────────────┘     │  • Checksums     │     │ • IPFS (future) │
                    │  • Upload        │     │ • S3 (future)   │
                    └────────┬─────────┘     └─────────────────┘
                             │
                             ▼
                    ┌──────────────────┐
                    │ Recovery Manifest│
                    │                  │
                    │ • Remote URL     │
                    │ • SHA256 hash    │
                    │ • Fetch steps    │
                    └──────────────────┘

CLI Reference

Basic Usage

node src/shadow-replica.js [options]

Options

Option	Required	Default	Description
`--in <file>`	No	Newest in backups/	Sealed backup to upload
`--backups-dir <dir>`	No	`backups/`	Directory containing backups
`--provider <name>`	No	`catbox`	Storage provider
`--out-manifest <file>`	No	Auto-generated	Output manifest path
`--dry-run`	No	false	Skip actual upload

Examples

# Upload newest backup in backups/ directory
node src/shadow-replica.js

# Upload specific backup
node src/shadow-replica.js --in backups/my-identity.json

# Dry run (no network)
node src/shadow-replica.js --in backups/my-identity.json --dry-run

# Custom manifest location
node src/shadow-replica.js \
  --in backups/my-identity.json \
  --out-manifest manifests/recovery.json

Output

Shadow Replica complete.
Input: backups/my-identity-sealed.json
SHA256: a1b2c3d4e5f6...
Bytes: 1234
Provider: catbox
Remote: https://files.catbox.moe/abc123.json
Manifest: backups/recovery_manifest_2026-02-06T12-00-00-000Z.json

Recovery Manifest

Format

interface RecoveryManifest {
  format: "ShadowReplica/RecoveryManifest";
  formatVersion: 1;
  createdAt: string;  // ISO 8601 timestamp

  artifact: {
    type: "IdentitySentinel/SealedBackup";
    localFile: string;       // Relative path to original
    bytes: number;           // File size
    sha256: string;          // SHA-256 hash
    sealedHeader: {          // Copy of sealed backup header
      format: string;
      formatVersion: number;
      createdAt: string;
      cipher: object;
      kdf: object;
      saltB64: string;
      ivB64: string;
      contentType: string;
    };
  };

  storage: {
    provider: string;        // Storage provider name
    url: string;             // Remote URL
    deleteToken: string | null;  // For deletion (if supported)
  };

  recovery: {
    fetch: {
      method: "GET";
      url: string;           // Same as storage.url
    };
    verify: {
      sha256: string;        // Expected hash
      bytes: number;         // Expected size
    };
    warning: string;         // Security reminder
  };
}

Example Manifest

{
  "format": "ShadowReplica/RecoveryManifest",
  "formatVersion": 1,
  "createdAt": "2026-02-06T12:00:00.000Z",

  "artifact": {
    "type": "IdentitySentinel/SealedBackup",
    "localFile": "projects/sovereign-agent/backups/my-identity-sealed.json",
    "bytes": 1234,
    "sha256": "a1b2c3d4e5f6789...",
    "sealedHeader": {
      "format": "IdentitySentinel/SealedBackup",
      "formatVersion": 1,
      "createdAt": "2026-02-06T11:00:00.000Z",
      "cipher": { "name": "aes-256-gcm" },
      "kdf": { "name": "scrypt", "N": 32768, "r": 8, "p": 1, "keyLen": 32 },
      "saltB64": "...",
      "ivB64": "...",
      "contentType": "application/json; charset=utf-8"
    }
  },

  "storage": {
    "provider": "catbox",
    "url": "https://files.catbox.moe/abc123.json",
    "deleteToken": null
  },

  "recovery": {
    "fetch": {
      "method": "GET",
      "url": "https://files.catbox.moe/abc123.json"
    },
    "verify": {
      "sha256": "a1b2c3d4e5f6789...",
      "bytes": 1234
    },
    "warning": "This manifest intentionally contains NO master key material. Preserve your Master Key separately; without it, the sealed backup remains unrecoverable."
  }
}

Storage Providers

Catbox (Default)

Anonymous file hosting service.

Property	Value
URL	`https://catbox.moe`
Max file size	200 MB
Retention	Indefinite (for files accessed regularly)
Authentication	None required
Delete support	No

Usage:

node src/shadow-replica.js --provider catbox

Future Providers (Planned)

Provider	Status	Notes
IPFS	Planned	Content-addressed, distributed
S3	Planned	AWS S3 compatible
Backblaze B2	Planned	Low-cost cloud storage
Storj	Planned	Decentralized storage

Recovery Process

Step-by-Step Recovery

# 1. Locate recovery manifest
cat backups/recovery_manifest_*.json | jq '.storage.url'

# 2. Fetch remote backup
curl -o recovered-backup.json "$(cat manifest.json | jq -r '.storage.url')"

# 3. Verify integrity
EXPECTED_HASH=$(cat manifest.json | jq -r '.recovery.verify.sha256')
ACTUAL_HASH=$(sha256sum recovered-backup.json | cut -d' ' -f1)

if [ "$EXPECTED_HASH" = "$ACTUAL_HASH" ]; then
  echo "Integrity verified ✓"
else
  echo "INTEGRITY CHECK FAILED!"
  exit 1
fi

# 4. Verify can unseal
node src/identity-sentinel-core.js verify \
  --key "$IDENTITY_KEY" \
  --in recovered-backup.json

# 5. Unseal if needed
node src/identity-sentinel-core.js unseal \
  --key "$IDENTITY_KEY" \
  --in recovered-backup.json

Automated Recovery Script

const fs = require('fs');
const crypto = require('crypto');

async function recoverFromManifest(manifestPath) {
  // 1. Read manifest
  const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8'));

  // 2. Fetch remote backup
  const response = await fetch(manifest.storage.url);
  if (!response.ok) {
    throw new Error(`Fetch failed: ${response.status}`);
  }
  const data = await response.arrayBuffer();
  const buffer = Buffer.from(data);

  // 3. Verify integrity
  const actualHash = crypto.createHash('sha256').update(buffer).digest('hex');
  if (actualHash !== manifest.recovery.verify.sha256) {
    throw new Error('Integrity check failed');
  }

  // 4. Verify size
  if (buffer.length !== manifest.recovery.verify.bytes) {
    throw new Error('Size mismatch');
  }

  // 5. Return recovered backup
  return JSON.parse(buffer.toString('utf8'));
}

// Usage
const backup = await recoverFromManifest('backups/recovery_manifest_*.json');

Security Considerations

What Shadow Replica Protects

Threat	Protection
Local disk failure	Remote copy exists
Accidental deletion	Remote copy preserved
Single provider outage	Multiple providers (future)

What Shadow Replica Does NOT Protect

Threat	Why Not
Key compromise	Backup is encrypted, but key is separate
Manifest tampering	Verify hash before using
Provider disappearance	Use multiple providers

Best Practices

# ✅ Upload to multiple providers
node src/shadow-replica.js --provider catbox
node src/shadow-replica.js --provider ipfs  # when available

# ✅ Store manifests in multiple locations
cp backups/recovery_manifest_*.json ~/Dropbox/
cp backups/recovery_manifest_*.json /mnt/usb/

# ✅ Periodically verify remote backups
for manifest in backups/recovery_manifest_*.json; do
  URL=$(jq -r '.storage.url' "$manifest")
  HASH=$(jq -r '.recovery.verify.sha256' "$manifest")
  ACTUAL=$(curl -s "$URL" | sha256sum | cut -d' ' -f1)
  if [ "$HASH" = "$ACTUAL" ]; then
    echo "✓ $manifest"
  else
    echo "✗ $manifest - INTEGRITY FAILED"
  fi
done

Validation

Shadow Replica validates sealed backups before upload:

Validation Checks

Check	Error if Failed
File exists	`Input file not found`
Valid JSON	`Sealed Backup parse failed`
Has header	`Sealed Backup missing header`
Correct format	`Unexpected Sealed Backup format`
Has ciphertext	`Sealed Backup missing ciphertextB64`
Has auth tag	`Sealed Backup missing authTagB64`

Validation Code

function validateSealedBackupObject(obj) {
  if (!obj || typeof obj !== "object") {
    throw new Error("Sealed Backup must be an object");
  }

  const h = obj.header;
  if (!h || typeof h !== "object") {
    throw new Error("Sealed Backup missing header");
  }

  if (h.format !== "IdentitySentinel/SealedBackup") {
    throw new Error(`Unexpected Sealed Backup format: ${h.format}`);
  }

  if (typeof obj.ciphertextB64 !== "string") {
    throw new Error("Sealed Backup missing ciphertextB64");
  }

  if (typeof obj.authTagB64 !== "string") {
    throw new Error("Sealed Backup missing authTagB64");
  }

  return h;
}

Error Reference

Error	Cause	Solution
`Input file not found`	File doesn’t exist	Check `--in` path
`No .json backups found`	Empty backups directory	Create a sealed backup first
`Sealed Backup parse failed`	Invalid JSON	Check file is valid JSON
`Sealed Backup missing header`	Not a sealed backup	Use Identity Sentinel output
`Unexpected Sealed Backup format`	Wrong file type	Use a sealed backup file
`Catbox upload failed`	Network/service error	Retry or check connectivity
`Unsupported provider`	Unknown provider name	Use supported provider