Operations

This section covers operational aspects of running Fluree in production, including configuration, storage backends, monitoring, and administrative operations.

Operation Guides

Scaling and Resilience Architecture

How Fluree’s deployment layers fit together:

Horizontal query scaling with replaceable, cache-warm peers
Independent transaction, query, and indexing capacity
Serverless and scale-to-zero deployment boundaries
Raft write availability within a region
Multi-cloud read distribution and disaster recovery

Configuration

Server configuration options:

Command-line flags
Configuration files
Environment variables
Runtime settings
Tuning parameters

Running with Docker

Configuring the official fluree/server image:

Image internals (entrypoint, volumes, runtime user)
Three configuration approaches: env vars, mounted JSON-LD/TOML config, CLI flags
Common recipes: LRU cache sizing, background indexing, auth, S3+DynamoDB, query peers
Full annotated Docker Compose example
Troubleshooting (volume permissions, RUST_LOG vs FLUREE_LOG_LEVEL, cache auto-sizing under cgroup limits)

Storage Modes

Storage backend options:

Memory storage (development)
File system storage (single server)
AWS S3/DynamoDB (distributed)
IPFS / Kubo (experimental, Rust API only)
Storage selection criteria
Switching between storage modes

Serverless Storage Choices

Cloud/serverless storage placement guidance:

Standard S3 vs S3 Express One Zone for index storage
Why commits should normally remain on Standard S3
Expected transaction, query, and indexing latency ranges
Lambda disk cache and S3 concurrency tuning notes

Hardware sizing: CPU vs disk (benchmark)

A worked benchmark (SPARQLoscope on ~574 M-triple DBLP) showing how hardware maps to performance:

Import is storage-bound — local NVMe imports faster even on slower CPUs
Query latency is CPU-bound — served from cache, tracks single-thread speed
Sizing guidance: when to prioritize disk vs core vs RAM

IPFS Storage

Experimental IPFS storage for programs embedding Fluree through the Rust API:

Kubo node installation and setup
Using FlureeBuilder::build_ipfs, and its in-memory nameservice
Content addressing and CID mapping
Pinning strategies
Operational considerations

DynamoDB Nameservice

DynamoDB-specific setup and configuration:

Table creation (CLI, CloudFormation, Terraform)
Schema reference (v2 attributes)
AWS credentials and permissions
Local development with LocalStack
Production considerations

Telemetry and Logging

Monitoring and observability:

Logging configuration
Metrics collection
Tracing
Health monitoring
Performance metrics
Integration with monitoring systems

Admin, Health, and Stats

Administrative operations:

Health check endpoints
Server statistics
Manual indexing triggers
Backup and restore
Maintenance operations

Query peers and replication

Run the Fluree server as a read-only query peer:

SSE nameservice events (GET /v1/fluree/events)
Peer mode (refresh on stale + write forwarding)
Storage proxy endpoints (/v1/fluree/storage/*) for private-storage deployments

Deployment Patterns

Development

Single process, file storage in .fluree/storage under the working directory (the default):

fluree server run --log-level debug

Single Server Production

File-based storage:

fluree server run \
  --storage-path /var/lib/fluree \
  --listen-addr 0.0.0.0:8090 \
  --log-level info

AWS-backed distributed deployment. S3 buckets, DynamoDB table, and region are set in a JSON-LD connection config file (see Storage Modes and the connection config reference); requires a build with the aws feature:

fluree server run \
  --connection-config /etc/fluree/connection.jsonld \
  --listen-addr 0.0.0.0:8090

Key Configuration Areas

Server Settings

Listen address (host and port)
Request body size limit
Timeout values
CORS (on/off)

TLS is not terminated by the server; put a reverse proxy or load balancer in front of it.

Storage Configuration

Storage mode selection
Storage path (file mode)
AWS credentials (S3 mode)
Cache settings

Indexing Configuration

Novelty thresholds (soft reindex trigger, hard write backpressure)
Stalled-indexing catch-up sweep interval
Index retention (GC)

Security Configuration

Authentication mode per endpoint group (data, events, admin)
Trusted token issuers
Signed request validation
Policy enforcement

The server has no built-in rate limiting; apply it at a reverse proxy or API gateway.

Monitoring

Health Checks

curl http://localhost:8090/health

Response:

{
  "status": "ok",
  "version": "0.1.0"
}

Server Statistics

curl http://localhost:8090/v1/fluree/stats

Response:

{
  "uptime_secs": 3600,
  "storage_type": "file",
  "indexing_enabled": true,
  "cached_ledgers": 3,
  "version": "0.1.0"
}

Metrics Collection

Use GET /v1/fluree/stats for built-in server statistics. Prometheus-style /metrics export is not currently part of the standalone server API.

Operational Tasks

Backup

File storage backup:

# Backup data directory
tar -czf fluree-backup-$(date +%Y%m%d).tar.gz /var/lib/fluree/

AWS storage backup:

# S3 versioning enabled - automatic backups
aws s3 ls s3://fluree-prod-data/ --recursive

# Point-in-time recovery via S3 versions

Restore

File storage restore:

# Stop server
systemctl stop fluree

# Restore backup
tar -xzf fluree-backup-20240122.tar.gz -C /

# Start server
systemctl start fluree

Manual Indexing

Trigger indexing manually:

curl -X POST http://localhost:8090/v1/fluree/reindex \
  -H "Content-Type: application/json" \
  -d '{"ledger": "mydb:main"}'

Compaction

There is no standalone HTTP compaction endpoint. Reindexing rebuilds index artifacts when you need to force a full refresh.

Performance Tuning

Memory Settings

The in-memory cache budget is global (there is no per-query memory setting):

fluree server run -- --cache-max-mb 1024

Indexing Tuning

fluree server run -- \
  --indexing-enabled \
  --reindex-min-bytes 100000 \
  --reindex-max-bytes 1000000

Query Tuning

fluree server run -- \
  --query-timeout-ms 30000 \
  --body-limit 1048576

--body-limit caps every request body (queries and transactions alike). There is no query thread-count setting.

High Availability

Load Balancing

Run multiple Fluree instances behind load balancer:

          ┌─────────────┐
          │   Clients   │
          └──────┬──────┘
                 │
          ┌──────▼──────┐
          │    Load     │
          │  Balancer   │
          └──────┬──────┘
                 │
    ┌────────────┼────────────┐
    │            │            │
┌───▼────┐  ┌───▼────┐  ┌───▼────┐
│Fluree 1│  │Fluree 2│  │Fluree 3│
└───┬────┘  └───┬────┘  └───┬────┘
    │           │           │
    └───────────┼───────────┘
                │
         ┌──────▼──────┐
         │  S3/Dynamo  │
         │  Nameservice│
         └─────────────┘

Failover

Configure health checks in load balancer:

health_check:
  path: /health
  interval: 10s
  timeout: 5s
  healthy_threshold: 2
  unhealthy_threshold: 3

Security Hardening

TLS/SSL

The server listens on plain HTTP only. Terminate TLS at a reverse proxy or load balancer in front of it.

Require Authentication

fluree server run -- \
  --data-auth-mode required \
  --data-auth-trusted-issuer did:key:z6Mk... \
  --admin-auth-mode required \
  --events-auth-mode required

With --data-auth-mode required, data endpoints accept either a Bearer token or a signed request. See Configuration for issuer and audience options.

Development: memory
Single server: file
Production/Distributed: AWS

2. Enable Monitoring

Set up monitoring for:

Health status
Query latency
Transaction rate
Indexing lag
Error rates

3. Regular Backups

Automate backups:

# Daily backup cron
0 2 * * * /usr/local/bin/backup-fluree.sh

4. Capacity Planning

Monitor growth:

Storage usage
Query volume
Transaction rate
Index sizes

5. Security Best Practices

Use TLS in production (terminated at a reverse proxy)
Require authentication
Rate-limit at a reverse proxy or API gateway
Regular security audits

6. Log Management

Rotate logs regularly
Ship logs to centralized system
Set appropriate log levels
Monitor error rates

Configuration - Detailed configuration reference
Storage - Storage backend details
Telemetry - Monitoring and metrics
Admin and Health - Administrative operations
Getting Started: Server - Initial setup

Keyboard shortcuts

Fluree DB