Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Troubleshooting

This section helps you diagnose and resolve common issues with Fluree deployments.

Troubleshooting Guides

Common Errors

Reference for frequently encountered errors:

  • Ledger not found
  • Invalid IRI errors
  • Transaction failures
  • Query timeouts
  • Permission errors
  • Storage issues
  • Indexing problems

Debugging Queries

Tools and techniques for query debugging:

  • Using EXPLAIN plans
  • Query tracing
  • Performance profiling
  • Identifying slow queries
  • Optimizing query patterns

Quick Diagnostics

Health Check

First step for any issue:

curl http://localhost:8090/health

Check for unhealthy components.

Server Status

Check overall server state:

curl http://localhost:8090/v1/fluree/stats

Look for:

  • High error counts
  • Active queries/transactions stuck
  • High indexing lag
  • Memory issues

Logs

Check server logs:

# Recent errors
tail -f /var/log/fluree/server.log | grep ERROR

# Recent warnings
tail -f /var/log/fluree/server.log | grep WARN

Common Issue Categories

Connection Issues

Symptoms:

  • Cannot connect to server
  • Connection refused
  • Connection timeout

Common Causes:

  • Server not running
  • Wrong port
  • Firewall blocking
  • Network issues

Quick Checks:

# Is server running?
ps aux | grep fluree-db-server

# Is port listening?
netstat -an | grep 8090

# Can you reach it?
curl http://localhost:8090/health

Query Issues

Symptoms:

  • Queries return no results
  • Queries timeout
  • Unexpected results
  • Query errors

Quick Checks:

# Enable explain
curl -X POST http://localhost:8090/v1/fluree/explain \
  -d '{...}'

# Check server stats
curl http://localhost:8090/v1/fluree/stats

See Debugging Queries.

Transaction Issues

Symptoms:

  • Transactions fail
  • Validation errors
  • Policy denials
  • Slow commits

Quick Checks:

# Validate JSON-LD
# Use online validator: json-ld.org/playground

# Check permissions
curl -X POST http://localhost:8090/v1/fluree/update?dryRun=true \
  -d '{...}'

# Check server stats
curl http://localhost:8090/v1/fluree/stats

Performance Issues

Symptoms:

  • Slow queries
  • Slow transactions
  • High latency
  • Timeouts

Quick Checks:

# Check indexing lag
curl http://localhost:8090/v1/fluree/info/mydb:main | jq '.t - .index.t'

# Check resource usage
curl http://localhost:8090/v1/fluree/stats

# Check active operations
curl http://localhost:8090/v1/fluree/stats

Storage Issues

Symptoms:

  • Cannot write data
  • Storage errors
  • Disk full
  • AWS errors

Quick Checks:

# Check disk space
df -h /var/lib/fluree

# Check AWS connectivity
aws s3 ls s3://fluree-prod-data/

# Check server stats
curl http://localhost:8090/v1/fluree/stats

Error Code Reference

See Common Errors for complete error code reference.

Most Common:

  • LEDGER_NOT_FOUND - Ledger doesn’t exist
  • PARSE_ERROR - Invalid JSON-LD or SPARQL
  • INVALID_IRI - Malformed IRI
  • QUERY_TIMEOUT - Query took too long
  • POLICY_DENIED - Not authorized

Diagnostic Tools

Enable Debug Logging

./fluree-db-server --log-level debug

Runtime log-level changes are not currently exposed through the standalone HTTP API; restart with the desired --log-level or RUST_LOG.

Enable Query Tracing

curl -X POST http://localhost:8090/v1/fluree/query \
  -H "X-Fluree-Trace: true" \
  -d '{...}'

Enable Policy Tracing

curl -X POST http://localhost:8090/v1/fluree/query \
  -H "X-Fluree-Policy-Trace: true" \
  -d '{...}'

Get Query Plan

curl -X POST http://localhost:8090/v1/fluree/explain \
  -d '{...}'

Getting Help

Diagnostic Information to Collect

When reporting issues, include:

  1. Server version:

    curl http://localhost:8090/health

  2. Configuration:

    ./fluree-db-server --help
# Include relevant config values

  3. Error messages:

    • Complete error response
    • Relevant log entries

  4. Reproduction steps:

    • Minimal example to reproduce
    • Sample data if needed

  5. Environment:

    • OS and version
    • Storage mode
    • Available resources (RAM, disk)

Log Collection

Collect diagnostic logs:

# Last 1000 lines
tail -n 1000 /var/log/fluree/server.log > fluree-diagnostic.log

# Specific time range
grep "2024-01-22T10:" /var/log/fluree/server.log > issue-logs.log

Best Practices

1. Check Logs First

Always check logs before deeper investigation:

tail -f /var/log/fluree/server.log

2. Start with Health Check

curl http://localhost:8090/health

3. Isolate the Issue

Test components independently:

  • Can you connect?
  • Can you query?
  • Can you transact?

4. Use Debug Mode Carefully

Debug logging is verbose:

  • Use temporarily
  • Disable in production
  • May impact performance

5. Test on Development

Reproduce on development environment before investigating production.

6. Keep Logs

Retain logs for historical analysis:

# Logrotate config
/var/log/fluree/*.log {
    daily
    rotate 30
    compress
}