Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Tests

This guide covers testing practices, test organization, and how to run tests in the Fluree codebase.

Test Organization

Unit Tests

Tests in the same file as code:

#![allow(unused)]
fn main() {
// src/query.rs
pub fn execute_query(query: &Query) -> Result<Vec<Solution>> {
    // Implementation
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_execute_query() {
        let query = Query::parse("SELECT ?s WHERE { ?s ?p ?o }").unwrap();
        let results = execute_query(&query).unwrap();
        assert!(!results.is_empty());
    }
}
}

Integration Tests

Tests in tests/ directory:

#![allow(unused)]
fn main() {
// tests/integration_test.rs
use fluree_db_api::{Dataset, query};

#[test]
fn test_query_workflow() {
    let dataset = Dataset::new_memory();
    
    // Insert data
    dataset.transact(test_data()).unwrap();
    
    // Query data
    let results = query(&dataset, test_query()).unwrap();
    
    // Verify
    assert_eq!(results.len(), 5);
}
}

Grouped integration binaries (fluree-db-api, fluree-db-server)

Every top-level tests/*.rs file is a separate test binary that links the entire crate dependency graph. With ~160 integration files that meant ~160 full link steps per cargo test — the dominant cost of building the test suite.

fluree-db-api therefore groups its integration files into a handful of binaries by domain (grp_query, grp_policy, grp_import, grp_index, grp_transact, grp_ledger, grp_graphsource, grp_reasoning, grp_misc, …). The crate sets autotests = false and declares each binary explicitly via [[test]] in Cargo.toml. Each case file is included as a module:

#![allow(unused)]
fn main() {
// tests/grp_query.rs
#[path = "support/mod.rs"]
mod support;            // shared harness, compiled once per binary

#[path = "it_query_jsonld.rs"]
mod it_query_jsonld;
#[path = "it_query_sparql.rs"]
mod it_query_sparql;
// …
}

Within a case file, refer to the shared harness as crate::support (not mod support;). Each file becomes its own module, so item names never collide across files.

To add a new integration test: create tests/it_<name>.rs, then add a #[path = "it_<name>.rs"] mod it_<name>; line to the appropriate grp_*.rs binary. No new [[test]] entry is needed unless you are creating a new group.

Kept standalone (their own [[test]] binary, not grouped):

  • Feature-gated suites (credential, iceberg, aws-testcontainers, vector) — declared with required-features.
  • Tests that mutate process-global env vars (e.g. FLUREE_* toggles). A binary runs its tests as threads in one process under plain cargo test, so a mutating test must be alone in its binary: moving a whole file out of a group is not enough, because the mutation still leaks to every sibling test in that file. it_sync_graph_scan_backstop holds one test for exactly this reason; the rest of the graph-sync suite stays in grp_ledger. (Under cargo nextest, every test already runs in its own process, so this only matters for bare cargo test.)
  • Tests whose assertions are about instrumentation — presence/absence of tracing events captured via a thread-local span_capture subscriber (e.g. it_cyclic_bgp_probe, it_minmax_fast_path_fired, it_upsert_skip_fires). Tracing’s callsite-interest cache is process-global and the capture is thread-local, so these assertions are only reliable when the test is alone in its process. (Same caveat as above: bites bare cargo test only.)

fluree-db-server uses the same pattern and the same standalone rules, with harnesses named for its domains (grp_http, grp_query, grp_proxy, grp_policy, grp_search, grp_raft, grp_bolt). It has no shared support/ module, so its case files are self-contained; telemetry_test and multi_query_tracing_integration are kept standalone under the instrumentation-assertion rule above.

Because autotests = false means an unwired file silently never compiles and never runs — with cargo test still reporting success — both crates carry a tests/harness_coverage.rs that asserts every tests/*.rs is either declared as a [[test]] target or reachable from one through #[path = "…"] mod x; or plain mod x; lines. It derives reachability from Cargo.toml outwards rather than from file names, so an undeclared grp_*.rs is flagged as an orphan itself instead of being assumed to be a target. Only those two line-anchored forms are followed; a file included any other way is reported as an orphan rather than missed, and the fix is to wire it in with #[path].

The other two standalone rules — env mutation and instrumentation assertions — are not mechanically enforced. Detecting them means scanning Rust source for call shapes, which costs more in false positives and blind spots than review does. They are conventions; honour them when adding a test.

The check lives in the fluree-test-support crate, so adopting it in a third crate is a dev-dependency plus a tests/harness_coverage.rs calling assert_every_test_file_is_reachable(env!("CARGO_MANIFEST_DIR")), declared as its own [[test]] target. env! expands at the call site, so each crate is checked against its own manifest.

Example Tests

Tests in examples/:

// examples/basic_query.rs
fn main() -> Result<()> {
    let dataset = Dataset::new_memory();
    dataset.transact(sample_data())?;
    let results = dataset.query(sample_query())?;
    println!("Results: {:?}", results);
    Ok(())
}

Run with:

cargo run --example basic_query

Running Tests

All Tests

cargo test --all

Opt-in LocalStack (S3/DynamoDB) tests

Some AWS/S3 tests are intentionally opt-in and will not run during typical cargo test runs. They require Docker and start LocalStack automatically.

cargo test -p fluree-db-connection --features aws-testcontainers --test aws_testcontainers_test -- --nocapture

Specific Crate

cargo test -p fluree-db-query

Specific Test

cargo test test_query_execution

With Output

cargo test -- --nocapture

Integration Tests Only

cargo test --test '*'

Doc Tests

cargo test --doc

With Nextest (Faster)

cargo nextest run

Writing Tests

Unit Test Example

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_parse_simple_query() {
        let input = r#"{"select": ["?s"], "where": [{"@id": "?s"}]}"#;
        let query = parse_query(input).unwrap();
        
        assert_eq!(query.select_vars.len(), 1);
        assert_eq!(query.where_patterns.len(), 1);
    }

    #[test]
    fn test_parse_invalid_query() {
        let input = "invalid json";
        let result = parse_query(input);
        
        assert!(result.is_err());
        assert!(matches!(result.unwrap_err(), ParseError::InvalidJson));
    }
}
}

Integration Test Example

#![allow(unused)]
fn main() {
// tests/it_query.rs
use fluree_db_api::*;

#[tokio::test]
async fn test_basic_query() {
    // Setup
    let dataset = Dataset::new_memory().await.unwrap();
    
    // Insert test data
    let txn = r#"{
        "@context": {"ex": "http://example.org/ns/"},
        "@graph": [{"@id": "ex:alice", "ex:name": "Alice"}]
    }"#;
    dataset.transact(txn).await.unwrap();
    
    // Execute query
    let query = r#"{
        "from": "test:main",
        "select": ["?name"],
        "where": [{"@id": "?s", "ex:name": "?name"}]
    }"#;
    let results = dataset.query(query).await.unwrap();
    
    // Verify
    assert_eq!(results.len(), 1);
    assert_eq!(results[0]["name"], "Alice");
}
}

Async Tests

Use tokio test runtime:

#![allow(unused)]
fn main() {
#[tokio::test]
async fn test_async_operation() {
    let result = async_function().await.unwrap();
    assert_eq!(result, expected);
}
}

Property-Based Tests

Use proptest for property-based testing:

#![allow(unused)]
fn main() {
use proptest::prelude::*;

proptest! {
    #[test]
    fn test_parse_roundtrip(s in "\\PC*") {
        let iri = Iri::parse(&s)?;
        let serialized = iri.to_string();
        let reparsed = Iri::parse(&serialized)?;
        assert_eq!(iri, reparsed);
    }
}
}

Test Fixtures

Test Data

Create reusable test data:

#![allow(unused)]
fn main() {
// tests/fixtures/mod.rs
pub fn sample_person_data() -> &'static str {
    r#"{
        "@context": {"schema": "http://schema.org/"},
        "@graph": [
            {"@id": "ex:alice", "@type": "schema:Person", "schema:name": "Alice"},
            {"@id": "ex:bob", "@type": "schema:Person", "schema:name": "Bob"}
        ]
    }"#
}

pub fn sample_query() -> &'static str {
    r#"{
        "select": ["?name"],
        "where": [{"@id": "?p", "schema:name": "?name"}]
    }"#
}
}

Use in tests:

#![allow(unused)]
fn main() {
#[test]
fn test_with_fixtures() {
    let dataset = Dataset::new_memory();
    dataset.transact(fixtures::sample_person_data()).unwrap();
    let results = dataset.query(fixtures::sample_query()).unwrap();
    assert_eq!(results.len(), 2);
}
}

Test Helpers

#![allow(unused)]
fn main() {
// tests/helpers/mod.rs
pub async fn setup_test_dataset() -> Dataset {
    let dataset = Dataset::new_memory().await.unwrap();
    dataset.transact(sample_data()).await.unwrap();
    dataset
}

pub fn assert_query_results(results: &[Solution], expected: &[(&str, &str)]) {
    assert_eq!(results.len(), expected.len());
    for (result, (var, value)) in results.iter().zip(expected) {
        assert_eq!(result.get(var).unwrap().to_string(), *value);
    }
}
}

Test Categories

Fast Tests

Quick unit tests:

#![allow(unused)]
fn main() {
#[test]
fn test_fast_operation() {
    // < 1ms execution
}
}

Slow Tests

Tests that take longer:

#![allow(unused)]
fn main() {
#[test]
#[ignore]  // Ignored by default
fn test_slow_operation() {
    // > 1s execution
}
}

Run slow tests:

cargo test -- --ignored

Integration Tests

End-to-end workflows:

#![allow(unused)]
fn main() {
// tests/it_full_workflow.rs
#[tokio::test]
async fn test_complete_workflow() {
    let dataset = setup_test_dataset().await;
    
    // Multiple operations
    transact_initial_data(&dataset).await;
    query_and_verify(&dataset).await;
    update_data(&dataset).await;
    query_history(&dataset).await;
}
}

Benchmarking

Criterion Benchmarks

Create benchmarks:

#![allow(unused)]
fn main() {
// benches/query_bench.rs
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use fluree_db_query::*;

fn benchmark_query_execution(c: &mut Criterion) {
    let dataset = setup_benchmark_dataset();
    let query = parse_query(QUERY).unwrap();
    
    c.bench_function("query execution", |b| {
        b.iter(|| {
            execute_query(black_box(&dataset), black_box(&query))
        });
    });
}

criterion_group!(benches, benchmark_query_execution);
criterion_main!(benches);
}

Run benchmarks:

cargo bench

Comparison Benchmarks

Compare different approaches:

#![allow(unused)]
fn main() {
fn benchmark_approaches(c: &mut Criterion) {
    let mut group = c.benchmark_group("approach_comparison");
    
    group.bench_function("approach_1", |b| {
        b.iter(|| approach_1(black_box(&data)))
    });
    
    group.bench_function("approach_2", |b| {
        b.iter(|| approach_2(black_box(&data)))
    });
    
    group.finish();
}
}

Continuous Integration

GitHub Actions

Tests run automatically on:

  • Pull requests
  • Commits to main
  • Scheduled (nightly)

Workflow: .github/workflows/test.yml

name: Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
      - run: cargo test --all
      - run: cargo clippy -- -D warnings
      - run: cargo fmt -- --check

Pre-commit Checks

Run before committing:

#!/bin/bash
# .git/hooks/pre-commit

cargo fmt --check || exit 1
cargo clippy -- -D warnings || exit 1
cargo test --all || exit 1

Make executable:

chmod +x .git/hooks/pre-commit

Test Best Practices

1. Test One Thing

Each test should verify one behavior:

Good:

#![allow(unused)]
fn main() {
#[test]
fn test_query_returns_correct_count() {
    let results = query(&dataset, &query).unwrap();
    assert_eq!(results.len(), 5);
}

#[test]
fn test_query_returns_correct_values() {
    let results = query(&dataset, &query).unwrap();
    assert_eq!(results[0]["name"], "Alice");
}
}

Bad:

#![allow(unused)]
fn main() {
#[test]
fn test_query() {
    let results = query(&dataset, &query).unwrap();
    assert_eq!(results.len(), 5);
    assert_eq!(results[0]["name"], "Alice");
    assert_eq!(results[1]["name"], "Bob");
    // Too many assertions
}
}

2. Use Descriptive Names

#![allow(unused)]
fn main() {
#[test]
fn test_query_with_filter_returns_only_matching_results() {
    // Clear what's being tested
}
}

3. Arrange-Act-Assert

Structure tests clearly:

#![allow(unused)]
fn main() {
#[test]
fn test_example() {
    // Arrange: Setup
    let dataset = setup_test_dataset();
    let query = parse_query(TEST_QUERY);
    
    // Act: Execute
    let results = execute_query(&dataset, &query).unwrap();
    
    // Assert: Verify
    assert_eq!(results.len(), 3);
}
}

4. Test Error Cases

#![allow(unused)]
fn main() {
#[test]
fn test_invalid_query_returns_error() {
    let result = parse_query("invalid");
    assert!(result.is_err());
}

#[tokio::test]
async fn test_missing_ledger_returns_ledger_not_found() {
    let result = fluree.ledger("nonexistent:main").await;
    assert!(matches!(result.unwrap_err(), Error::LedgerNotFound(_)));
}
}

5. Avoid Flaky Tests

Don’t depend on:

  • Timing
  • Random values (use seeded RNG)
  • External services
  • File system state

6. Clean Up Resources

#![allow(unused)]
fn main() {
#[test]
fn test_with_temp_file() {
    let temp_dir = tempfile::tempdir().unwrap();
    let file_path = temp_dir.path().join("test.db");
    
    // Test with file_path
    
    // temp_dir automatically cleaned up
}
}

7. Use Test Utilities

#![allow(unused)]
fn main() {
// tests/common/mod.rs
pub fn assert_solution_contains(solutions: &[Solution], var: &str, value: &str) {
    let found = solutions.iter().any(|s| {
        s.get(var).map(|v| v.to_string() == value).unwrap_or(false)
    });
    assert!(found, "Expected to find {}={} in results", var, value);
}
}

W3C SPARQL Compliance Tests

The testsuite-sparql crate runs official W3C SPARQL test cases against Fluree’s parser and query engine. Tests are discovered automatically from W3C manifest files — zero hand-written test cases.

# Run all W3C SPARQL tests
cargo test -p testsuite-sparql

# Run with verbose output
cargo test -p testsuite-sparql -- --nocapture 2>&1

The suite covers SPARQL 1.0 and 1.1 syntax tests (293 tests) plus query evaluation tests across 12 categories (233 tests). Eval tests are #[ignore]’d by default — run with --include-ignored or via make test-eval in testsuite-sparql/.

For the full guide on interpreting results, debugging failures, and contributing fixes, see the W3C SPARQL Compliance Suite guide.

Test Coverage

Generate Coverage Report

Using tarpaulin:

cargo install cargo-tarpaulin

cargo tarpaulin --out Html --output-dir coverage/

View: coverage/index.html

Coverage Goals

  • Core functionality: 90%+ coverage
  • Edge cases: Tested
  • Error paths: Tested
  • Public APIs: 100% covered