|
|
|||||||
|
Copyright © 2014 bizmodeller ltd - iHomeServer Release 3.1.46.0 - iHomeServer is a trademark of bizmodeller ltd
|
files( namespace: "archive" predicate: "mime_type = application/x-parquet" sort: "size DESC" limit: 100 ) file_id version content_hash metadata retention_days, owner
with filebase.transaction(isolation="SERIALIZABLE") as txn: f = txn.get("spt://abc123/raw/sensor.bin") f.metadata["processed_at"] = now() f.content = new_data txn.commit() # creates new version atomically The index is built on a reverse index from metadata predicates to file versions . Supported index types: spt filebase
1. Abstract The SPT Filebase is a specialized, high-performance data storage and retrieval architecture designed for managing structured, semi-structured, and unstructured file assets within an SPT (Single Point of Truth or Stream-Process-Transform) ecosystem. Unlike traditional file systems or blob stores, the SPT Filebase implements a dual-hash content addressing scheme, transactional versioning, and a predicate-based query engine over file metadata. This document explores its internal design, indexing mechanics, concurrency model, and fault-tolerance guarantees. 2. Core Architecture An SPT Filebase instance is composed of four logical layers: Unlike traditional file systems or blob stores, the
| Layer | Function | |-------|----------| | | Handles file writes, chunking, deduplication, and initial hash computation. | | Index Layer | Maintains a persistent LSM (Log-Structured Merge) tree over metadata tags, file IDs, and temporal markers. | | Storage Layer | A content-addressable block store (CABS) with optional erasure coding and tiered compression (Zstandard, LZ4, or LZMA). | | Query Layer | Exposes a SQL-like or GraphQL interface for predicate-based file discovery and retrieval. | 3. File Identity & Addressing Every file in the SPT Filebase is identified by a triple: Core Architecture An SPT Filebase instance is composed