Honest answers about ALOS DB, its current state, sync modes, concurrency, crash protection, and how it compares to other databases.
ALOS DB is a document-oriented database written in Go. It stores data as MessagePack-encoded documents in append-only shard files, using memory-mapped I/O for reads and a single-writer-per-shard pattern for writes. It is designed to be a lightweight, high-performance alternative to MongoDB and SQLite for applications that need document storage without the overhead of a full database server.
Key design goals:
ALOS DB is not a SQL database. It does not support joins, foreign keys, or complex aggregations. If you need relational features, use PostgreSQL or SQLite.
No. ALOS DB is currently in testing and pre-release. It has not been deployed to production environments handling millions of users. The codebase is actively evolving, APIs may change, and while we perform regular testing, you should not rely on it for critical data without thorough validation in your own environment.
What has been tested:
What has not been tested at scale:
Run your own benchmarks on your hardware before making any deployment decisions. No database is magic; only your measurements matter.
Go is the only officially supported language right now. The server is a single Go binary, and the official client library is also written in Go. The wire protocol uses length-prefixed MessagePack frames over TCP, so in theory any language with TCP and MessagePack support could write a client — but none exist yet.
Planned support:
If you want to write a client in another language (Rust, C#, JavaScript, etc.), the protocol is documented in the Go source. However, there are no stability guarantees for the wire protocol yet, as it may change between pre-release versions.
ALOS DB is proprietary software. It is not open source. You may use it according to the terms provided by the author. For commercial licensing questions, contact the developer directly.
Join the community on Discord: https://alos.gg/discord
alosdbclient.SetClientLogging(true) to diagnose connection issues.ALOS DB can run in two modes, set at startup via the --sync-mode flag. This is the single most important configuration decision:
In async mode, when the client sends a write, the server appends it to an in-memory buffer and immediately returns success. The buffer is flushed to disk periodically by a background worker. If the machine loses power before the flush, the buffered data is gone. This is the same trade-off MongoDB makes with w: 1, j: false or Redis with appendfsync everysec.
In sync mode, the server calls file.Sync() (fsync on Unix, FlushFileBuffers on Windows) on the data file before sending the success response. The client only proceeds once the data is physically on the storage medium. This guarantees that any acknowledged write survives a crash, at the cost of latency.
You cannot mix sync and async mode in the same server instance. If you need both behaviors, run two ALOS DB instances on different ports — one with --sync-mode async and one with --sync-mode sync.
No. The sync mode is set at server startup via --sync-mode and cannot be changed without restarting the server. Switching at runtime would be unsafe: inflight async writes might not be on disk yet, and suddenly demanding fsync for every operation would cause a massive latency spike.
No. "Async" means the server does not wait for fsync, but the operating system still holds the data in its page cache. On a clean shutdown (SIGTERM), the OS flushes everything. Data loss only happens on:
If your server is on a UPS (uninterruptible power supply) with battery-backed RAID cache or enterprise NVMe (which have capacitors to flush in-flight writes), the window for data loss is extremely small. For game telemetry or session state where losing the last 1-2 seconds is acceptable, async mode is fine. For financial transactions, use sync mode.
ALOS DB uses a multi-stage write pipeline:
sync_file_range to hint the kernel to flush dirty pages, without blocking the write path.The write path is designed to be CPU-efficient: documents are serialized to MessagePack once, then written directly to the file with minimal copying.
Here are actual benchmark numbers from the client library over localhost TCP (AMD Ryzen 7 5700X, NVMe SSD, async mode, 16 workers, pool=64):
These numbers will vary dramatically based on:
In sync mode, write throughput is roughly 2-3x lower because the server fsyncs every write. However, the exact multiplier depends on your disk's fsync latency.
These are single-machine, single-client numbers from testing. They are not production benchmarks. Always run your own benchmarks with your document sizes and query patterns.
ALOS DB splits data across multiple independent shards (default 256). Each shard owns:
.db data file.sidx, .sgidx, .svidx)Because each shard is isolated, there is no global lock on the database. A write to shard 42 never blocks a read from shard 7. Within a single shard:
mmap). A reader sees the data file as it existed at the moment the read began. Even if the writer appends new data, existing mmap'd pages remain valid for the reader.The only synchronization is the shard worker channel, which is a buffered Go channel. Writers send; the worker receives. No mutex, no futex, no kernel syscalls for locking.
Most databases use a reader-writer lock (like sync.RWMutex) to protect the B-tree or LSM-tree in memory. When a writer holds the lock, all readers block. Under heavy write load, readers can starve.
ALOS DB avoids this by giving each shard exactly one writer goroutine. All writes for that shard go through that goroutine's channel. Because there is only one writer, there is no need for a lock around the data structure. The writer simply appends to the log and updates the index.
Readers access the mmap'd file directly without any locks. They use the index to find offsets, then read the data. The index is an immutable sorted slice that the worker replaces atomically after a batch. Readers hold a pointer to the old index until they finish; Go's garbage collector cleans it up later.
The result: readers never block writers, and writers never block readers. Only write-write conflicts on the same document require coordination, which is handled by MVCC.
Yes. Reads are fully concurrent and lock-free. You can have hundreds of goroutines reading the same document simultaneously, and they will all see a consistent snapshot without blocking each other or the writer.
Each read gets a reference to the current index snapshot (a pointer to an immutable sorted slice). The document data is read directly from the mmap'd file. Because the file is append-only, old data is never overwritten in place. A reader holding a pointer to offset 1,204,832 will always find valid data there, even if the writer has appended megabytes of new documents after it.
ALOS DB uses Multi-Version Concurrency Control (MVCC). When a transaction begins, it captures the current database timestamp. All reads inside the transaction use that timestamp to look up the index snapshot that existed at that moment.
While the transaction is running:
If two transactions try to write the same document, the second one to commit detects the conflict (the document's version timestamp changed since the transaction began) and rolls back automatically. This is called optimistic concurrency control.
ALOS DB has backpressure mechanisms to prevent unbounded memory growth:
Under sustained load beyond hardware capacity, latency increases gracefully rather than crashing. The server logs warnings when queues exceed 80% capacity.
Sync mode provides the strongest durability guarantee:
.db file.file.Sync() (fsync on Unix, FlushFileBuffers on Windows) is called on the data file. This blocks until the OS confirms the data is on the storage medium.Sync() returns does the server send the success response to the client.If the server crashes between steps 1 and 3, the client never receives an acknowledgement and can retry. If it crashes after step 3, the data is safe. On restart, ALOS DB replays the append-only log from the last known good offset and reconstructs the indexes. Any partial write at the end of the file is detected via length validation and discarded.
Additionally, sync mode computes a CRC32C checksum for every document. If a document's checksum fails on read, it is treated as corrupted and skipped during index rebuild.
Async mode recovery follows the same replay logic, but with a smaller safety window:
.db file from the beginning.The key difference from sync mode is how much data might be at the trailing edge. In async mode, the kernel might have a few seconds of unflushed dirty pages. If power is lost, those pages evaporate. ALOS DB will recover to the last point the kernel actually wrote to disk, which could be slightly behind the last acknowledged write.
This is identical to how MongoDB (without journal) or Redis (with AOF but appendfsync everysec) behaves. It is a deliberate performance trade-off.
Modern storage devices guarantee atomic sector writes (typically 4KB or 512 bytes). ALOS DB structures every document so that its header (length + CRC) fits within a single sector. This means:
For documents larger than the sector size, the data is written as multiple sectors, but the header is always the last sector written. This ensures that if a large document is torn, the header will either be present (document is valid) or absent (document is ignored). You cannot have a valid header with invalid data.
Yes, with --disable-crc. This removes the CRC computation and validation from the hot path. On small documents, this can improve write throughput by a small percentage.
However, you lose protection against:
Only disable CRC if you have error-correcting RAM (ECC) and you have benchmarked that CRC is actually your bottleneck. For most users, CRC overhead is negligible compared to network and disk I/O.
Yes, because ALOS DB files are append-only and never modified in place. You can safely:
cp or rsync while the server is running.tar on a live database.The copied files will be a consistent snapshot of the database at the moment the copy started. Any writes that happen during the copy will not appear in the backup. This is the same principle as PostgreSQL's pg_basebackup.
For portable backups, use the built-in Export and Import APIs, which produce a msgpack stream.
Use MongoDB if you need a mature ecosystem, full aggregation pipelines, GUI tools, or multi-document ACID across sharded clusters.
Use ALOS DB if you want a lightweight, single-binary document store with minimal memory usage and simple deployment. Be aware it is pre-release software.
PostgreSQL is a general-purpose relational database. ALOS DB is a specialized document store. They solve different problems:
Many architectures use both: PostgreSQL for relational data and ALOS DB for high-velocity event streams.
Redis and ALOS DB both prioritize speed, but for different use cases:
If your dataset fits in RAM and you can tolerate losing a few seconds of data on crash, Redis is the better cache. If your dataset is larger than RAM or you need durable document storage, ALOS DB is more appropriate.
A common pattern is Redis as the hot cache in front of ALOS DB, with ALOS DB as the durable source of truth.
SQLite is the gold standard for embedded relational databases. ALOS DB differs in several ways:
Use SQLite for mobile apps, local tools, and small embedded systems. Use ALOS DB when you need a network-accessible database with high concurrency.
ScyllaDB and Cassandra are distributed wide-column stores for multi-datacenter replication. ALOS DB is a single-node document store.
ALOS DB runs well on modest hardware, but for high throughput:
The bottleneck is usually disk I/O in sync mode, or network latency in remote deployments. CPU is rarely the bottleneck unless you are doing complex aggregations.
Several design decisions reduce overhead:
However, these optimizations are specific to ALOS DB's design. A general-purpose database like PostgreSQL or MongoDB does more work per query (query planning, secondary index maintenance, replication logs) because they support more features. Speed is a trade-off against flexibility.
The hot cache is an LRU cache of recently accessed documents, stored in deserialized form. When a read request arrives:
The cache is per-shard. Writes invalidate affected cache entries. The default size is small; tune it with --cache-mb if you have RAM to spare.
For read-heavy workloads where the same documents are accessed repeatedly, the hot cache can significantly improve read latency by avoiding disk access entirely.
Because ALOS DB is append-only, updates and deletes leave stale data behind. Over time, data files grow larger than the live data. Compaction rewrites each shard's data file, keeping only live documents.
Compaction triggers when:
--compact-stale-ratio)--compact-min-size)--compaction is enabledDuring compaction, the shard briefly locks to swap the old file for the new one. Reads continue via the old mmap'd file until the swap. Writes are buffered and applied after compaction.
Compaction runs in the background at low priority. On fast NVMe, a 1GB shard compacts in a few seconds.
Documents are sharded by hashing their _id field. The hash modulo --shards determines which shard owns the document.
You set the shard count at database creation time via --shards. Once data exists, changing the shard count requires a full rebuild (export and re-import) because every document would hash to a different shard.
Guidelines:
Each database is a directory containing subdirectories per shard:
data/
mydb/
shard_000.db # Append-only document log
shard_000.sidx # Primary index (id -> offset)
shard_000.sgidx.0 # Group index segment
shard_000.sgidx.1 # Group index segment
shard_000.sgidx.2 # Group index segment
shard_000.sgidx.3 # Group index segment
shard_000.svidx.0 # Value index segment
shard_000.svidx.1 # Value index segment
shard_001.db
...
.db — The append-only document store. Each document is prefixed by a 4-byte length and a 4-byte CRC32C..sidx — The sorted primary index. Rebuilt on startup from the .db file..sgidx.N — Group indexes for indexed fields. Documents with the same field value are grouped..svidx.N — Value indexes for range queries. Stores sorted field values with document offsets.ALOS DB indexes are fully automatic and zero-config:
_id to disk offset. Built automatically from the .db file on startup.CreateIndex needed. Every field is indexed as soon as it is queried.Indexes are maintained automatically on all writes. You never create, drop, or manage indexes manually.
Queries on indexed fields are O(log n). Unlike MongoDB, ALOS DB does not support compound indexes (multi-field) yet.
In practice, ALOS DB has been tested on a simulated dataset of 16 million Discord message logs with every field automatically indexed. Queries across the entire dataset — including range lookups by timestamp, regex searches within message content, and filtered aggregations — consistently complete in under 15 ms. The auto-indexing system handles all keys without any manual intervention.
Cluster mode enables multi-node replication. When enabled:
--cluster-port.Important limitations:
For most users, a single ALOS DB node with good backups is recommended. Use cluster mode only for experimentation or warm standbys.
TCP connections can drop for many reasons: idle timeouts, NAT evictions, server restarts, or network hiccups. The Go client has automatic reconnection:
To debug disconnections, enable client logging:
import "github.com/guno1928/alosdbclient"
func init() {
alosdbclient.SetClientLogging(true)
}
Common log patterns:
READ_LEN_FAILED err=EOF — Server closed the connection (idle timeout or restart).READ_LEN_FAILED err=i/o timeout — Network latency spike or server overload.CONNECT_FAILED — Server is down or unreachable.By default, the client creates a single TCP connection. For higher throughput, increase the pool size:
db, err := alosdbclient.Connect("localhost:6900",
alosdbclient.WithPoolSize(8),
)
The pool uses round-robin: each request cycles through available connections. This prevents head-of-line blocking where one slow request stalls others.
Each connection maintains its own TCP socket and encryption state. If one drops, only requests routed to it are affected. Dropped connections are re-established automatically on next use.
For most workloads, 4-8 connections is optimal. More than 16 rarely helps and can exhaust server file descriptors.
When credentials are configured, ALOS DB uses a PSK (Pre-Shared Key) handshake instead of TLS. This avoids X.509 certificate parsing overhead:
Every packet is encrypted with AES-256-GCM and a monotonically increasing counter. Replay attacks are impossible because the counter must always increase. The handshake takes ~1ms on modern CPUs.