• Sources: primary
  • Summary: Acta takes batches of JSON events with source, timestamp and data fields, buffers them for 10 seconds, writes Parquet files with a Variant data column, and compacts them into 5-minute, hourly, daily and monthly files. DuckDB reads the files through glob imports and uses Parquet footer statistics to skip row groups. A Parquet export of the old OTel traces data came to 1.2 GB, and after cleanup and some dropped data the new files total 65 MB. The service runs under podman on an Asahi Linux box, backs itself up hourly to S3 with age encryption, and was built with DeepSeek V4.1 Flash for about 8 dollars in tokens.
  • Why it matters: It shows a small-server design for wide events where the storage format is open Parquet and the query engine can be swapped, with no database server to run.
  • Background: Parquet is a columnar file format whose footer holds per-column minimum and maximum values, so a query engine can skip data that cannot match. The Variant type stores semi-structured JSON-like values in binary form, and writers can shred common fields into typed columns for compression and fast scans. Compacting many small files into larger ones is the usual way lakehouse systems keep file counts and per-file overhead low.