Open Source Tech

Different Protobuf types dramatically impact your binary size—sint32 encoding needs just 1 byte for negative numbers while int32 wastes 10 bytes, repeated fields can balloon with large tag numbers, and poor type choices can triple your network traffic. Learn the exact patterns to optimize your Protobuf messages for production.

When evaluating, instant vectors provide current snapshots, while range vectors give you multiple values over a period of time. But how do they work?

Metrics come in different types: counters that only increase, gauges that fluctuate, histograms that show value distributions, and summaries that pre-calculate statistics.

FOSDEM (Free and Open Source Software Developers’ European Meeting) is a huge, free, gathering for open-source software enthusiasts that happens every February in Brussels, Belgium. It’s a non-profit event put together by the community, and it’s one of the biggest of its kind. See more in our recap!

Protocol Buffers is faster and smaller than JSON, but the interesting part is understanding why. This article breaks down the encoding techniques that make Protobuf efficient, backed by benchmark results and practical examples

When vmselect asks vmstorage for data, there’s actually quite a bit happening behind the scenes. This article breaks down the process of how vmstorage locates and retrieves your metrics, from finding the right TSIDs to sending back the actual data points.

IndexDB acts as vmstorage’s memory - it remembers which numbers (TSIDs) belong to which metrics, making sure your queries get answered fast. This article walks through how this system works, from the way it organizes data to how it keeps track of millions of timeseries.

HTTP/2 solves head-of-line blocking at the application layer by multiplexing multiple streams over a single TCP connection. While HTTP/1.1 requires requests to be processed sequentially, HTTP/2 allows parallel processing through independent streams, each with its own ID. The Go standard library supports HTTP/2 out of the box when using HTTPS, and with some configuration, it can work over plain HTTP too

vmstorage takes data from vminsert through a concurrency limiter, creates TSIDs for each row, and puts them in memory buffers. Every few seconds, it moves data to storage parts on disk. The system merges parts, removes duplicates, and cleans old data. This turns raw metrics into data users can search.

This article explains how vmstorage processes incoming metrics, assigns unique IDs to timeseries, and organizes everything into different types of storage parts. The whole system is pretty clever - it uses in-memory buffers for speed, smart compression to save space, and has various watchdogs keeping an eye on things like disk space and data retention.