Phuong Le

This article explains how vmstorage processes incoming metrics, assigns unique IDs to timeseries, and organizes everything into different types of storage parts. The whole system is pretty clever - it uses in-memory buffers for speed, smart compression to save space, and has various watchdogs keeping an eye on things like disk space and data retention.

vminsert acts as a gateway for incoming monitoring data. It receives data in different formats, processes it by parsing and adjusting labels, then uses memory buffers to send this data to storage nodes. It’s smart enough to always send the same type of data to the same storage node and can redirect data if a node isn’t working properly.

The net/rpc package in Go demonstrates basic RPC concepts by establishing TCP connections between clients and servers, using sequence numbers to match requests with responses, and supporting both gob (Go-specific) and JSON codecs for data serialization. While net/rpc is simpler and limited to Go services by default, gRPC offers advanced features like HTTP/2 streaming, cross-language support, and better performance

Through the weak package, you can create these special pointers that automatically become nil when their target memory gets collected. While they’re a bit trickier to use than regular pointers, they’re super useful for things like canonicalization maps and memory-efficient caching. The implementation is pretty clever too, using an 8-byte indirection object to make garbage collection more efficient.

VictoriaMetrics agent, or vmagent, is a lightweight tool designed to gather metrics from a number of different sources. Once it pulls in all those metrics, vmagent lets you ‘design’ them (through ‘relabeling’) or filter them down (doing things like reducing cardinality, stream aggregation, deduplication, and so on) before shipping them off to wherever you want to store them.

Go’s runtime package provides two intriguing features: Finalizers and KeepAlive, which help manage object lifecycle in unique ways. Finalizers let you attach cleanup functions to objects that run when they’re garbage collected. Meanwhile, KeepAlive serves as a tool to prevent premature object collection, especially when dealing with resources that need to stay alive longer than the compiler might expect.

The sync.Once is probably the easiest sync primitive to use, but there’s more under the hood than you might think. It’s also a good opportunity to understand how it works by juggling both atomic operations and mutexes.

Still, we haven’t really covered some other important interfaces, like Closer, Seeker, and a few others. And honestly, if you’re learning Go, you probably don’t want to leave those in the blind spot.

The io.Reader and io.Writer interfaces are probably some of the most common tools. Today, we’re kicking off the I/O series by taking a look at a lot of these readers and writers, and pointing out some common mistakes — like using io.ReadAll in ways that can backfire.

Go’s sync.Map isn’t a magic bullet for all concurrent map needs. It’s got some good tricks up its sleeve, like handling reads without locking, but it’s not always the best choice. This article dives into how sync.Map works under the hood, from its two-map system to the bottom line of expunged entries.