Open Source Tech

HTTP/2 solves head-of-line blocking at the application layer by multiplexing multiple streams over a single TCP connection. While HTTP/1.1 requires requests to be processed sequentially, HTTP/2 allows parallel processing through independent streams, each with its own ID. The Go standard library supports HTTP/2 out of the box when using HTTPS, and with some configuration, it can work over plain HTTP too

vminsert acts as a gateway for incoming monitoring data. It receives data in different formats, processes it by parsing and adjusting labels, then uses memory buffers to send this data to storage nodes. It’s smart enough to always send the same type of data to the same storage node and can redirect data if a node isn’t working properly.

The net/rpc package in Go demonstrates basic RPC concepts by establishing TCP connections between clients and servers, using sequence numbers to match requests with responses, and supporting both gob (Go-specific) and JSON codecs for data serialization. While net/rpc is simpler and limited to Go services by default, gRPC offers advanced features like HTTP/2 streaming, cross-language support, and better performance

This article walks through how data flows from collection to storage, explaining how vmstorage processes incoming metrics, assigns unique IDs to time series, and organizes everything into different types of storage parts. The whole system is pretty clever - it uses in-memory buffers for speed, smart compression to save space, and has various watchdogs keeping an eye on things like disk space and data retention.

Through the weak package, you can create these special pointers that automatically become nil when their target memory gets collected. While they’re a bit trickier to use than regular pointers, they’re super useful for things like canonicalization maps and memory-efficient caching. The implementation is pretty clever too, using an 8-byte indirection object to make garbage collection more efficient.

VictoriaMetrics agent, or vmagent, is a lightweight tool designed to gather metrics from a number of different sources. Once it pulls in all those metrics, vmagent lets you ‘design’ them (through ‘relabeling’) or filter them down (doing things like reducing cardinality, stream aggregation, deduplication, and so on) before shipping them off to wherever you want to store them.

Go’s runtime package provides two intriguing features: Finalizers and KeepAlive, which help manage object lifecycle in unique ways. Finalizers let you attach cleanup functions to objects that run when they’re garbage collected. Meanwhile, KeepAlive serves as a tool to prevent premature object collection, especially when dealing with resources that need to stay alive longer than the compiler might expect.

The sync.Once is probably the easiest sync primitive to use, but there’s more under the hood than you might think. It’s also a good opportunity to understand how it works by juggling both atomic operations and mutexes.

Still, we haven’t really covered some other important interfaces, like Closer, Seeker, and a few others. And honestly, if you’re learning Go, you probably don’t want to leave those in the blind spot.

The io.Reader and io.Writer interfaces are probably some of the most common tools. Today, we’re kicking off the I/O series by taking a look at a lot of these readers and writers, and pointing out some common mistakes — like using io.ReadAll in ways that can backfire.