Q1/2022 Release Roundup: Announcing VictoriaMetrics v1.76 & More

Since the beginning of the year, our team has been busy working with the open source community of VictoriaMetrics users and our customers as we continuously enhance and improve Vicky!

Thanks to everyone who has contributed with their feedback, questions, feature requests, bug reports, etc.

We push out a new release every 3-4 weeks, sometimes more, to make sure that the user community and our customers can benefit as quickly as possible from the new features and improvements that are being requested by users and customers alike.

We publish release notes for each new release on GitHub and will now also be publishing a release roundup blog every quarter that presents a summary of the releases published in the previous three months.

This is our first release roundup blog and the past quarter saw some great additions to VictoriaMetrics, including:

• 6 new releases
• 54 new features
• 36 bug fixes
• 23 contributors across 6 releases
• 332 commits

• Multi-level downsampling (the most wanted Vicky feature in 2021)
• Performance for arm64 builds of VictoriaMetrics components improved by up to 15%
• Improve ingesting performance for series with high level of churn (e.g. Kubernetes metrics)
• VictoriaMetrics Cluster (Enterprise version): now includes support for mTLS communications (and Zero-Trust) between cluster components
• Vicky now has its own built-in UI (vmui) for plotting ad hoc graphs. Try it, tell us what you think!

With this release roundup blog post we’re happy to announce the availability of our latest release, VictoriaMetrics v1.76.0.

• We’ve added the ability to verify files obtained via native export to vmctl. See these docs and this feature request.
• New feature in VictoriaMetrics Cluster: reduce memory usage by up to 50% for vminsert and vmstorage under high ingestion rate.
• Added five command-line flags, which can be used for fine-grained limiting of CPU and memory usage during various API calls.

Read the release notes for all the details on the new v1.76 release: https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.76.0

And see below the summaries of all of our Q1/2022 releases:

• VictoriaMetrics Cluster (Enterprise version): now includes support for mTLS communications (and Zero-Trust) between cluster components - this is typically only needed in enterprise environments. See these docs and this feature request for more details on this feature.
• Properly free up memory occupied by deleted cache entries for the following caches: indexdb/dataBlocks, indexdb/indexBlocks, storage/indexBlocks. This should reduce the increased memory usage starting from v1.73.0. See this and this issue for more information.
• vmalert: add ability to use OAuth2 for -datasource.url, -notifier.url and -remoteRead.url. See the corresponding command-line flags containing oauth2 in their names here.
  • And ability to use Bearer Token for -notifier.url via -notifier.bearerToken and -notifier.bearerTokenFile command-line flags. See this issue.

• Support for conditional relabeling via if filter
• Improve performance when registering new time series. See this issue. Thanks to @ahfuzhang
• How to push data from vmagent to Kafka: reuse Kafka clients when pushing data from many tenants to #Kafka. See the details here: https://docs.victoriametrics.com/vmagent.html#writing-metrics-to-kafka

• Allow overriding default limits for the following in-memory caches, which usually occupy the most memory:
  • storage/tsid - the cache speeds up lookups of internal metric ids by metric_name{labels...} during data ingestion. The size for this cache can be tuned with -storage.cacheSizeStorageTSID command-line flag.
  • indexdb/dataBlocks - the cache speeds up data lookups in <-storageDataPath>/indexdb files. The size for this cache can be tuned with -storage.cacheSizeIndexDBDataBlocks command-line flag.
  • indexdb/indexBlocks - the cache speeds up index lookups in <-storageDataPath>/indexdb files. The size for this cache can be tuned with -storage.cacheSizeIndexDBIndexBlocks command-line flag. See also cache tuning docs. See this issue.
• Add -influxDBLabel command-line flag for overriding db label name for the data imported into VictoriaMetrics via InfluxDB line protocol. Thanks to @johnatannvmd for the pull request.
• Return X-Influxdb-Version HTTP header in responses to InfluxDB write requests. This is needed for some InfluxDB clients. See this comment and this issue.

Amongst other things, we’re now publishing VictoriaMetrics binaries for MacOS amd64 & MacOS arm64 (aka MacBook M1) in our release notes.

From this release onwards, performance for arm64 builds of VictoriaMetrics components improved by up to 15%!

Further Highlights include:

• Reduce CPU and disk IO usage during indexdb rotation once per -retentionPeriod. See this issue.
• VictoriaMetrics Cluster: add -dropSamplesOnOverload command-line flag for vminsert. If this flag is set, then vminsert drops incoming data if the destination vmstorage is temporarily unavailable or cannot keep up with the ingestion rate. The number of dropped rows can be monitored via vm_rpc_rows_dropped_on_overload_total metric at vminsert.
• VictoriaMetrics Cluster: improve re-routing logic, so it re-routes incoming data more evenly if some of vmstorage nodes are temporarily unavailable and/or accept data at slower rate than other vmstorage nodes. Also significantly reduce possible re-routing storm when vminsert runs with -disableRerouting=false command-line flag. This should help the following issues: one, two, three, four, five.
• MetricsQL: cover more cases with the label filters’ propagation optimization. This should improve the average performance for practical queries.
• MetricsQL: optimize joining with *_info labels. For example: kube_pod_created{namespace="prod"} * on (uid) group_left(node) kube_pod_info now automatically adds the needed filters on uid label to kube_pod_info before selecting series for the right side of * operation. This may save CPU, RAM and disk IO resources. See this article for details on *_info labels. See this issue.

• Extended support for @ modifier from PromQL, which is enabled by default in Prometheus starting from Prometheus v2.33.0
  • It can contain arbitrary expression
  • It can be put anywhere in the query
• Ability to keep the original metric names when calculating arbitrary MetricsQL and PromQL functions. For example, rate(foo) keep_metric_names .
• Improved UX in vmui based on user requests (thank you!)

And as reminder, some feature highlights from release v1.71.0:

• Multi-level downsampling (the most wanted Vicky feature in 2021)
• Ability to compare two queries on the same graph in vmui
• Ability to read configs from http and https urls in vmagent

This sums up our release roundup for the beginning of 2022!

You can find all our release notes here: https://github.com/VictoriaMetrics/VictoriaMetrics/releases

If you need support, please visit our support page: https://victoriametrics.com/support/

And if you’re interested in finding out more about our Enterprise offering, please visit: https://victoriametrics.com/products/enterprise/