What’s new in VictoriaMetrics 2021?

The 2021 year is finished, so it’s time to look at changes VictoriaMetrics has gained during the past year. The first release in 2021 was v1.52.0. The last release in 2021 was v1.71.0. More than 20 new releases of VictoriaMetrics were published during the 2021. The full changelog is available at this page. Let’s look at the most interesting changes.

Querying Graphite data

VictoriaMetrics was able to accept Graphite data from 2019 - see these docs. It provides much lower disk space usage for the ingested Graphite data and uses much lower disk IO compared to the default Graphite database - Whisper. The ingested Graphite data could be queried via MetricsQL - PromQL-compatible query language. While MetricsQL and PromQL are powerful query languages, Graphite users wanted to use Graphite query language directly from VictoriaMetrics, since this allows seamless migration from Graphite to VictoriaMetrics without the need to modify dashboards and alerting rules in Grafana. So we added support for Graphite query language into VictoriaMetrics in the beginning of 2021 - see these docs for details. This allows using VictoriaMetrics as a drop-in replacement for other Graphite backends.

Additionally, we introduced the following Graphite-related features to VictoriaMetrics in 2021:

• __graphite__ pseudo-label in MetricsQL, which allows selecting metrics with Graphite query paths and wildcards. For example, {__graphite__=~"dc*.{appA,appB}.host*.memory"} matches Graphite metrics for the following Graphite wildcard - dc*.{appA,appB}.host*.memory. See these docs for more details.
• label_graphite_group(q, groupNum1, …, groupNumN) function, which allows selecting particular groups from Graphite metric names. See these docs.
• Ability to use Graphite queries for alerting and recording rules in vmalert. See these docs.

Downsampling for the stored data

Downsampling automatically reduces the number of stored raw samples per time series. This may help reducing disk space usage, since lower number of samples are stored to disk. This also may improve performance for queries over long time ranges, since lower number of samples are scanned during the query. VictoriaMetrics gained support for downampling in 2021 - see these docs for details.

Web UI for data exploring and troubleshooting

VictoriaMetrics now supports web UI for data exploring and troubleshooting - see vmui docs. It supports the same set of query args as /graph web UI in Prometheus, so it can be seamlessly used for query troubleshooting when editing graphs in Grafana.

Prometheus-compatible staleness markers

Prometheus-compatible staleness markers fix many edge cases when monitoring highly dynamic environments with frequently changed Prometheus-compatible scrape targets (for example, Kubernetes monitoring). See these docs for details.

Horizontally scalable scraping for Prometheus-compatible targets

vmagent gained the ability to spread scrape targets among multiple vmagent instances. This may be useful if you need to scrape tens of thousands of targets. In this case a single vmagent instance can reach scalability limits, so the solution is to spread the targets among multiple vmagent instances. This is easy to do now according to these instructions.

Backfilling for recording rules in vmalert

vmalert gained support for backfilling historical data for recording rules in 2021. See these docs.

vmctl tool

vmctl tool has been introduced in 2020 in a separate Github project. This tool can be used for migrating data from other monitoring systems to VictoriaMetrics. vmctl has been moved into the main VictoriaMetrics repository in 2021, so it is released together with other VictoriaMetrics components. See vmctl docs for more details.

vmctl tool gained support for data migration from OpenTSDB to VictoriaMetrics in 2021 thanks to John Seekins.

Other interesting features

VictoriaMetrics gained many other features during 2021. Below is a list of the most interesting features:

• All the enterprise components of VictoriaMetrics became available for download and evaluation at releases page and at DockerHub.

• Ability to accept metrics from DataDog agent and DogStatsD.

• Support for data reading and data writing from/to Kafka.

• The official alerting rules for VictoriaMetrics components.

• Many new MetricsQL functions, which were requested by our users - see these docs.

• Ability to set additional labels for all the ingested samples when ingesting data into VictoriaMetrics via supported HTTP-based data ingestion protocols. Additional labels can be set via extra_label query argument. See these docs.

• Ability to set additional label filters to all the queries (MetricsQL and Graphite) via extra_label and extra_filters[] query args. This can be used for label-based multitenancy setup. See these docs.

• Numerous improvements to the official Grafana dashboards for VictoriaMetrics components. See VictoriaMetrics dashboards for Grafana.

• Improved routing rules and load balancing for incoming queries in vmauth. See these docs.

• Official Windows builds for vmagent, vmctl, vmauth and vmalert are now published at releases page.

• Improved service discovery in vmagent for Kubernetes, EC2, GCE and Consul.

• Added Prometheus-compatible service discovery in vmagent for Docker, DockerSwarm, DigitalOcean and http_sd_config.

• Reduced memory usage and improved query performance for typical workloads at VictoriaMetrics. Reduced memory usage for vmagent.

• Official ARM and ARM64 builds for VictoriaMetrics components are now published at releases page and at DockerHub.

• Ability to limit the number of unique time series during data ingestion at vmagent and VictoriaMetrics. See these and these docs.

• Multitenant support in vmalert.

• Multitenant support in vmagent.

• Ability to filter /api/v1/status/tsdb output with arbitrary label filters - see these docs.

• Improved logging for VictoriaMetrics components aimed towards simplified troubleshooting.

• Ability to read Prometheus-compatible scrape configs from multiple files.

• Ability to read file-based configs such as -promscrape.config from http and https urls.

• Web UI in vmalert for alerting and recording rules.

• Improved relabeling - see these docs.

• Ability to scrape targets via http, https or socks5 proxies.

• Automatic switching to read-only mode under low amounts of free disk space.

• Improved compatibility with PromQL.

Conclusion

There were many useful changes in VictoriaMetrics during 2021 thanks to our users and customers. The full changelog is available here. Probably it is time to upgrade VictoriaMetrics to newer versions according to these instructions. The 2022 year should bring more new features and improvements. If you miss some features in VictoriaMetrics, then please file a feature request of vote for already existing features at GitHub project for VictoriaMetrics.