ARM processors become more popular and more cost-effective according to many benchmarks. One of them was made by Percona for MySQL.
Some of our users reported issues with VictoriaMetrics at AWS Graviton instances. The main concerns were higher CPU and disk IO usage compared to x86 instances of the same size and for the same workload.
By that time, we verified that VictoriaMetrics works fine for raspberry and IoT devices, but didn’t do any optimizations for ARM builds.
The main difference between x86 and ARM builds is the library we use for data encoding. x86-build uses gozstd library, a wrapper over Facebook’s zstd written in C.
Cross-compiled ARM64 build uses compress library by Klaus Post written in native Go.
So in many aspects, the performance difference between x86-build and cross-compiled ARM64 build heavily depends on the performance of these libraries.
That’s why, in order to improve performance of the ARM builds we added CGO build with some tweaks.
For my test I created 3 instances at AWS:
m5.4xlarge
- intel x86 based instances with 16 CPU and 64 RAM $0.768 hour
to run x86 build of VictoriaMetrics.m6g.4xlarge
- graviton2 based instance with 16 CPU and 64 RAM $0.61 hour
to run cross-compiled ARM64 build without CGO.m6g.4xlarge
- graviton2 based instance with 16 CPU and 64 RAM $0.616 hour
to run CGO build for ARM64.And installed vmsingle v1.72.0 on each of the instances.
For workload generation, I’ve used our benchmark suite and set up a separate vmsingle node for metrics collection.
Initial CPU profiling proves this theory and shows performance improvements with gozstd lib:
CPU usage by VictoriaMetrics before optimizations
CPU usage by VictoriaMetrics after optimizations
Optimized ARM and x86 versions show almost the same result for disk IO usage:
Disk writes/reads during the benchmark
Query performance for x86 version outperforms optimized ARM by ~10% and unoptimized ARM by ~25%:
Query latency during the benchmark
One of major challenges was to add CGO build into our cross-compilation pipeline. We are using musl based builds and the default musl compiler isn’t aware of how to build code for ARM. Instead, special aarch64-musl-gcc compiler must be used:
CC=/path_to_folder/bin/aarch64-linux-musl-gcc \
GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go build main.go
Important note, your C-lang dependencies must be built with the same compiler. In my case, I had to rebuild gozstd lib.