Go and SIMD: One Codebase, Different Speed Limits

Key takeaways

• SIMD applies one instruction to multiple data elements.
• Portable source code can have different execution costs on different CPUs.
• Memory access and data layout can matter as much as vector width.
• Performance gains need validation on each target CPU and across the whole service.

SIMD offers Go services a tempting bargain: process more data per instruction and spend less time on repetitive calculations. Across a fleet of different CPUs, the payoff depends on which processor runs the code—and how the data reaches it.

What SIMD actually speeds up

Suppose you need to add 10 to every number in an array. You could handle each number individually, or load several numbers together and add to them in one instruction.

That is SIMD, short for Single Instruction, Multiple Data. The group of values being processed is called a vector.

A 128-bit vector can hold four 32-bit integers. An instruction that supports addition on those integers can operate on all four together.

Four additions at once sounds like a fourfold speedup. The rest of the program has other ideas.

Loading values takes time. Storing results takes time. Everything outside that calculation still needs to run. SIMD increases the amount of data an instruction can process within a core; it is distinct from distributing work across multiple CPU cores.

Portable code still has a hardware bill

CPUs differ in the SIMD instructions and vector widths they support. An operation that takes one instruction on one processor may require several on another.

A platform-independent SIMD approach needs something below the application code to absorb those differences. The developer expresses an operation on a group of values, and a compiler or library maps that request to an implementation for the target CPU.

The portability here is source portability. You still build for the target architecture. The goal is to reduce the CPU-specific code developers must maintain.

Choosing the shared interface creates a trade-off. Offer only operations that every target handles easily, and some processors’ stronger capabilities may go unused. Offer more elaborate operations, and some targets may need expensive sequences of instructions to reproduce them.

A common interface can make an operation available everywhere. It cannot make that operation equally cheap everywhere.

Wider vectors need data to work on

Moving from a 128-bit vector to a 256-bit vector doubles the space available for values. Whether that helps depends on the workload.

An array with consecutive values in memory is convenient for loading data in groups. Values scattered around memory, reached by following pointers, can cost more to gather. A wider vector does little good while the processor waits for its contents.

Some workloads already run into a limit on how quickly memory can supply data. Reducing the number of arithmetic instructions may barely change their total runtime.

Input size also matters. For a short array, the overhead of setting up SIMD processing can take a larger share of the work. Arrays that do not divide neatly into vector-sized groups also leave leftover elements to handle.

“Uses SIMD” is therefore an implementation detail, not a performance result. A useful result explains the input size, data layout, and operation being measured.

Put the speedup in the service’s budget

To evaluate a portable SIMD approach in Go, start with an ordinary implementation that performs the same task. Then compare it with CPU-specific optimized code to understand how much performance the shared implementation gives up.

The useful checks are concrete:

• Compare small and large inputs.
• Record the CPU model and supported instruction set.
• Measure both the calculation and the service’s response time.
• Check correctness when the SIMD capability being used is unavailable.

The distinction between a faster function and a faster service is especially easy to lose in an impressive benchmark.

Suppose a calculation accounts for 20% of total execution time. Making it twice as fast reduces that share to 10% of the original runtime. With everything else unchanged, total execution time falls by 10%.

That can be valuable. It is also a much smaller gain than the function’s “2× faster” headline suggests.

A portable SIMD approach could reduce the burden of maintaining separate optimizations for Go services. Whether the last few percent justify CPU-specific code depends on the service you operate. One codebase can simplify maintenance; each CPU still gets a vote on performance.