How significant is struct alignment for GPU performance today? I'm wondering whether layouts such as std140 and std430 are overly conservative, especially when their padding increases buffer sizes. Is the alignment overhead usually negligible, or can tighter packing noticeably improve bandwidth, cache usage, or PCIe transfers? For GPUs without scalar layout support, could calculating offsets manually on the GPU avoid array-stride padding, or would the extra address arithmetic generally cost more than the saved memory?
2 Answers
It depends heavily on the buffer size and what is limiting performance. For small uniform or storage buffers, the padding from std140 or std430 often has little practical impact. With millions of vertices or large arrays, though, extra padding increases memory traffic and can reduce effective bandwidth, so the layout becomes more important.
Memory movement and layout are major factors on virtually every GPU architecture. Tighter packing can save VRAM, reduce transfers over PCIe, and decrease cache pressure, but it is not automatically faster. Misaligned or awkward accesses can require more transactions, reduce coalescing, or add address-calculation work. The best choice depends on the access pattern, and profiling a representative workload is more reliable than assuming a fixed percentage savings will win.
The tradeoff is really whether the memory savings outweigh the possible cache and access penalties. A 30% reduction in storage can matter for large, bandwidth-bound resources, but for a compute-bound workload or one with scattered reads, tighter packing may provide little benefit or even hurt performance.
For hardware without scalar layout, would computing element offsets manually on the GPU be a reasonable way to avoid padded array strides, or is the extra arithmetic usually not worth it?