Repository navigation
Reproducibility guarantees and data alignments #6216
Description
There appears to be conflicting info throughout GH around expected numerical reproducibility of BLAS functions on the same data, particularly around pointer alignments, and no explicit mention of reproducibility guarantees in the docs.
One of the kernels for ASUM has this note:
OpenBLAS/kernel/x86_64/dasum_microk_skylakex-2.c
Lines 25 to 29 in 3561e5e
Deliberately no peeling to an alignment boundary: peeling makes the
grouping of the sum into accumulators - and hence the rounding of the
result - depend on the buffer address. Unaligned loads keep the result
a function of n alone, and cost the same as aligned loads when the
address happens to be aligned.
.. which IMO would make sense even if there was some performance penalty from unaligned loads, but then other kernels for the same ASUM function do the peeling anyway:
OpenBLAS/kernel/x86_64/dasum_microk_haswell-2.c
Lines 23 to 25 in 3561e5e
In the GH discussion from the PR that introduced this note, it seems to be considered a bug that calling the same function on the same data could produce different results depending on pointer alignments:
#5972 (comment)
Yet in other GH issues, it is considered as expected behavior:
#317 (comment)
What is the intended behavior here? Should there be a guarantee that calling the same funcion on the same data and same machine should produce the same result regardless of data alignments?
Activity
martin-frbg commentedon Nov 2, 2026
Honestly I assume few if anybody in the mixed development history of OpenBLAS have given bitwise exact reproducibility a thought, and the original GotoBLAS that this project is striving to continue had next to no documentation to begin with. To my knowledge, there are no written guarantees of accuracy either, although it was obviously always assumed that it would need to be good enough to pass at least the original testsuite from the Reference BLAS.
It made sense to me when it was pointed out recently that reproducibility on the exact same machine would be desirable, xianyi appears to have seen it differently years ago, and I suspect the issue hinges on the type of application and thus permissible error.
I wonder if any any optimizing compiler could be expected to choose instructions to preserve bitwise reproducibility, if the hypothetical BLAS kernel was written in plain C ?
david-cortes commentedon Nov 2, 2026
I don't know for sure. I'm not familiar with assembly, but I generated this example in godbolt (
-mavx2 -O3, with an OMP SIMD reduction):https://godbolt.org/#z:OYLghAFBqd5QCxAYwPYBMCmBRdBLAF1QCcAaPECAMzwBtMA7AQwFtMQByARg9KtQYEAysib0QXACx8BBAKoBnTAAUAHpwAMvAFYTStJg1AB9U8lJL6yAngGVG6AMKpaAVxYMQAJlIOAMngMmABy7gBGmMQgABykAA6oCoS2DM5uHt7xickCAUGhLBFRsZaY1ilCBEzEBGnunj6l5QKV1QR5IeGRMRZVNXUZjX3tgZ2F3dEAlBaorsTI7BwA9EsA1AAqAJ5xmKubs8SraFirCJGYpKskq7SoTOirhquYqqxx9AB0AKQaAILoszC9FWCncxmqxAgaAYCgIqwBriBu1UXwArAAhNEAEUu0Nhq0CcIYkx%2Bvy%2BAHZMX9VjT4YDgbM4V8AMxY1Yab7Mqm/WmrL5eZlxYhMYAsJhXFhxEF4FgPYiYdCuZoMCD89EgRkk6m0/iHCCEgmqPms9ks9GG42OVYMM2GtVqyZXVxMryYk0ojF4D1Ys2k3nyghzBhOgi%2Bv4UrEcaa0Tio3ieDhaUioThWhQHBZ8gU8UgETRR6YAaxAzOZH1LFcrVYAbPpOJJ4/nk5xeAoQBpc/npnBYEg0JK6JFyJR%2B3FB1FkMgDEZsx2aLQCJE2xAwk2woFqptODn18xiJsAPJhbRlPPcXj9tiCA8MWhbxO8LBhVzARxiWht8%2BkLBioziB/fng8rlAAbpgn5Ji8ZTOosOaEpgMYAbQeBhMK%2B7OFgTYEMQMrbrwYHEGEiSYFimC/sAyFGF2fAGMACgAGp4JgADuB47AmOb8IIIhiOwUgyIIigqOoAG6Fw%2BiGCYZj6ChbaQNMqBxDYAifgAtPw7GSqsqkHsy2liiBqheLwqAEThWByRA0xNMpngQA4AyeOJ/ijAURR6AkSS2Y5HnZLZHRud04k2RUww%2BcFCGnqFbQBV0UTBWFLj1HosIxa5cUSNZGZ8dGsaNgBKYcKsqjRNWqnVpIRzTsAqwCh8XAfBoqwQLghDXPyzJcJMvBnlokxFiWZZVsNFa1ohDakAmSaFa27adg%2B3YwIgKCoAO9BkBQUKrWO60oMAXDVj486LsQy6rgBu6bnhpCXfuR4ntY12XowBA3neTZPi%2Bb60B%2B10/pJ/5JvgwE2GBEG8FByAwdd8GIUmyGoZuGGLEm2G4V%2BBFEUopHkZRoALTRIoMUxrHsddXHCKI4j8RTQlqE2ug%2BNVICmMY5gI5ZClKSkakaYwWk6XpqkGUZJlmXgFnwNZkXKvYDBOElGTOfLsXjPFWReSk4UazkDCq%2B5EVWLZrT9IrTkWDLxvDPrQW9G02upTUNvxVlcwLJldYcHGk1NoVxWleVlXAJOqwHR8XjNa1RCHB1XU9V2A2luWI3DZ7E1TSZLYWHNvUFp7xk%2BwVWe5/1pAEUkdiSEAA%3D%3D%3D
Then asked LLMs about it, and it appears that both GCC and CLANG always use unaligned loads.