Should I normalize 8-bit RGB values by 255 or 256?

For ordinary normalized RGB, divide by 255. That maps 0 to exactly 0.0 and 255 to exactly 1.0, which is usually what other image and graphics code expects.

Dividing by 256 without an offset maps 255 to about 0.9961. If the result is then treated as a conventional normalized value, white will come out slightly dimmer.

There is another version of the 256 approach: use (value + 0.5) / 256. That treats each integer as the center of one of 256 equal-width bins rather than placing the endpoints at 0 and 1.

It can be useful when thinking about quantization and uniform distributions, but it maps black to about 0.00195 and white to about 0.99805. Both sides of the pipeline need to agree on that convention. It is not a drop-in replacement for ordinary normalized RGB.

The bin choice can show up in statistics. For example, generating uniform noise and then quantizing it can produce a different histogram at 0 and 255 depending on how the endpoints are handled.

If those details matter, check the whole conversion and quantization process rather than just the divisor. Higher precision may also be worth considering instead of repeatedly saving to 8-bit values.

Also, normalization is separate from color space. Dividing an sRGB value by 255 does not make it linear light. For operations that need linear light, convert from sRGB first, do the calculation, then convert back before quantizing.