HACKER Q&A
📣 jpegqs

“undefined behavior” or “target machine behavior”?


I have encountered a bug in GCC that can lead to security vulnerabilities in software, but the GCC developers refuse to consider this a bug.

So I ask for your opinion on this matter.

Here's the URL: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98168

Copied the text below.

Here GCC thinks that if both signed integers are positive, then the sum of these integers is also positive. And removes the next bounds check for the negative values (it could be written different, but this is the common way).

  int test(int a, int b, int *buf) {
    if (a >= 0 && b >= 0) {
      a += b;
      // let's check that we are not reading outside the buffer
      if (a >= 0 && a < 8) return buf[a];
    }
    return -1;
  }
So this code supposed to read the element A+B from a buffer of 8 values. And if the sum is out of the buffer, then return -1. But when compiling with GCC -O2/O3 on x86/x86_64 (and possibly others), you can pass A=0x7fffffff, B=0x7fffffff and access buf[-2] (as with any negative value except -1).

Thus, optimizations that falsely assume that the target machine is performing signed integer saturation when it is not - should be considered dangerous.

In my opinion, UB in C has a different purpose, it exists because C is a low-level language and in most cases can use a single machine instruction for a general operation. So for compilers it should be "target machine behavior", not "we can do anything". And compilers must maintain this behavior while removing some operations when optimizing the code.


  👤 bobthebuilders Accepted Answer ✓
Undefined behavior is undefined behavior. Signed integer has been undefined behavior since 1989. If you want wrapping arithmetic, using a library or langauge with those primitives built in.

If you want to use GCC for example,

-fwrapv This option instructs the compiler to assume that signed arithmetic overflow of addition, subtraction and multiplication wraps around using twos-complement representation. This flag enables some optimizations and disables others. The options -ftrapv and -fwrapv override each other, so using -ftrapv -fwrapv on the command-line results in -fwrapv being effective. Note that only active options override, so using -ftrapv -fwrapv -fno-wrapv on the command-line results in -ftrapv being effective.


👤 detaro
The standard is clear about this: It's undefined behavior, and the compiler is allowed to make this assumption. The part that does the optimization might not even be aware what the target platform is or does, and doesn't need to be. (And the discussion specifically about integer addition is probably decades-old, so it's unlikely you're going to change any compiler maintainers view on this now)

👤 jpegqs
The better way to check for buffer overflow here is "((unsigned T)i < n)".

But the "(i >= 0 && i < n)" way that I used in this example - I have seen many times.

So hackers can scan open source software compiled with GCC for such issue and can exploit this vulnerability, since GCC developers will not fix it.