[工作积累] 32bit to 64bit: array index underflow

先贴一段C++标准(ISO/IEC 14882:2003):

5.2.1 Subscripting:

1 A postfix expression followed by an expression in square brackets is a postfix expression. One of the
expressions shall have the type “pointer to T” and the other shall have enumeration or integral type. The
result is an lvalue of type “T.” The type “T” shall be a completely-defined object type.56) The expression
E1[E2] is identical (by definition) to *((E1)+(E2)). [Note: see 5.3 and 5.7 for details of * and + and
8.3.4 for details of arrays. ]

 5.7 Additive operators:

5 When an expression that has integral type is added to or subtracted from a pointer, the result has the type of the pointer operand. If the pointer operand points to an element of an array object, and the array is large enough, the result points to an element offset from the original element such that the difference of the subscripts of the resulting and original array elements equals the integral expression. In other words, if the expression P points to the i-th element of an array object, the expressions (P)+N (equivalently, N+(P)) and (P)-N (where N has the value n) point to, respectively, the i+n-th and i–n-th elements of the array object, provided they exist. Moreover, if the expression P points to the last element of an array object, the expression (P)+1 points one past the last element of the array object, and if the expression Q points one past the last element of an array object, the expression (Q)-1 points to the last element of the array object. If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.

最近工作中遇到的问题是在64位IOS上:

uint32 i;
...
char* n = (char*)&array[i-1] + n;

当i为0的时候, 下标溢出.

对照标准, 可以看出, array[i-1]相当于*(array+uint32(i-1)), 当i-1计算溢出时, 下标的数值很大(0xFFFFFFFFU), 数组的寻址也溢出了, 结果是undefined behavior.

然而在32位上的结果正确, 为什么呢?

因为0xFFFFFFFF用来寻址的时候, 对于大多数32位CPU来说, 都是带符号的数值, 所以0xFFFFFFFF被CPU解释为-1.

这个时候array[0xFFFFFFFF]相当与array[-1].

然而对于C++来说, 这只是一个巧合. 根据标准定义, 他是undefined behavior. 所以在64位上出错也不奇怪.

对于64位, 当i为0时, 下标计算结果仍然是0xFFFFFFFF, 然而64位上的-1是0xFFFFFFFFFFFFFFFF, 显然0xFFFFFFFF是一个很大的正整数, 导致真正溢出.

解决办法是它转换为带符号数, 即: array[ int(i-1) ]. 这样的结果是-1, 而不是0xFFFFFFFFU.

回想blade的下标索引, 基本上都用的是size_t:

typedef size_t index_t;
index_t i;

因为size_t在64位系统上是64字节, 所以i-1得到的结果是0xFFFFFFFFFFFFFFFF, 这样也能保证基本上不出错. 

然而这么做仍然是undefined behavior, 因为0xFFFFFFFFFFFFFFFF在用作寻址的时候, 是作为有符号数还是无符号数, 这个是由CPU来定义的, 虽然大部分CPU为了灵活都是用的是带符号的.

但是这是CPU的细节. 假如现在有一个公司, 如BMD或者UNTEL公司生产了新型的CPU, 其对与类似

mov eax, [ebx + esi*4]
lea eax, [ebx + esi]

的指令, 把索引寄存器(例如esi)解释为无符号数, 而在地址溢出时做了处理, 比如CPU异常信号或者地址回卷等等, 那么结果又很难说了.

这已经设计到CPU的细节, 超出了C++可控的范围, 所以C++标准才认为这是undefined behavior.

最好的方式是用带符号的类型, 比如int/long/long long/ptrdiff_t来作为下标, 当然有时候不能控制下标变量i的类型(来源), 那么在使用时注意转换.

或者, 将&array[i-1]改为 array + i - 1,  注意不是array + (i-1)

这样能够避免i为0时的溢出.