# Common problems

From OpenGENIE

Revision as of 17:29, 27 May 2009 by Freddie Akeroyd (talk | contribs) (New page: ==Why does A/B give the wrong answer?== If A and B are both integers, then integer division is performed; if either is a floating point number then the integer value is first converted int...)

## Why does A/B give the wrong answer?

If A and B are both integers, then integer division is performed; if either is a floating point number then the integer value is first converted into a floating point number before division is performed. Thus:

1/2 == 0 # 1 and 2 are of Integer type

but

1/2.0 == 0.5 # 2.0 is a floating point number, so integer 1 is converted to 1.0 before division is done

If you are unsure what data type two variables may be, then you can force real division using either of these statements

RES = AS_REAL(A) / B RES = 1.0 * A / B # equivalent to (1.0*A)/B as "*" and "/" are equal priority so the expression is evaluated left to right