There are a variety of number systems in which a number can be represented. In the common base 10 (decimal) system each digit takes on one of 10 values, from 0 to 9. In base 2 (binary), each digit takes on one of 2 values, 0 or 1.
For a given
Example:
Some common bases used for numbering systems are:
Modern computers use transistors to store data. These transistors can either be ON (1) or OFF (0). In order to store integers in a computer, we must first convert them to binary. For example, the binary representation of 23 is
Converting an integer from binary representation (base 2) to decimal representation (base 10) is easy. Simply multiply each digit by increasing powers of 2 like so:
Converting an integer from decimal to binary is a similar process, except instead of multiplying by 2 we will be dividing by 2 and keeping track of the remainder:
Thus,
becomes
You may find these additional resources helpful for review: Decimal to Binary 1 and Decimal to Binary 2
Real numbers add an extra level of complexity. Not only do they have a leading integer, they also have a fractional part. For now, we will represent a decimal number like 23.375 as
Converting a number with a fractional portion from binary to decimal is similar to converting to an integer, except that we continue into negative powers of 2 for the fractional part:
To convert a decimal fraction to binary, first convert the integer part to binary as previously discussed. Then, take the fractional part (ignoring the integer part) and multiply it by 2. The resulting integer part will be the binary digit. Throw away the integer part and continue the process of multiplying by 2 until the fractional part becomes 0. For example:
By combining the integer and fractional parts, we find that
Not all fractions can be represented in binary using a finite number of digits. For example, if you try the above technique on a number like 0.1, you will find that the remaining fraction begins to repeat:
As you can see, the decimal 0.1 will be represented in binary as the infinitely repeating series
The floating point representation of a binary number is similar to scientific notation for decimals. Much like you can represent 23.375 as:
you can represent
A floating-point number can represent numbers of different orders of magnitude(very large and very small) with the same number of fixed digits.
More formally, we can define a floating point number
where:
Aside from the special case of zero and subnormal numbers (discussed below), the significand is always in normalized form:
where:
Whenever we store a normalized floating point number, the 1 is assumed. We don’t store the entire significand, just the fractional part. This is called the “hidden bit representation”, which gives one additional bit of precision.s
A number
The exponent is shifted by 127 to avoid storing a negative sign. Instead of storing
Convert the binary number
Answer:
From here we can obtain
Thus
For additional reading about IEEE Floating Point Numbers
The exponent is shifted by 1023 to avoid storing a negative sign. Instead of storing
There are several corner cases that arise in floating point representations.
In our definition of floating point numbers above, we said that there is always a leading 1 assumed. This is true for most floating point numbers. A notable exception is zero. In order to store zero as a floating point number, we store all zeros for the exponent and all zeros for the fractional part. Note that there can be both +0 and -0 depending on the sign bit.
If a floating point calculation results in a number that is beyond the range of possible numbers in floating point, it is considered to be infinity. We store infinity with all ones in the exponent and all zeros in the fractional.
Arithmetic operations that result in something that is not a number are represented in floating point with all ones in the exponent and a non-zero fractional part.
The above image shows the number line for the IEEE-754 floating point system.
A normal number is defined as a floating point number with a 1 at the start of the significand. Thus, the smallest normal number in double precision is
However, we can go even smaller than this by removing the restriction that the first number of the significand must be a 1. These numbers are known as subnormal, and are stored with all zeros in the exponent. Technically, zero is also a subnormal number.
It is important to note that subnormal numbers do not have as many significant digits as normal numbers.
IEEE-754 Single precision (32 bits):
IEEE-754 Double precision (64 bits):
The use of subnormal numbers allows for more gradual underflow to zero (however subnormal numbers don’t have as many accurate bits as normalized numbers).
Machine epsilon (
For IEEE-754 single precision,
For IEEE-754 double precision,
Or for a general normalized floating point system
In programming languages these values are typically available as predefined constants.
For example, in C, these constants are FLT_EPSILON
and DBL_EPSILON
and are defined in the float.h
library.
In Python you can access these values with the code snippet below.
import numpy as np
# Single Precision
eps_single = np.finfo(np.float32).eps
print("Single precision machine eps = {}".format(eps_single))
# Double Precision
eps_double = np.finfo(np.float64).eps
print("Double precision machine eps = {}".format(eps_double))
Note: There are many definitions of machine epsilon that are used in various resources, such as the smallest number such that