Basic theory of numerical accuracy and precision

We deal with numbers in every aspect of life. In mechanics, understanding the more obscure concepts and definitions can be critical to our designs. Most people understand that there are two kinds of numbers:

1) Exact numbers: You could safely state there is 1 balloon or 6 eggs in the fridge.
2) Inexact numbers: This could refer to rounded decimals.

Physics has proven however that all quantities are inexact, no quantity is absolute, and the closer you examine the nature of any quantity or number the more inexactitude is revealed.

You don’t have one balloon. Relativity shows us that a balloon that is moving or that is subject to its environment will have its mass increase or decrease on a microscopic but constant level (depending on the orientation of the observer). At a given time, you might have one+100pm of a balloon and 6+320pm of eggs. As Mandelbrot showed, the closer you look the more deviation is revealed.

In mechanical design engineering we constantly use numbers, which can be based on our own choices and statements as well as the ranges of numbers established by manufacturing and quality processes. Therefore, the mechanical design engineer must be careful with how he manipulates numerical input data and understand how the result will be affected.

Most importantly, a mechanical design engineer must be aware that, since exact values are impossible to achieve, we must control a range of errors.

To understand error, you need to first understand the difference between accuracy and precision:

  • Accuracy is the convergence of an observed value to a theoretical one.
  • Precision is the convergence of observed values between each other.

We can use a visual to better understand the subtle difference:
precision1In this example, the pink values are more accurate than the green ones because they converge more closely to the actual (true) value, represented by the blue dot.

In the following figure, the green dots are not very accurate but they are very precise, because the range between them is small (high precision, or repeatability) but they are not close to the theoretical blue center (low accuracy).

precision-2In order to get high accuracy and high precision, you need to get your values close to the theoretical center and have a small cloud of values, with a small variable distance between each value:

In order to reduce error in a measurement, accuracy and precision should both be taken into account.

To take accuracy into account, you must use a properly-calibrated measurement system. If you are positive that the precision of a manufacturing tool is acceptable and that it is calibrated according to the ambiant temperature and air, then you know that you will approach the correct value (which we can’t achieve) with a very tight set of measurements.

To determine precision, there are two key mathematical concepts:

  1. Significant figures.
  2. Precision number.

To determine significant figures in a given number, you count the number of figures, not counting leading zeroes.

For example:
0.00043 has only two significant figures (3 and 4). All the zeroes are leading and therefore not accountable.

Embedded zeroes are taken into account.

For example:
0.00403 has three significant figures (4, 0, and 3), since the zero is imbedded.

If the decimal point is displayed, trailing zeroes are taken into account.

For example:
300 has one significant figure (3) since the zeroes are trailing with no decimal point. If we want those zeroes to be taken into account, we must write the number as 300.

Scientific representation helps with the accountability of zeroes. It represents the significant figures, but multiplied by the exponential, or to the power of 10.

To determine the precision of a number is simple: it is the unit value of the least significant figure in a number. For example, the number 123.456 has 6 significant figures and a precision of 0.001.

To precisely represent a real number, several representations are used:

  • Scientific notation with exponentials, which clearly shows significant figures and the precision used.
  • Fixed point representation.
  • Floating point representation.

Fixed point representation is more convenient for summation and subtraction operations, and measurements taken together at the same level of precision. The result is presented with the same precision level as the least precise term involved. Therefore, summation and subtraction operations may result in loss of significant figures, especially if they’re near zero.

For example:
Let’s consider the following summation : 4.03 + 5.963 + 9.9934

The numbers have the following significant figures and accuracy

4.03 : 3 significant figures and 0.01 precision

5.963 : 4 significant figures and 0.001 precision

9.9934 : 5 significant figures and 0.0001 precision

The least precise term is 4.03

The summation’s result given by the calculator is 19.9864, which presents a precision of 0.0001. Following the rules of precision, the number should be rounded in order to meet the least precision used (0.01).

Therefore the proper result is 19.99

Floating point representation is the best for multiplication and division operations and for use with very small or very large numbers. The result should be written with the smallest number of significant figures of any of the factors involved. It often doesn’t have the same precision, but the significant figures should not increase. You must be careful with the precision of the input data to get the suitable precision for the output data.

For example:
Here is a simple operation : 6 x 8 = 48

-6 has one significant figure (6) and a precision of 1

-8 has one significant figure (8) and a precision of 1

The multiplication requires the result to have the smallest number of significant figures involved in the input operation. The result must therefore have a significant figure of 1.
Given that 48 has 2 significant figures, the result is rounded to the closest number with one significant figure, which is 50.

The accurate representation is 6 x 8 = 50

Of course there is a loss of precision between the first result that had a precision of 1 and the second one that has a precision of 10, which displays the paradox of “accuracy”. In order to solve this dilemma, the mechanical design engineer has to be careful with the input data significant figures and precision. A better (more accurate) way to write the operation could be:

6.0 x 8.0 = 48

6.0 has two significant figures and a precision of 1

8.0 has two significant figures and a precision of 1

48 has two significant figures as a well and there is no loss of precision.

An understanding of accuracy and precision comes in handy at all steps of the design process that deals with figures. If you are aware of the tools and processes that will manufacture your parts, you will have an idea about the precision required on the 2D plans. You are also aware of the precision of any number involved in an operation or equation that you might use to generate a length or radius. Finally, you are able to control your range of error by predicting it using the described methods, then narrowing or widening it according to resources.

About: Khadija Ouajjani

Since 2012. Mechanical Design Engineer in the aeronautics industry. Mainly dealing with CAD, FEA, simulation and analysis for turbo-engines. Writing for EC since 2014. Garlic, Color Pencils, Open Systems, Coffee, Herbert, Final Fantasy VII, Writing, Tolkien, Mechanics, Deutsch, Nihongo, Herbs, Aïkido, Tea, Cinnamon, Motion, Friends.

Leave a Reply

Skip to toolbar