In this post, we attempt to unravel the mysteries of the “magic” constant found in the fast inverse sqrt method in an intuitive fashion.

# Currently Browsing

## Posts Tagged ‘ floating point ’

28 May

Posted in
Article, Numerical Analysis

# Introduction to Scientific Computing: Error Propagation

The first part on a series designed to survey the design and analysis of various numerical methods will look at error propagation.

18 January

Posted in
Article, Numerical Analysis

# Fixing floating point cancellation I

How can you compute to 14 digits of accuracy within the domain ?

17 January

Posted in
Numerical Analysis

# Floating point quirks

In this post, we’ll explore a scenario where the non-~~commutativity~~associativity of floating point arithmetic can lead us into trouble.

Let and . Write a computer program to compute this sum.