Much dark data arises in the form of entire records missing – people who refuse to take part in surveys, stars which do not radiate in the visible spectrum, patients not yet showing symptoms, and so on. But other dark data are more subtle. One example is when numbers are rounded. Of course, apart from counts all data are rounded – no measurement can be taken to an infinite number of decimal places. But this rounding means there are details, possibly very small details, that you don’t know. Details that are dark.
Normally this does not matter. For many questions the error introduced by rounding is still small, and indeed often positive and negative rounding errors tend to cancel out. Sometimes, however, the errors can propagate, adding up to a number which is of a sufficient size to matter. This happened on 25 February 1991 during Operation Desert Storm, when a Patriot missile defense system failed to track and intercept a Scud missile, which went on to hit an army barracks and kill 28 soldiers.
Interception of a missile is based on knowing when it was at a specific point, how fast and in what direction it is moving, and how much time has elapsed since it was observed. For the Patriot system, the internal clock measured time by counting tenths of a second from when the system was switched on – which means it was a number which increased the longer the system had been running. To convert the measured time to seconds, this count was multiplied by 1/10. But 1/10 in decimal translates into an unending series in binary: 0.00011001100110011… and the system used a 24 bit register. This meant that the expansion was truncated at 24 bits, so introducing a tiny rounding error (in decimal, about 0.000000095). Tiny, certainly, but multiply that by the time since the system had been switched on, which was about 100 hours, and it added up to a number which mattered: just over 1/3 rd of a second, A Scud missile will travel over half a kilometer in that time. It meant the Patriot system would be looking for the Scud in the wrong place.
It’s an example of tiny fragments of dark data adding up to have major consequences.
Recent Comments