Category Archives: Informational

On “random errors”

The differences between the dotted and straight lines is what a 2013 US government report calls “random errors”. In reality it is a non-linearity error, a systematic error that is repeated cyclically every centimeter. By labeling it a “random” error the accuracy estimate of the instrument was downgraded by more than on order of magnitude, which was then used to justify not to use it to monitor sediment spill at the dredging of the Port of Miami—an operation that killed some 200 acres of coral reef due to siltation that was not monitored in real time.

The data in the USACE report clearly shows that the errors are not random, yet they assumed that they were and calculated statistics as if they were.
The data in the government report clearly shows that the errors are not random, yet they assumed that they were and used statistics completely wrong. Data from Appendix B in the report  DOERT11.

Of course, this non-linearity error does not in any way prevent the instrument from measuring siltation with sub-mm accuracy, since the error is proportional to the distance measured: The shorter the distance, the smaller the error. The report also, for good measure, doubled the error in the conclusions, without any valid justification whatsoever (they claimed that difference data were level data and therefore doubled the error to account for difference data—which it already was from the start).

Furthermore, what matters in siltation monitoring is actually not the accuracy in determining the bottom level, but rather if it’s possible to detect siltation. This self-evident truth is easy to forget. The SediMeter measures the turbidity at 37 different levels. It’s like a scanner that delivers a vertical profile through the bottom, like a “photo” with elevation up and time to the right. It’s much more information than just a bottom level, all of which makes the SediMeter a tremendous asset for monitoring siltation in real time.

Sedimentation from suspension measured with a SediMeter in a 40 cm deep tank. And the USACE claim it can't be used to detect siltation. Hard to imagine a more absurd claim.
Sedimentation from suspension measured with a SediMeter in a 40 cm deep tank. And the government claims it can’t be used to detect siltation! Hard to imagine a more absurd claim. The red curve shows the level with cm scale on the left, while the right side shows relative turbidity.

If you have something to hide don’t deploy a SediMeter, since it was designed specifically for siltation monitoring with utmost transparency.

SediMeter Precision Demonstrated Publicly

On February 1st, 2016, a public demonstration of the SediMeter™ precision was held in Miami, Florida. During the course of the day, people were given a chance to put sand in the tank and see in real time on a connected computer the sediment level the instrument was measuring with a resolution of 0.001 cm (10 µm, equivalent to the thickness of a typical human hair). The instrument never failed to detect the addition of the 2.2 g of sediment, which corresponds to 0.1 mm in the 162 cm2 tank (given a bulk density of the top few centimeters of 1360 kg·m-3). The event was summed up in this video.

The demonstration tank with the SediMeter allowed attendees to compare what happened on the bottom with the SediMeter measurements displayed in real time on the computer screen.
The clear SediMeter demonstration tank allowed the attendees to compare what happened on the bottom, with the measurements displayed in real time on the computer screen.

 

 

SediMeter Accuracy Revisited

The second and third generation of the SediMeter™ were developed with the express purpose to work as siltation monitoring instruments in coral reefs, where 1 mm of sediment accumulation can be enough to kill corals. With its detection threshold of 0.1 mm, the SediMeter™ can be used for detecting such siltation in real time. It was designed as part of a system that allows networking, including wireless, of up to hundreds of instruments. It is manufactured in Miami and was developed with protecting the South Florida environment in mind.

A SediMeter™ SM3A in the test tank that was used for the accuracy tests. On Monday February 1st, 2016, a public demonstration of the accuracy will be made using this tank.
Figure 1. A SediMeter™ SM3A in the test tank that was used for the accuracy tests. On Monday February 1st, 2016, a public demonstration of the accuracy will be made using this tank.

Recently we found out that the US government decided to not even consider using the SediMeter™ for monitoring siltation on the coral reefs while dredging the harbor of Miami, because a US government study had concluded that it was not suitable for the purpose, since it allegedly did not have enough accuracy in distance measurements. If they had used the SediMeter™ to monitor siltation during the dredging, it might have been possible to avoid some or most of the costly damage to the coral reef.

It should be noted that the evaluation of the SediMeter™ was not conducted in order to evaluate its usefulness for the Miami dredging project, but for using it in a research project (cf. page 8). However, it was left out of consideration in the Miami project exactly because of the previous study by the same organization. Since the US government study landed two orders of magnitude wrong, we feel we need to put the record straight.

The US government measured and calculated the error in distance measurements (differential level) of various lengths from 1 mm and up; calculated the standard deviation; assumed normal distribution (thus justifying multiplying the standard deviation by 3 to estimate the 99.7% confidence interval); and in the conclusion stated that the calculated error value referred to a level (rather than a distance as they had said earlier in the report) why they doubled the error (in the conclusions!) in order to get the uncertainty of distance measurements. Based on such flawed arguments they concluded that the SediMeter™ is only suitable for cm-scale measurements, two orders of magnitude more than the detection threshold guaranteed by the manufacturer.

Dr. Ulf Erlingsson of Lindorm, Inc. has recently made a study of the accuracy by adding a small amount of sand 68 times to a tank with a SediMeter, once every 2 minutes. In contrast, the US government moved the SediMeter rather than adding sand, which is known to introduce small variations in the measurements. On the other hand, adding sand suffers from several sources of errors in the “true” value: The sand falls randomly and may not fall equally each time; consolidation may start after deposition; and a fine fraction may stay in suspension only to be deposited later, after measurements were made. These effects were minimized by depositing sand at an approximately even rate, and deposit several batches before starting to measure so that the bottom adjusts to the spatial pattern of deposition (so that the new layers become conformal). Also, values that were visibly affected by consolidation (the bottom level falls over time) were excluded from the final analysis. The sand was weighed with a 0.1 g resolution electronic scale.

Blue dots: Sand added on the X axis (g), and level measured on the Y axis (cm). Black line: Best linear interpolation. Red dots: Difference between blue dots and black line.
Figure 2: Blue dots: Sand added on the X axis (g), and level measured on the Y axis (cm). Black line: Best linear interpolation. Red dots: Difference between blue dots and black line (mm). This graph shows only the 32 values taken before the sand started to compact under its own weight (corresponding to 3 cm).

Most of the errors detected were found to be systematic with a period of 1 cm, and not random (Figure 2). It is due to the non-linearity in the interpolation between the detectors, and since they are spaced 1 cm apart, the error is cyclical. The US government report assumed that the errors were random, which they are not. Since the errors are not random, one cannot use a multiple of the standard deviation to calculate the confidence interval. Rather, the analysis of the confidence level must be made based on an understanding of the nature of the error.

The distance error in mm and % is here plotted as a function of the distance (the distances in Figure 1 have been added to get values up to and above 10 mm).
Figure 3. The distance error in mm (red) and percent (blue), plotted as a function of distance. The 338 error values were calculated by comparing over distances from 1 to 14 level measurements in Figure 2. The maximum absolute error varies cyclically with a peak at 5 mm, 15 mm, etc. The maximum relative error goes towards a max of ≤50% at distances below 1 mm, but decreases to ≤5% at 10 mm and all longer distances. The shortest distance in the data set is 0.64 mm, the longest 17.3 mm.

Figure 3 shows that the maximum absolute error in distance measurements varies with the same distance as the spacing between the optical backscatter detectors, again revealing that the error is not random. The maximum error was under 1 mm, and the occurred at a distance of between 4 and 6 mm. Looking instead at the relative error it increases towards shorter distances. The largest calculated relative error was 44% (the SediMeter had recorded 1.66 mm when the calculated sediment accumulation based on added sand was only 1.15 mm).

It may be noted that the level determination in the SediMeter was originally only made in software, and the process allowed for user input and interactivity. While it’s still possible for the user to adjust the correction of non-linearity errors, new SediMeters output a level directly. They also allow for burst sampling, with 20 burst samples in a single measurement. Taking the average of those measurements the noise is further reduced. It’s perfectly possible to get many measurements in a row with the same level value to 1/100th of a millimeter.

The only way to determine the detection threshold is to measure very small differences, and calculate if the change in the measurement data is statistically significant. The SM3 instrument resolution is 10 µm, and the detection threshold is advertised as 100 µm. Previous measurements have shown the detection threshold to be 50 g/m2 at 95% confidence level and 100 g/m2 at 99% confidence level, which corresponds to about 37 µm and 75 µm, respectively. For the SM2 instruments tested by the US government, the only sensitivity value given in advertisement was the resolution of 0.1 mm.

The new study confirms that the instrument indeed is capable of detecting 0.1 mm, and that the conclusion from the US government study simply is wrong. Surely many are asking by now, how could they come to a result that is off by two orders of magnitude? We can identify four contributing factors:

First, they measured by moving the SediMeter which introduced additional errors and appears to have increased the standard deviation of the data by about a factor 2.

Second, not realizing the distance-dependence of the error they assumed random data, why they calculated the standard deviation without taking into account the distance. This exaggerates the short-distance error by about a factor 25.

Third, again assuming that their measured error was random they doubled the erroneous standard deviation to calculate the 95% confidence level. This also doubled the error, but since the same doubling is made using the lower and true random error values for the detection threshold study made by Lindorm, the factor will be entered as 1 in the equation.

Fourth, they forgot that the measured standard deviation referred to a distance, and thinking it referred to a level they erroneously doubled it in order to make it apply to a distance (differential level). This increased the error by exactly a factor 2.

Multiplying 2 x 20 x 1 x 2 = 80, two orders of magnitude. Thus, instead of the US government value of 3.4 mm error at the 95% confidence level, the Lindorm estimate of the 95% uncertainty level in detection of siltation is 37 µm (50 g/m2), and the data shows that sedimentation of less than 1 mm can be measured well enough (±20%) for the requirements for siltation monitoring, QED.


On February 1st, 2016, 10 AM, a public demonstration will be made of the SediMeter accuracy. Anybody who has doubts about its ability to detect the sedimentation of 0.1 mm of sand (corresponding to 2.2 g in the 162 cm2 tank) is welcome to come and see with his or her own eyes. UPDATE: The demonstration was carried out and a video made from it. See later posts.

SediMeter proven to have high precision

We have conducted experiments to determine the SediMeter sensor response using a method that as closely as possible mirrors an actual measurement situation. The instrument was mounted in a tank and sediment was added in a controlled manner. The analysis showed that as expected most of the error is not random, but a cyclical nonlinearity error. It also proved that the detection limit is ≤0.1 mm, just as advertised.

The video presents the study and its findings.

SediMeter Warranty Increased to 2 Years

A couple of years ago Lindorm, Inc. introduced the third generation SediMeter™. While the second generation had featured a display, an input device, USB connector, SD card, and RS485 communication, and relied on primary batteries, the third generation only features RS485 communication in a permanently sealed unit with a built-in rechargeable battery. The goal was simplicity and durability for monitoring sediment accumulation, erosion, and siltation around dredging sites, in shipping channels, and so on.

SediMeter™ SM3A instrument
SediMeter™ SM3A instrument

Unfortunately, the battery management was not ideal for research users who tend to store instruments most of the time, with maybe a year between deployments. The battery would run empty and require special reset procedures to start the instrument up again, and sometimes even a battery replacement at the factory.

To overcome this, Lindorm is now, from January 1st, equipping all SediMeter™ units with a backup battery. When the rechargeable battery runs empty, the backup battery prevents it from getting over-discharged for up to 10 years. When charging again, a new automatic reset is designed to turn the instrument on automatically. This is expected to completely eliminate the potential for hassle when taking instruments out of storage.

Several other changes have also been made: A stronger handle to eliminate bent bayonets during anchor retrieval; thicker military-standard internal cables to avoid malfunction due to vibration; a much improved line for the cleaner shuttle to avoid snags; and numerous process improvements during assembly to increase chemical compatibility and strength in the long run.

To reflect this increase in quality, Lindorm increases the warranty for SediMeter™ instruments from 1 to 2 years. Furthermore, the warranty now also covers the batteries since there is no longer a requirement to periodically charge the battery to keep it from getting ruined.

 

SediMeter software update now available

Version 3.4 of the SediMeter™ software is now available on the Lindorm web site. It features a new Export function that allows the user to export all data from a SediMeter™ in a text file, suitable for import to a spreadsheet.

The generated file includes the UTC time, temperature, mean bottom level, and mean turbidity from the 37th optical backscatter center. Furthermore it contains the individual turbidity data from each of the 36 optical backscatter detectors in the SediMeter™ probe. Finally it contains the burst data. These are up to 20 measurements of level and turbidity that are used to calculate the mean level and mean turbidity.

The previous update, version 3.3, provided e-mail alerts. The software can be used for real-time monitoring of a single or a network of instruments. The user can define thresholds for erosion, accumulation, and turbidity. If any of these thresholds is exceeded, for any SediMeter™ in the network, an e-mail alert is sent. This is useful in a dredging operation, or a beach replenishment, where there are environmental concerns. The e-mail can go to the dredge operator, to the supervisor, to the regulatory agency, or to all three.

Version 3.2 provided burst sampling of the bottom level and turbidity from the 37th optical backscatter detector. The user can take from 1 to 20 samples in each measurement, with an interval from 1 s to 15 s. This can be a huge benefit if the SediMeter™ is deployed in an environment with wave action. Without burst sampling, the short-term variability within a single wave period can be larger than the variability in a tidal cycle, for instance. The burst sampling allows for this “nugget effect” to be quantified, and also to get a much more reliable average value in each measurement.

Version 3 of the sedimeter.exe software is the one that corresponds to the third generation SediMeter™. The SediMeter™ was invented in the 1980’s and patented in Sweden. SediMeter™ is a trade name of Lindorm, Inc.