stats.standard-deviation

def standard_deviation(values: Sequence[float], kind: DeviationKind, decimals: int) -> float

DeviationKind = Literal["population", "sample"]

from fune.stats.standard_deviation import standard_deviation  # stats.standard-deviation@^2

import math
from typing import Sequence

from .math_round_float import round_float  ← from math.round-float ^1.0.0 · built alongside by fune
from .stats_standard_deviation_types import DeviationKind


def standard_deviation(values: Sequence[float], kind: DeviationKind, decimals: int) -> float:
    """Population or sample standard deviation, two-pass.

    The one-pass "mean of squares minus square of mean" shortcut cancels
    catastrophically on large, close values; two passes do not.
    """
    if isinstance(values, (str, bytes)) or not isinstance(values, (list, tuple)):
        raise TypeError("values must be a list of numbers")
    for v in values:
        if isinstance(v, bool) or not isinstance(v, (int, float)) or not math.isfinite(v):
            raise TypeError("values must be finite numbers, received %r" % (v,))
    if kind not in ("population", "sample"):
        raise ValueError('unknown standard deviation kind "%s"' % (kind,))
    # Checked up front as well as in round_float (same wording), so a bad
    # decimals is reported before an empty list, as in 1.x.
    if isinstance(decimals, bool) or not isinstance(decimals, int) or decimals < 0 or decimals > 12:
        raise ValueError("decimals must be a whole number from 0 to 12, received %r" % (decimals,))
    n = len(values)
    if n == 0:
        raise ValueError("values must not be empty")
    if kind == "sample" and n < 2:
        raise ValueError("sample standard deviation needs at least 2 values")

    # Plain left-to-right float additions; sum() on ints would be exact and
    # math.fsum compensated, and either would differ from the other languages.
    total = 0.0
    for v in values:
        total = total + float(v)
    mean = total / n
    squares = 0.0
    for v in values:
        d = float(v) - mean
        squares = squares + d * d
    divisor = n - 1 if kind == "sample" else n
    # math.round-float rounds on the exact value of the double: 2.675 gives 2.67.
    return round_float(math.sqrt(squares / divisor), decimals)

Install

fune build

With that line in your source, in a Python project (language python in fune.project), fune build resolves it and its 1 dependency, pins them in fune.lock, downloads only the Python package of each, and builds the code above into your project’s .fune/build, one readable file per capability with a header linking back here. Or pin a range in fune.project and build in one step:

fune add stats.standard-deviation

The manifest, vectors and README with only the Python implementation. Install it without the registry with fune add ./stats.standard-deviation-2.0.0-python.fune, or fetch it from a terminal with fune pull stats.standard-deviation@2.0.0:python.

The whole function, every language, is one file too: stats.standard-deviation-2.0.0.fune, 13,777 bytes, sha256 21e27bf19a71cebfd8a92b40113bd4376f71d00616efdbdcdfc31ca968a95f44. It installs into a project of any language.

# fune: before stats.standard-deviation

# fune: after stats.standard-deviation

# fune: replace math.round-float in stats.standard-deviation

# fune: step stats.standard-deviation after <n|label>

More from the author

**Algorithm.** Two passes: the mean first, then the sum of squared deviations from it. The one-pass textbook shortcut, mean of squares minus square of the mean, cancels catastrophically when the values are large and close together (four readings near one billion come out as garbage or even a negative variance); the vectors include that case.

**Precision.** The result is rounded to `decimals` places (0 to 12) by `math.round-float`: half away from zero, decided on the exact value of the double. So a deviation of exactly 0.5 rounds to 1 at zero places, where Python's `round()` gives 0, and a deviation of 2.675 (stored as 2.67499999999999982...) rounds to 2.67 at two places.

**Why the three languages agree to the bit.** Sums run left to right, and only IEEE-754 +, -, x, /, square root and floor are used. All of these are correctly rounded by the standard (square root included, unlike sin or exp, whose last bit varies between math libraries), so TypeScript, Python and Rust hold the same double before rounding, and `math.round-float` rounds it the same way in all three.

## Changes in 2.0.0

2.0.0 rounds on the exact value of the double, so a deviation of 2.675 (the population deviation of 0 and 5.35) now gives 2.67. 1.x rounded the scaled product instead (floor of x x 10^decimals, compared with a half), and 2.675 x 100 is exactly 267.5 in floating point, so 1.x said 2.68. The private rounding helper is gone: this version requires `math.round-float ^1.0.0` and rounds with it, so every float capability in the registry rounds alike.

None of the 1.x vectors changed answers (each was recomputed from the exact value of its double). New vectors pin the difference: the population deviation of 0 and 5.35 gives 2.67 (1.x 2.68), of -2.675 and 2.675 gives 2.67 (1.x 2.68), and of 1.45 and -1.45 at one place gives 1.4 (1.x 1.5).

`decimals` is still 0 to 12, and still refused up front with the same message ("decimals must be a whole number from 0 to 12"), before an empty list or a one-value sample is reported, as in 1.x.

Sources: NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.5.6 "Measures of Scale"; B. P. Welford, "Note on a Method for Calculating Corrected Sums of Squares and Products", Technometrics 4(3), 1962, on why the one-pass formula fails.

Files