collections.diff
Compare two lists of records by key: which were added, which removed, and which changed and in what fields.
1.0.0 · published 2026-10-03 by charlie · Anterra
Pinned by 18 tests, run in TypeScript, Python and Rust.
What it does
Compares two lists of records matched by `key` and says what happened between them: records that were added, records that were removed, and records present in both whose contents changed, with the names of the fields that differ. It is the heart of a sync job, an audit log or a "review changes before import" screen.
ORDER: `added` and `changed` follow the order of the `after` list, `removed` follows the order of the `before` list. `unchanged` is only a count, because a caller showing a diff never wants the untouched rows back.
For example
diffByKey(before ×3, after ×3, id)→ added ×1, removed ×1, changed ×1, unchanged 1 one of each: an addition, a removal, a change and an untouched recorddiffByKey(before ×2, after ×2, k)→ added , removed , changed , unchanged 2 identical lists: nothing added, removed or changeddiffByKey(before ×2, after ×2, k)→ added , removed , changed , unchanged 2 reordering is not a change
The function
The same function in TypeScript, Python and Rust, pinned by the same tests. Pick your language; the choice follows you around the registry.
export function diffByKey(before: readonly Readonly<Record<string, unknown>>[], after: readonly Readonly<Record<string, unknown>>[], key: string): RecordDiff<Readonly<Record<string, unknown>>>
| before | record[] | the old list; every record needs a unique value at key |
| after | record[] | the new list; every record needs a unique value at key |
| key | string | the field that identifies a record in both lists |
| returns | RecordDiff<record> |
The types it declares, generated into your project
/** What changed between two lists of records matched by key. */
export interface RecordDiff<T> {
/** in after but not before, in after's order */
readonly added: readonly T[];
/** in before but not after, in before's order */
readonly removed: readonly T[];
/** in both but different, in after's order */
readonly changed: readonly RecordChange<T>[];
/** how many records are in both and identical */
readonly unchanged: number;
}
/** One record present in both lists whose contents differ. */
export interface RecordChange<T> {
/** the key value, rendered as text */
readonly key: string;
readonly before: T;
readonly after: T;
/** names of the fields that differ, sorted by code point */
readonly fields: readonly string[];
}
Your code names it in one line, in the file that uses it
import { diffByKey } from "#fune/collections.diff@^1";
import { type RecordChange, type RecordDiff } from "./collections_diff_types.ts";
/** An open record, the manifest's `record`: a JSON-ish map whose shape is not known ahead of time. */
type DiffRecord = Readonly<Record<string, unknown>>;
/** Largest integer JavaScript can hold exactly; beyond it the three languages disagree. */
const SAFE_INTEGER = 9007199254740991;
function isMap(value: unknown): value is DiffRecord {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
/**
* A record's key as text, rendered the way collections.group-by-key renders a
* group name so the two agree on what "the same key" means. Null when absent.
*/
function keyOf(value: unknown, key: string): string | null {
if (value === undefined || value === null) return null;
if (typeof value === "string") return value;
if (typeof value === "boolean") return value ? "true" : "false";
if (typeof value === "number") {
if (!Number.isInteger(value)) {
throw new TypeError(`cannot diff by the fractional number ${value} at "${key}"`);
}
if (Math.abs(value) > SAFE_INTEGER) {
throw new RangeError(`cannot diff by the out-of-range number ${value} at "${key}"`);
}
return String(value);
}
throw new TypeError(`cannot diff by the list or map at "${key}"`);
}
/**
* Deep JSON equality, spelled out so all three languages agree: numbers by
* value (1 equals 1.0), no coercion between types (true is not 1, "1" is not
* 1), lists in order, and a missing map field equal to a null one.
*/
function same(a: unknown, b: unknown): boolean {
const aNull = a === undefined || a === null;
const bNull = b === undefined || b === null;
if (aNull || bNull) return aNull && bNull;
if (typeof a === "boolean" || typeof a === "number" || typeof a === "string") {
return typeof a === typeof b && a === b;
}
if (Array.isArray(a)) {
if (!Array.isArray(b) || a.length !== b.length) return false;
for (let i = 0; i < a.length; i++) if (!same(a[i], b[i])) return false;
return true;
}
if (isMap(a) && isMap(b)) {
for (const k of Object.keys(a)) if (!same(a[k], b[k])) return false;
for (const k of Object.keys(b)) if (!(k in a) && !same(undefined, b[k])) return false;
return true;
}
return false;
}
/** Code point order; JavaScript's `<` compares UTF-16 code units instead. */
function compareCodePoints(a: string, b: string): number {
const ca = Array.from(a);
const cb = Array.from(b);
const shared = Math.min(ca.length, cb.length);
for (let i = 0; i < shared; i++) {
const x = ca[i].codePointAt(0) as number;
const y = cb[i].codePointAt(0) as number;
if (x !== y) return x < y ? -1 : 1;
}
return ca.length === cb.length ? 0 : ca.length < cb.length ? -1 : 1;
}
/** Key every record of one list, refusing missing and duplicate keys. */
function index(records: readonly DiffRecord[], key: string, side: string): Map<string, number> {
const byKey = new Map<string, number>();
records.forEach((record, i) => {
const k = keyOf(isMap(record) ? record[key] : undefined, key);
if (k === null) {
throw new TypeError(`record ${i} in ${side} has no value at "${key}"`);
}
// Picking one of two duplicates would report changes that never happened.
if (byKey.has(k)) {
throw new RangeError(`duplicate key "${k}" in ${side}`);
}
byKey.set(k, i);
});
return byKey;
}
/**
* Compare two lists of records matched by `key`: what was added, what was
* removed, and what changed and in which fields.
*/
export function diffByKey(
before: readonly DiffRecord[],
after: readonly DiffRecord[],
key: string,
): RecordDiff<DiffRecord> {
if (!Array.isArray(before) || !Array.isArray(after)) {
throw new TypeError("diffByKey needs two lists of records");
}
if (typeof key !== "string" || key.length === 0) {
throw new TypeError("diffByKey needs a non-empty key name");
}
const beforeKeys = index(before, key, "before");
const afterKeys = index(after, key, "after");
const added: DiffRecord[] = [];
const changed: RecordChange<DiffRecord>[] = [];
let unchanged = 0;
for (const [k, i] of afterKeys) {
const next = after[i];
const j = beforeKeys.get(k);
if (j === undefined) {
added.push(next);
continue;
}
const prev = before[j];
const names = new Set<string>([...Object.keys(prev), ...Object.keys(next)]);
const fields = [...names].filter((name) => !same(prev[name], next[name])).sort(compareCodePoints);
if (fields.length === 0) unchanged++;
else changed.push({ key: k, before: prev, after: next, fields });
}
const removed: DiffRecord[] = [];
for (const [k, j] of beforeKeys) {
if (!afterKeys.has(k)) removed.push(before[j]);
}
return { added, removed, changed, unchanged };
}Install
fune build
With that line in your source, in a TypeScript project (language typescript in fune.project), fune build resolves it and nothing else, pins them in fune.lock, downloads only the TypeScript package of each, and builds the code above into your project’s .fune/build, one readable file per capability with a header linking back here. Or pin a range in fune.project and build in one step:
fune add collections.diff
The manifest, vectors and README with only the TypeScript implementation. Install it without the registry with fune add ./collections.diff-1.0.0-typescript.fune, or fetch it from a terminal with fune pull collections.diff@1.0.0:typescript.
The whole function, every language, is one file too: collections.diff-1.0.0.fune, 27,655 bytes, sha256 0416ed8558f592216df058b45d0fc42a83ef3f616cee47d44df8507dcd1a0546. It installs into a project of any language.
Customise it in your app
The seams this capability offers. Put a marker directly above a function of your own and fune build wires it into the built code; the package on the registry is not changed, the built file’s header lists it under CUSTOMISED, and fune hooks lists every hook in the project. How hooks work.
before — your function gets the arguments and returns them, changed or not, or throws to refuse the call.
// fune: before collections.diff
after — your function gets the result and the arguments, and returns the final result.
// fune: after collections.diff
replace — it requires no other capability, so there is no dependency to replace.
step — your function runs at a numbered point inside the function’s body, receives the in-scope values it names as parameters, and may return replacements. List the points with fune show collections.diff --steps.
// fune: step collections.diff after <n|label>
Tests
A version published now needs at least 8 tests for every function, and one that expects the error for each function that throws; the registry refuses it otherwise. fune verify --all runs each case in TypeScript, Python and Rust, and a project runs them again with fune verify. This page lists the cases; it does not run them. The exact JSON is vectors.json.
| Case | Arguments | Expected | |
|---|---|---|---|
| one of each: an addition, a removal, a change and an untouched record | before ×3, after ×3, id | → | added ×1, removed ×1, changed ×1, unchanged 1 |
| identical lists: nothing added, removed or changed | before ×2, after ×2, k | → | added , removed , changed , unchanged 2 |
| reordering is not a change | before ×2, after ×2, k | → | added , removed , changed , unchanged 2 |
| from nothing: everything is added, in after's order | , after ×2, k | → | added ×2, removed , changed , unchanged 0 |
| to nothing: everything is removed, in before's order | before ×2, , k | → | added , removed ×2, changed , unchanged 0 |
| changed fields are sorted by code point, including a field added and a field removed | before ×1, after ×1, id | → | added , removed , changed ×1, unchanged 0 |
| changed records follow after's order | before ×2, after ×2, k | → | added , removed , changed ×2, unchanged 0 |
| 1 and 1.0 are the same number, so a float round trip is not a change | before ×1, after ×1, k | → | added , removed , changed , unchanged 1 |
| a number that became a string did change | before ×1, after ×1, k | → | added , removed , changed ×1, unchanged 0 |
| true is not 1 | before ×1, after ×1, k | → | added , removed , changed ×1, unchanged 0 |
Show the other 8 tests
| Case | Arguments | Expected | |
|---|---|---|---|
| a null field and a missing field are the same, so dropping nulls is not a change | before ×1, after ×1, k | → | added , removed , changed , unchanged 1 |
| nested lists and maps are compared deeply; list order matters | before ×1, after ×1, k | → | added , removed , changed ×1, unchanged 0 |
| a key of 1 and a key of "1" match; the key field itself is reported as changed type | before ×1, after ×1, id | → | added , removed , changed ×1, unchanged 0 |
| a duplicate key in before is an error, not a guess | before ×2, , k | → | error: duplicate key "a" in before |
| a duplicate key in after is an error | , after ×2, k | → | error: duplicate key "7" in after |
| a record with no value at the key is an error | before ×1, after ×1, k | → | error: record 0 in after has no value at "k" |
| a fractional key is an error | before ×1, , k | → | error: cannot diff by the fractional number |
| an empty key name is an error | , , | → | error: needs a non-empty key name |
More from the author
CHANGED FIELDS are listed by name, sorted by Unicode code point. Sorting, rather than keeping the records' own field order, is deliberate: JavaScript reorders integer-like object keys ("2" before "a") and the three languages would otherwise disagree about the same records.
EQUALITY is deep JSON equality, spelled out the same way in all three languages rather than inherited from each one's `==`:
- numbers compare by value, so `1` and `1.0` are equal (JSON does not distinguish them, and a round trip through a float column must not show up as a change); - `true` is not `1`, and `"1"` is not `1`: a value that changed type did change; - lists are equal when they have the same length and equal items in the same order; - maps are equal when every field is equal, where a missing field and a null field count as the same. That is the house rule across the collections capabilities, and it stops a serialiser that drops nulls from making every record look changed. It applies to the top-level fields too: a field that goes from null to absent is not reported.
KEYS are rendered the way `collections.group-by-key` renders them: strings are themselves, whole numbers their digits, booleans `true`/`false`. So a record keyed `1` in one list and `"1"` in the other is the same record (and the key field itself is then reported as changed, because its type did change). A fractional number, a list or a map at the key is an error.
ERRORS, not guesses: a record with no value (or null) at the key, and a key that appears twice in the same list. A diff that silently picked one of two duplicates would report changes that never happened. Also an empty key name and a non-list argument. Inputs are never modified; the records in the result are the records passed in.
The result types are generic (`RecordDiff<T>`, `RecordChange<T>`) and this function returns them over `record`. That keeps the door open for a typed variant, and it is also how the manifest gets a record-valued field past the Python type generator, which does not yet import `Any` for a bare `record` field.
Files
| Path | Bytes |
|---|---|
| README.md | 2,642 |
| impl/python.py | 4,233 |
| impl/rust.rs | 5,898 |
| impl/typescript.ts | 4,816 |
| vectors.json | 5,433 |