Performance

Every Yuneta release is measured before it is tagged, and a release that moves a lot of code ships a performance report. This page keeps the history: the trend of the main figures, report after report, and a link to each report.

Two kinds of figures are kept:

• A/B figures. The same benchmark, built with the module of the new release and with the module of the release before, the two binaries run alternately on one machine. They say what the release changed. A change inside the spread of its rounds is noise, and it is called noise.

• Absolute figures. The benchmarks run on the build of the release at tag time. They say what one machine does with the release: records stored per second, messages per second, what one operation costs.

A figure that got slower is never left out. It is shown with its reason, as the CHANGELOG states it: most are the price of a guarantee (a file that survives a power cut, a record that reaches the disk before a command runs), kept by decision.

Reports¶

The report is one self-contained page (no script, no external file). GitHub shows it as source code; the “view” link renders it. How a report is made, and the schema of its .json, are in performance/reports/README.md.

The trend¶

The C_TREEDB open of 7.25.5 compares two separate runs: 7.25.4 (medians of an earlier run) and 7.25.5 as shipped (the tag-time run). Every other point of the table comes from one alternated A/B run. Each point is the figure its own report measured, so two neighbouring points come from two runs, days apart: the A/B of 7.25.20 measured 7.25.5 again, the A/B of 7.25.21 measured 7.25.20 again, the A/B of 7.25.22 measured 7.25.21 again, and the A/B of 7.26.0 measured 7.25.22 again; their figures are in the sections below. Between two such reports a figure can move by a few percent with no change of the code (the master open of 7.25.20 measured 78.2 ms in its own report and 82.4 ms in the A/B of 7.25.21, another day): read a trend against the A/B of its report.

The appends of 7.25.20, 7.25.21, 7.25.22 and 7.26.0 are measured on whole-release builds (see below), not on the four padded links of 7.25.5. The tm query of 7.25.5 to 7.25.22 is the one of a topic its release had marked; 7.26.0 has no tm markers and reads every row of the key (a topic 7.25.22 had not marked took 402 ms).

What each figure means¶

7.26.0 against 7.25.22¶

7.26.0 changes how timeranger2 reads and how its followers hear deletes: the md2 rows of a scan are read 1024 at a time with one pread(), and the match condition is parsed once per scan; the tm markers of 7.25.5..7.25.22 are gone, so a tm condition filters every row of any topic; a key delete reaches the rt_disk followers with the master’s delete sequence; and a link in a treedb publishes the relationship (with_link_events on by default), not the parent collapsed whole.

Fixed before the tag: a master that opens a topic makes its delete_seq.json (the record of the delete sequence, which a follower takes as the sign that its master signals each delete in order). Written durably -- a file fsync, a rename and a directory fsync per topic -- it doubled the creation of 10 topics (131.4 -> 258.5 ms) and made the first open of 40 treedbs (400 topics) 55% longer (10.42 -> 16.19 s). It holds 0 and is made again if lost, so it is written NOT durably now: 134.0 ms and 10.33 s, within the noise of 7.25.22 (the 7.26.0 side measured again after the fix, 8 rounds).

Not measured: a key delete on a topic with an rt_disk feed. No benchmark deletes keys under a feed; by construction such a delete writes the record of its sequence first, two fsyncs and a rename more than 7.25.22 did. A topic without feeds pays nothing.

Every other figure moved inside its spread (|t| < 3), on paths that changed or not. The binaries are 0.04-0.15% larger (stripped).

The method: 7.25.22 and 7.26.0 each built from their own tree (a git worktree of the 7.25.22 tag with the same .config and compiler, where only the kernel libraries, the modules, performance/c/ and the three ctest binaries were built), the two binaries of each benchmark run alternately, the order flipped every round, with a sync and a 3 s pause before each run: 8 rounds, 24 for the three timeranger2 ctest binaries. outputs_ext was shared: linux-ext-libs did not change. The treedb figures are CPU time of the process.

What one machine does with 7.26.0¶

The 7.26.0 side of the same run: 8 rounds (24 for the ctest binaries), mean of the rounds, on the machine described below.

7.25.22 against 7.25.21¶

7.25.22 closes two reviews -- of 7.25.21, and of the fixes made for it -- and the open defects of the list it inherited: the agents run as native systemd units; a connection can be dropped from any callback of TLS or TCP without a use of freed memory; a yuno alive but not connected can be killed; the rt_disk followers of timeranger2 hear what they used to lose; C_AUTHZ and the IdP stop handing roles and keys to whoever asks; a stats reset resets.

Four figures moved outside their spread on code that did not change in this release, and none is claimed:

Both releases are built whole, so the code lands at other addresses, which alone moves a figure by a few percent: that is placement, not speed, and it is said with its figure. The binaries are 0.4-0.5% larger (stripped).

The method: 7.25.21 and 7.25.22 each built from their own tree (a git worktree of the 7.25.21 tag with the same .config and compiler, where only the kernel libraries, modules/c/test, performance/c/ and the three ctest binaries were built), the two binaries of each benchmark run alternately, the order flipped every round, with a sync and a 3 s pause before each run: 8 rounds, 24 for the three timeranger2 ctest binaries. outputs_ext was shared: linux-ext-libs did not change.

What one machine does with 7.25.22¶

The 7.25.22 side of the same run: 8 rounds (24 for the ctest binaries), mean of the rounds, on the machine described below.

7.25.21 against 7.25.20¶

7.25.21 is a security and correctness release: credentials are masked in every trace, in every kw the kernel dumps, in the traffic dumps and in view-config; a gbuffer can be marked secret (hidden from the dumps, wiped when it is freed, never serialized); C_TCP and C_TCP_S stop and start again cleanly and count their connections; the rt_disk feeds of timeranger2 hear every delete. None of it is on the paths that run every second at the default trace levels: the masking runs only when a trace prints. On those paths a secret gbuffer costs the test of one flag, and a TCP send one key lookup (the __secret__ flag of EV_TX_DATA).

Nothing got slower beyond the spread of its rounds. Three figures moved outside it, all of them faster, and none is claimed as a gain:

Both releases are built whole, so the code lands at other addresses, which alone moves a figure by a few percent (2.5% on an append, measured for 7.25.5): that is placement, not speed. The main append benchmark, test_topic_pkey_integer, 24 rounds, moved +1.3% (223,031 -> 225,883 appends/s), inside its spread. The binaries are 0.6-0.9% larger (stripped), the code of the masking.

The method: 7.25.20 and 7.25.21 each built from their own tree (a git worktree of the 7.25.20 tag with the same .config and compiler, where only the kernel libraries, modules/c/test, performance/c/ and the three ctest binaries were built), the two binaries of each benchmark run alternately, the order flipped every round, with a sync and a 3 s pause before each run: 8 rounds, 24 for the three timeranger2 ctest binaries. outputs_ext was shared: linux-ext-libs moved from 1.22 to 1.23 in this release, but only in how openresty is downloaded, and no library changed version.

What one machine does with 7.25.21¶

The benchmarks of performance/c/ on the build of 7.25.21 at tag time, 5 rounds; the appends and reads of the ctest binaries are the 7.25.21 side of the A/B run (24 rounds). Mean of the rounds, on the machine described below.

7.25.20 against 7.25.5¶

The fifteen releases between the two reports fixed the agent, the file watcher, the TCP writes, the SWITCHS macro and the masking of secrets. None of them was meant to change speed, and one of them is the largest gain of the round: since 7.25.7 SWITCHS no longer compiles a regular expression each time a switch is entered (it leaked it on every return from a case, commit 3e8ff0508). tr_treedb switches on the type of every column, so every treedb write paid a regcomp(".*") and a regfree; tr_treedb.c and c_treedb.c did not change between the two releases.

Nothing got slower beyond the spread of its rounds. The closest, tm_build_appends (600 000 appends to one key, 1,632 -> 1,691 ms, +3.6%, at the edge of the spread), runs append code that is the same in both releases: timeranger2.c changed only in its rt-disk rescan. Both releases are built whole, so the code lands at other addresses, which alone moves an append by up to 2.5% (measured for 7.25.5); the main append benchmark, test_topic_pkey_integer, 24 rounds, moved +0.3%.

The method: 7.25.5 and 7.25.20 each built whole from their own tree (a git worktree of the tag for 7.25.5; the same .config, compiler and external libraries), the two binaries of each benchmark run alternately, the order flipped every round, with a sync and a 3 s pause before each run: 8 rounds, 24 for the three timeranger2 ctest binaries.

What one machine does with 7.25.20¶

The 7.25.20 side of the same run: 8 rounds (24 for the ctest binaries), mean of the rounds, on the machine described below.

7.25.5 against 7.25.4¶

The prices of 7.25.5, and what each one buys:

Every A/B of 7.25.5, change by change, with its rounds and spread, is in performance/c/README.md and in the CHANGELOG (“Performance, against 7.25.4”).

What one machine does with 7.25.5¶

Measured at tag time on the build of 7.25.5: 5 rounds (3 for perf_c_treedb and perf_auth_bff), mean of the rounds. One process; the event loop runs on one core, and Yuneta scales by running one yuno per core.

TLS with OpenSSL costs 24% of the round trips of plain TCP through the gobj stack (29,166 against 38,587). mbedTLS is not built on the measuring machine, so its figures are not in the report.

How the figures are measured¶

• The machine. One laptop, the same for every report until a report says otherwise: an Acer Nitro AN517-53 with an 11th Gen Intel Core i7-11370H (4 cores, 8 threads, up to 4.8 GHz), 31 GiB of RAM, a Samsung 990 PRO NVMe with ext4, Linux 7.0.0-34-generic, Ubuntu 26.04.1 LTS, glibc 2.43, gcc 15.2.0. Absolute figures compare only between reports of the same machine.

• The build. RelWithDebInfo (-O2 -g), gcc, fully static, OpenSSL, with CONFIG_DEBUG_TRACK_MEMORY on (every allocation is tracked).

• A/B. The module of the old release is compiled with the headers of the new one and linked before the same libraries; the two binaries run alternately, with a sync and a pause before each run. For timeranger2 the modules are padded to one size and linked four times at four addresses, because the address of the libraries alone moves an append by up to 2.5%. That is the A/B of 7.25.5. The A/B of 7.25.20 and of 7.25.21 builds each release whole from its own tree and runs the two binaries alternately, the order flipped every round: simpler, and blind to a change under ~2.5% on an append, which it says where it matters.

• Tag time. The benchmarks of performance/c/, in rounds, on the build of the release.

The benchmarks themselves are described in Benchmarks & Stress Tests.