Spike 08 — Native (C++) wave 2: change counters, pick data, missing instances
Status: IMPLEMENTED AND VERIFIED. Plancode-ownership.md §4 Task 6 (ReprVersion),
§4 Task 3 (pick data) and the instance half of defect D6 are done.
Every number, transcript and table below was produced on this machine by running the scripts
named in §7. Nothing is inferred from source reading alone. Where something is not verified it
says so in those words.
STATUS — re-verified against the C++ on 2026-08-02
packages/engine/layer4/CmdWebGeometry.cpphas been extended twice since this spike was written, so §0’s diffstat is stale. What is still exactly true, and what moved: STILL TRUE, re-measured today.CHANGED — §0 and §6 understate the file.
- The merge-surface guarantee, to the line. Counting lines inside
tenmol web client -- BEGIN … ENDblocks today:Executive.cpp10 blocks / 64 lines,ExecutiveDef.h1 block / 19 lines,Cmd.cpp2 blocks / 10 lines = 93 guarded lines, zero unterminated blocks — the same three numbers §6 recorded.BEGINandENDcounts match in every file. Zero upstream lines removed._cmd.web_get_versionspayload shape (§2). Live keys today:changed, counters, objects, recomputed, rehashed, serial, walks; per objectenabled, n_atom, n_state, reps, type, version. Exactly §2’s contract._cmd.web_resolve_pickpayload shape (§3.1). Live keys today:atom, bond, bond_atoms, coord, describe, index, message, object, ok, pick_kind, selection, state, status.resolve(1tii, index=10, bond=cPickableAtom)→status ok,selection u11,describe /u///HOH11/O,pick_kind atom.- §5.2’s status table. Re-run on 1tii with every rep shown:
labels → unsupported;slice/volume/callback/cgo→not-built;cell→okfromCoordSet::UnitCellCGO;extent→okfromCObject::ExtentMin/ExtentMax;ellipsoids→emptyon a structure with no ANISOU; everything elseok.STALE PATHS IN THE C++, reported not applied (
CmdWebGeometry.cppis now 2,660 lines, not the “+1012” of §0 (spike 06 created it at 1,451). The three upstream files are untouched beyond the 93 guarded lines above, so the growth is entirely inside the file upstream does not have.- A whole object type was added that §5 does not mention:
ObjectMesh— theisomesh/isodotOBJECTS the density wizard builds. They own noCoordSetand noRep, so nothing in §5’s molecular path could reach them. There are now two arms:extractObjectMesh()readsObjectMeshState::{N,V,VC}and reusesextractMesh’skind: "mesh"envelope, addingsource: "ObjectMeshState::V",levelandmap_name; and a matching arm in the object signature soweb_get_versionslists amesh|<state>row for them. Measured today onpept.pdb+map_new+isomesh/isodot:Widthis now hashed in TWO signature arms, and the second one is a bug this spike’s design would have shipped. §2.1 says the signature is a content hash of the rep’s CPU geometry — butmesh_widthchanges no vertex, soweb_get_versionsnever bumpedmesh|<state>and the client’s version poll, which gates every geometry pull, never refetched. Measured in the code comment atCmdWebGeometry.cpp:1776: Mode P went 22,780 → 36,444 ink pixels while Mode G stayed at 22,771 withgeometryFramesstuck at 1. Both arms now hash it —repSignature’scRepMesharm hashesRepMesh::Width, and theObjectMesharm hashes themesh_width/dot_widthsetting (an object-level setting, not state). Confirmed live today:set mesh_width, 3, mshtakesmesh|0from{version: 1}to{version: 2}withchanged=True. This is the same class of hole §5.1 already recorded fordash_radius/dash_width; treat “a setting that changes appearance but no vertex” as the standing failure mode of a pure content hash.- §6’s mirror table lists one mirror; there are ten, all in
namespace mirrorat the top of the file:RepSurface,RepCartoon,RepCylBond,RepWireBond,RepRibbon,RepNonbonded,RepNonbondedSphere,RepEllipsoid,RepMesh(all from spike 06 §8.1) plus this spike’sRepDistLines.extractObjectMeshneeds no mirror:ObjectMesh.his a public header.- §7’s
t*.pyprobe scripts lived in a session scratchpad and are gone. The behaviour they proved is pinned inpackages/bridge/tests/instead —test_p11_geom.pyis the isomesh /ObjectMeshsuite.packages/engine/is upstream and not this file’s to edit):CmdWebGeometry.cpp:8,:38,:2087and:2421still citedocs/webclient/spikes/…, the pre-monorepo path. The files are atdocs/spikes/….
0. TL;DR
The CmdWebGeometry.cpp row below is the size at the time of this spike. The file is 2,660
lines today; the three upstream files are unchanged. See the STATUS block above.
GL-free scorecard. The backend needed a GL context for (a) Mode P rasterising and (b) the pick
pass. This spike removes the need for (b): the geometry payload already carries stable
(atom index, bond index) per vertex/instance, and §3 proves that resolving those client-side
reproduces the backend pick exactly, on the same screen positions, for both an instance rep and
a triangle-mesh rep. Nothing in web_get_versions or web_resolve_pick touches GL — both were run
under pymol2.PyMOL() with no_gui=1 and no context at all.
1. What was added, and where
1.1 struct CExecutive (packages/engine/layer3/ExecutiveDef.h)
Four monotonic unsigned counters. They are hints, not content hashes:
1.2 Bump sites (packages/engine/layer3/Executive.cpp, 10 sentinel blocks, 0 lines removed)
The plan named five sites. Three of them are the right ones; two would have missed the single most
common change of all. What actually landed, and why:
1.3 SettingRec::setChanged() was deliberately NOT patched
The plan offered it as optional. It is not needed and it is the worst place to touch: Setting.h
is included by ~everything, so a field there is a full-tree rebuild at every upstream merge, and
setChanged() fires for every setting including bg_rgb, ray_trace_mode and the movie
panel — i.e. it would manufacture false wake-ups. Instead, the 113 ExecutiveInvalidateRep call
sites inside packages/engine/layer1/Setting.cpp are what actually rebuild a rep, and the single bump at
ExecutiveInvalidateRep covers all of them. Measured:
1.4 packages/engine/layer4/Cmd.cpp — 4 lines, both inside the sentinel blocks WP-26 already created
2. Task 6 — _cmd.web_get_versions()
active: false and a bumped version. That is precisely the signal D1 needs.
2.1 The counters are cheap and exact
counters alone is a hint. The versions are derived from a 64-bit FNV-1a content signature of
each rep’s CPU geometry, computed only when a counter moved. The signature is deliberately
pointer-free: Rep* is recycled by the allocator across a rebuild, and a CGO’s out-of-line
floatdata blocks move, so hashing either would produce phantom changes. For CGO_DRAW_ARRAYS the
signature hashes mode, arraybits, nverts and the out-of-line data block — never the pointer
that holds it.
2.2 D1, reproduced and fixed
(object, rep, state) whose version differs
from the one I cached, refetch; for every one that is active: false, drop.” No content hash, no
get_vis()/get_state() fingerprint, no self-declared inexactness.
2.3 No false positives — 700 idle polls, zero
changed stays false):
2.4 The change the old poll cannot see
get_vis() returns the same dict before and after color red, resi 1-20; get_state() is
unchanged. This is the single row that makes a ReprVersion worth having, exactly as the plan says.
2.5 An optimisation that was measured, then REJECTED
Full disclosure, because it is the only place this design is knowingly slower than it could be.
The panel counter moves on every
SelectorTmp construction — count_atoms, iterate, select,
get_model all bump it — so any of those costs one 47 ms walk on a structure that size. The obvious
fix is to gate the expensive per-Rep hashing on counters 1..3 only (enable / name / rep) and let a
panel-only move do just the cheap object-list pass.
It was implemented, and then a differential harness killed it: two PyMOL instances, the identical
400-command random sequence, one polling gated and one polling force=1 (which always re-hashes).
create <name> onto an existing object name, whose atom count changed with
no counter but the panel one moving. That is a D1-class stale-geometry hole, so the conservative
rule stands: any counter moving triggers a full re-hash. rehashed is therefore always equal to
recomputed; it is kept in the payload so the decision stays observable.
With the fast path reverted, the same harness over 800 random commands:
n_atom/n_state are now read and compared before the rep
walk and are part of the object-level change test in their own right, because remove can drop an
atom that no built rep was drawing, leaving every rep signature identical.
For the bridge: pollweb_get_versionsat whatever rate you like when idle (1.5 µs), but treat arecomputed: trueas costing up to ~50 ms on a very large structure. That is the same moment the client is about to refetch geometry anyway.
3. Task 3 — pick data, and why the backend no longer needs a pick pass
3.1 What is shipped, and what is deliberately not
Not shipped: the pick colour.PickColorManager::colorNext (packages/engine/layer1/Picking.cpp:150-186) is a
per-frame draw-order counter whose reverse map holds raw CObject* and is invalidated on every
rebuild. It is meaningless outside the frame that produced it.
Shipped: the identifiers behind CGO_PICK_COLOR (packages/engine/layer1/CGO.h:150-151) — a 0-based atom index
inside the object plus a bond index or a cPickable_t sentinel, aligned per vertex and per instance
with the geometry buffers WP-26 already emits (spheres.pick, cylinders.pick1/pick2,
cones.pick, ellipsoids.pick, lines.pick1/pick2, crosses.pick, draw_arrays[].pick, and
surface.atom from RepSurface::AT).
_cmd.web_resolve_pick(_self._COb, object, index, bond, state=-1) turns one back into a selection,
with no GL context:
u11is byte-for-byte whatpackages/engine/layer1/SceneMouse.cpp:245 builds for a real backend pick (”%s%d" % (obj->Name, index + 1)). Half-bond picks additionally get bond_atoms and a ready
bond_selection:
3.2 The round trip, against a REAL GL pick
Ground truth is a genuineScenePicking.cpp pick-colour render + glReadPixels on the headless
CGL + FBO context from spikes/picking.md §3 (GL: 2.1 Metal - 89.4 / Apple M4 Max), read out
through _cmd.get_click_string. The client-side answer is produced in pure Python from only the
web_get_rep_geometry buffers plus packages/viewport/src/camera.ts’s view maths — it never looks
at the GL result.
3.3 Disagreement #1 — PyMOL’s pick has a ±7 px snap radius
A naive exact ray-sphere test scored 16/18. Both misses were pixels where the ray passed just outside the sphere the GL pick returned:SceneRenderPickingSinglePick reads a 15×15 px window around the click and walks outward in
square rings, taking the first non-zero pick index. So a PyMOL click snaps to anything within
~7 px (scaled by DIP2PIXEL). Porting that ring scan client-side:
For the WebGL agent: implement the outward ring scan withcRange = 7 * devicePixelRatio, in the same order (ringsd = 0..6, row-major within a ring). Without it, clicks near a silhouette or on a thin rep will differ from Mode P.
3.4 Disagreement #2 — the pick pass is FLAT-shaded: use the LAST triangle corner
A naive “nearest barycentric corner” surface picker scored 10/15, and the disagreements were always a neighbouring atom (360 vs 361, 31 vs 32, 345 vs 346). The pick pass is flat-shaded:SceneSetupGLPicking does glShadeModel(GL_FLAT) (packages/engine/layer1/Scene.cpp:5186), the pick colour
travels through gl_FrontColor precisely so that glShadeModel applies to it
(packages/engine/data/shaders/default.vs:15-16, “using the built-in allows to use glShadeModel”), and
ScenePicking.cpp:229-234 restores GL_SMOOTH afterwards with the comment “Picking changes the
Shading model to GL_FLAT” (cSetting_pick_shading defaults to 0, so the restore is to
GL_SMOOTH). A flat-shaded triangle takes the colour of its provoking vertex, which for
GL_TRIANGLES is the last one. Measured over 72 GL picks on the 1UBQ surface:
third = 72/72. So the rule is: for a triangle hit, report atom[index[3*t + 2]], not the
nearest corner. With that rule plus the ±7 px snap, the surface round trip is 15/15.
3.5 A finding the client must know: the backend cannot pick a surface by default
cSetting_pick_surface defaults to 0 (packages/engine/layer1/SettingInfo.h:812), and
RepSurface.cpp:1565 does I->shaderCGO->no_pick = !pick_surface. Verified: with the default,
clicking dead centre of an opaque 1UBQ surface returns type=none. The pick_surface, on setting
had to be switched on to obtain any ground truth at all in §3.2.
Client-side picking has no such limit — it can pick the surface whether or not the setting is on.
That is a behavioural difference from Mode P, and the client should honour pick_surface to stay
consistent, or the same click will select an atom in Mode G and nothing in Mode P.
3.6 What is still NOT verified
mesh,dots,ribbon,nonbondedandnb_sphereswere probed for backend pickability at a single centre pixel and all returnedtype=none; that is most likely because those reps are sparse/thin and the pixel missed, not a proven “unpickable”. Not investigated further, and therefore not verified either way.- Multi-object and multi-state scenes were not part of the pick comparison.
web_resolve_picktakes an explicit object name and state and was exercised withstate=-1; other states are unverified. - Orthoscopic projection was not exercised in the pick comparison (the view was perspective,
view[17] = -20). The ortho branch of the ray construction is unverified.
4. Task 3 (D6) — cone and ellipsoid instances
Both are decoded into flat typed instance buffers. Nothing is tessellated.4.1 Cones
cone[2]: CGO_CYLINDER and every *_CYLINDER* variant land in the same cones bucket
with r1 == r2. A client that implements the truncated-cone instance gets cylinders for free.
Per-instance layout (cones bucket):
4.2 Ellipsoids
1EJG, 367ANISOU records — the rep every text exporter drops (spike 03 §4.1: .wrl 234 bytes,
.pov 0, .obj 0 faces, .dae 0 geometry nodes):
axes is the 3×3 from CGOSimpleEllipsoid (packages/engine/layer1/CGO.cpp:4535): three non-unit row vectors
whose lengths are the semi-axes. Draw as a unit sphere transformed by that matrix, translated to
xyzr[0:3]. xyzr[3] is the ellipsoid scale factor PyMOL passes alongside.
axes=3303 is 9 × 367 = 3303 floats. The buffer is correct; the odd-looking number is just
9 per instance.
5. The rest of D6 — audit of the previously unsupported reps
5.1 Now supported
cellcomes out ofCoordSet::UnitCellCGO, which is aCGO_DRAW_ARRAYSwithmode = GL_LINESand 24 vertices (12 edges). It is not alinesbucket — the client reads it fromdraw_arrays, which it already supports. Measured on 1UBQ the box spans(0,0,0) .. (50.840, 42.770, 28.950)Å, i.e. it is the crystal cell with its origin corner at the model-space origin, and the extents equalcmd.get_symmetry()[0:3]exactly.extentis synthesised fromCObject::ExtentMin/ExtentMax, falling back toExecutiveGetExtentbecauseObjectMoleculenever latchesCObject::ExtentFlag. 12 segments in thelinesbucket. Verified equal tocmd.get_extent().dashes/angles/dihedralsread the rawfloat* VofRepDistDash/RepAngle/RepDihedral(one mirror covers all three;RepAngleusespymol::vla<float>, which holds exactly onefloat*,packages/engine/layer0/vla.h:42). The dash pattern is already baked intoVby the rep builder, so the client draws the segments verbatim — nodash_gaparithmetic client-side. The strongest validator available isI->ds == ds; it must equal theDistSetthe rep was reached through, or the call returnslayout-mismatch.cgo(standaloneObjectCGO) is harvested fromObjectCGO::State[s].origCGO.
RepDistDash::radius and ::linewidth are only assigned
inside render() (packages/engine/layer2/RepDistDash.cpp:340, RepAngle.cpp:70, RepDihedral.cpp:75), so on a
never-rendered rep — which is exactly the GL-free case this accessor exists for — they read 0.
The accessor now reports the two settings those lines read from (dash_radius, dash_width)
under radius/linewidth, and the raw members under rep_radius/rep_linewidth so a layout
drift stays visible. The signature hashes the settings for the same reason, so set dash_radius
bumps the version.
5.2 Full status table (1UBQ, one object, one state)
5.3 Still unsupported, and why — these are not laziness
labels— a label is not geometry.RepLabelholds text plus acSetting_label_*font stack; drawing it means rasterising a font, and the on-screen result is a screen-space billboard whose size does not scale with the model. The right answer is DOM/canvas text in the client driven bycmd.get_model()’slabelfield, not a vertex buffer. Shipping a vertex buffer here would be actively wrong.volume— a 3-D texture plus a transfer function. There is no CPU triangle geometry to extract at all; the whole rep is a ray-marching shader over aGL_TEXTURE_3D. Porting it means porting a volume renderer to WebGL2 and shipping the map, which is a work package of its own.slice— a textured quad sampled from a map, regenerated per frame from the slice plane. Same problem as volume: the payload is a texture, not vertices.callback— by definition arbitrary user C/Python code that issues GL calls at render time. There is nothing to serialise.
6. Upstream-merge surface
The CmdWebGeometry.cpp line in this diffstat is historical (see STATUS). The three upstream
rows and the 93-guarded-line audit below were re-run on 2026-08-02 and are exact.
tenmol web client -- BEGIN … END block. No build-file change was needed — setup.py:808-816
globs packages/engine/layer4/*.cpp.
packages/engine/layer4/CmdWebGeometry.cpp gained one more layout mirror, which is the only thing that can drift:
Run
t3_instances.py at each merge; a layout-mismatch names the file that changed.
7. Reproducing
Build (unchanged fromspikes/build.md):
<scratch>/nat/:
Test suites:
symop_py.TestBondSymOp.test_commands) and the single error
(exporting_py.TestExporting.testglTF) are byte-identical to the spike-00 baseline and predate
this work. requests and biopython must be present in the venv or the suite reports 3–4 extra
import errors that have nothing to do with these changes.
8. What this leaves for the other agents
- Bridge — replace the 4 Hz
get_vis()+get_state()+content-hash fingerprint withweb_get_versions(). Cache per(object, rep, state)version; refetch only on a bump; drop onactive: false. That is D1, done properly. - Bridge / protocol — plumb
spheres.pick,cylinders.pick1/pick2,cones.pick,ellipsoids.pick,lines.pick1/pick2,crosses.pick,draw_arrays[].pickandsurface.atomthrough the wire format; add aresolve_pickRPC over_cmd.web_resolve_pick. - WebGL — add the
coneandellipsoidinstance draws (packages/viewport/src/modeG/instances.tscurrently lists both as “NOT DRAWN — reported as a fallback reason”). The buffer layouts are §4. Then implement picking with both rules from §3.3/§3.4 — thecRange = 7outward ring scan and the last-triangle-corner convention — or clicks will disagree with Mode P near silhouettes. - WebGL —
cellarrives as adraw_arraysblock withmode = GL_LINES, not in thelinesbucket;extent,dashes,angles,dihedralsall arrive in thelinesbucket. - Product — decide whether Mode G should honour
pick_surface(§3.5). Client-side picking is strictly more capable than the backend here, which is a parity difference, not a parity gap.