Skip to main content

Spike 00 — Build PyMOL from this tree so import pymol works (macOS arm64)

Status: SUCCESS. PyMOL 3.2.0a0 builds and imports on darwin 24.6.0 / arm64, headless, with no Qt, no PySide, no GLUT, no display, and with MMTF/BCIF support enabled. Everything below was executed on this machine. Every transcript is real output, copied verbatim. Follow-on doc: docs/build-and-tooling.md (the prior analysis this spike executes and corrects).

STATUS — read this before §6.1

The recipe shipped. scripts/bootstrap.sh IS this document, and cites it in six comments, by section: §1, §2.1 (why catch2 is deliberately absent), §2.2 (the vendored mmtf-cpp headers), §3, §3.1 twice (--no-build-isolation, and the keg-only libxml2) and §4.3 (copying _cmd*.so into the source tree). scripts/dev-bridge.sh cites §3 and scripts/doctor.mjs cites §5.2’s 877-byte MMTF round trip. The venv it builds is packages/bridge/.venv — substitute that for every <SCRATCH>/venv/bin/python below; the scratchpad interpreter in §0/§3/§8 is gone.

§6.1 is AMENDED, not overturned — and getting this backwards costs a wave

_cmd._draw() SEGFAULTS without a GL context” is true as stated but reads as “never call _draw”, and the architecture was written that way for a while. The precise rule, from 04-picking.md §1 and docs/code-ownership.md:103-107:
  • _cmd._draw segfaults only when options.no_gui == 0 (⇒ HaveGUI = pmgui = 1) and no GL context is current — at the glGetString on packages/engine/layer5/PyMOL.cpp:2307.
  • With a current GL context it is not merely safe, it is mandatory. ExecutiveDrawNow is the only caller of OrthoExecDeferred, and PyMOL_GetIdleAndReady only becomes true after PyMOL_Draw has set DrawnFlag three times. A bridge that never draws drains no deferred work at all: no clicks, no drags, no deferred cmd.png, no deferred ray. cmd.refresh() is not a substitute — CmdRefresh never sets DrawnFlag.
  • §6.1’s own remedy paragraph (“the rasterisation path for a headless backend is cmd.ray() + cmd.png() … not PyMOL’s GL renderer”) is therefore true only of a context-free bridge. The bridge that shipped creates a context first and drives _draw every tick.
§6.2 (C exit() skips atexit and Py_FinalizeEx) and §6.3 (_cmd._refresh does not exist) are unchanged and are cited from tenmol_bridge/dispatch.py:354 and policy/base.py:175. §7’s four recommendations are all done; their paths were webclient/… and are corrected in place below.

0. TL;DR

Three findings that change the bridge design are in §6. Read those.

1. Machine state (measured, not assumed)

python3 on PATH is a pyenv shim (/Users/amirangel/.pyenv/shims/python3) and /Users/amirangel/anaconda3 is also on PATH. setup.py:288-294 (is_conda_env()) changes prefix search when sys.prefix looks conda-ish. Always build from the explicit venv below, never from the ambient interpreter.

2. Native dependencies

2.1 Already installed via Homebrew — nothing had to be installed

brew install was NOT needed. If starting from a bare machine, the equivalent is:
catch2 is intentionally not installed: it is only used by --testing=true (packages/engine/layerCTest/Test.h:14 wants the Catch2 v2 umbrella header <catch2/catch.hpp>, and brew only ships v3). We do not build layerCTest.

2.2 The one dependency no package manager ships: mmtf-cpp

packages/engine/layer3/MoleculeExporter.cpp:16 does #include <mmtf.hpp>. It is header-only, it is not in Homebrew, and it is not vendored in this tree (packages/engine/include/ has only pymol/ and tnt/). Vendored out of tree (so packages/engine/include/ stays pristine — hard rule: do not touch upstream files):
setup.py scans each PREFIX_PATH entry for an packages/engine/include/ subdir (setup.py:783-796), so adding …/deps/mmtf-cpp to PREFIX_PATH is enough — no -I hacking, no source edits. This is a strict improvement on build-and-tooling.md’s recommended --config-settings use-msgpackc=no workaround, which loses MMTF and BCIF I/O and causes 5 test errors. With the vendored headers we get full MMTF/BCIF and 4 of those 5 errors disappear (§5).

3. The exact reproducible build

Tail of the real log ($SCRATCH/build-real-1.log):

3.1 Why each argument

3.2 Actual compile line produced (from the log, one object)

Note _PYMOL_NO_MSGPACKC is absent — msgpack is on. _PYMOL_NO_MAIN is present — no GLUT main. 254 .o files (find build -name '*.o' \| wc -l254).

4. Errors hit, and how they were resolved

4.1 'mmtf.hpp' file not found — the only genuine build failure

Reproduced deliberately with the naive command (default use-msgpackc=guess, no mmtf-cpp on PREFIX_PATH), i.e. what a new dev would type first:
Chain: options.use_msgpackc = "guess" (setup.py:198) → guess_msgpackc() (setup.py:297-312) sees brew’s /opt/homebrew/include/msgpack/version_master.hpp with MSGPACK_VERSION_MAJOR > 1 → returns "c++11"_PYMOL_NO_MSGPACKC is not defined → MoleculeExporter.cpp:15’s #ifndef _PYMOL_NO_MSGPACKC block is live → needs mmtf.hpp. Resolution: vendor mmtf-cpp headers out of tree and put them on PREFIX_PATH (§2.2). (The alternative, --config-settings use-msgpackc=no, also builds but costs MMTF and BCIF.)

4.2 Non-errors that look like errors

  • catch2 missing — expected; only --testing=true needs it, and brew’s v3 would not work anyway (packages/engine/layerCTest/Test.h:14 wants the v2 header). Not a blocker.
  • build/ appears in the repo (99 MB).gitignore:4 already ignores build, and .gitignore:3 ignores generated. Left in place so incremental rebuilds are ~1–2 s.
  • packages/engine/modules/pymol.egg-info/ appears in the repo and is NOT gitignored — this does dirty git status. Removed manually after each build (§3 step 4). See §7 for the recommended permanent fix.
  • packages/engine/testing/timings.tab appears after running the test suite — also not gitignored. Removed.

4.3 Why non-editable install

pip install -e . drops a 10 MB packages/engine/modules/pymol/_cmd.cpython-313-darwin.so into the source tree, and .gitignore does not ignore *.so (.gitignore:1-5 is only *.pyc, *.d, generated, build, .vscode). That violates “the repo must stay clean”. A plain pip install . copies everything into site-packages and leaves packages/engine/modules/ untouched. Consequence for downstream agents: edits to packages/engine/modules/pymol/*.py in the repo will NOT be picked up until you re-run the install command in §3 (~16 s). If you need a live-editable PyMOL, use -e and add packages/engine/modules/pymol/_cmd*.so + packages/engine/modules/pymol.egg-info/ to .git/info/exclude first (that file is local-only and git never merges it).

5. Verification — real transcripts

5.1 The required one-liner

Without PYTHONUNBUFFERED=1 (or an explicit flush) that exact command prints NOTHING and still exits 0. That is not a build defect — see §6.1. It is the single most likely thing to make another agent believe the build is broken when it is not. Note finish_launching’s docstring says “THIS IS NOT SUPPORTED ON macOS” (packages/engine/modules/pymol/__init__.py:435). It nevertheless works here, in the -cq (no-GUI) mode. The supported entry point for the bridge is pymol2.PyMOL(), verified next.

5.2 pymol2 headless, full pipeline including MMTF round-trip

Ray tracing, PNG encoding (libpng), surface computation and MMTF save/load all work with zero GL context.

5.3 pymol2.SingletonPyMOL and the _cmd bridge symbols

_cmd._refresh does not exist and never has. The complete set of underscore-prefixed “raw hook” entry points in _cmd is exactly:
The refresh equivalents are the ordinary (non-underscore) _cmd functions that take the _COb handle: _cmd.refresh(_COb), _cmd.refresh_now(_COb), _cmd.refresh_later(_COb) — all present, and used by packages/engine/modules/pymol/internal.py:551-558. The bridge should call cmd.refresh() / cmd.refresh_now(), not a nonexistent _cmd._refresh. Also newly surfaced and relevant to the bridge: _cmd._pushValidContext / _cmd._popValidContext (not mentioned in any prior doc) — these are how PyMOL is told a GL context is current.

5.4 Companion modules and data files

All 44 GLSL shader sources are installed and readable at $PYMOL_DATA/shaders — that is where the @tenmol/viewer shader port should read them from. setting_help.csv is there too (settings panel parity).

5.5 Full upstream test suite

The two non-passes, in full:
  • testglTF — needs the external collada2gltf binary; no Homebrew formula exists. Environmental, not a build defect. Only affects cmd.save('*.gltf').
  • symop_py.test_commands — a ray-traced image byte-diff; ray-traced images are not bit-stable on darwin/arm64. Do not gate CI on image-diff tests on macOS.
Compare against build-and-tooling.md §2.6.5, which reported failures=1, errors=5 using use-msgpackc=no. The 4 MMTF errors (testMMTF, testMMTFExportEmpty, testMMTFExportSele, testSave_symmetry__mmtf) and the BCIF pytest failures are gone because we vendored mmtf-cpp. That is the concrete payoff of §2.2. Also correcting build-and-tooling.md §2.6.6: pymol -ckq … --run all returned EXIT=2, not 0, when tests failed. Exit codes are not reliably meaningless — but still parse the output, don’t trust the code.

5.6 Repo cleanliness

No upstream file was modified. The only new file is this document.

6. Findings that change the bridge design — READ THESE

6.1 _cmd._draw() SEGFAULTS without a GL context

This is the big one. p.button(), p.drag(), p.reshape(), p.idle(), p.getRedisplay() are all safe headless. p.draw() is not.
Backtrace from the macOS crash report (~/Library/Logs/DiagnosticReports/Python-2026-07-30-173634.ips):
Implication for the architecture: the bridge must never call _cmd._draw / SingletonPyMOL.draw(). The rasterisation path for a headless backend is cmd.ray() + cmd.png() (verified working, 6 ms for 120×90 in §5.2), or geometry extraction — not PyMOL’s GL renderer. Any design doc that proposes “call _draw and scrape the framebuffer” is proposing a guaranteed segfault unless it first creates and makes-current a real GL context (and then _cmd._pushValidContext). Note this crash takes down the whole process — there is no exception to catch — so a single bad RPC would kill the user’s session.

6.2 PyMOL tears the process down with C exit(), skipping Python shutdown

atexit handlers do not run and buffered stdout is discarded:
END OF SCRIPT printed, so the script ran to completion — but 10 (buffered stdout) and ATEXIT RAN were both lost. sys.stdout is not replaced (verified: sys.stdout is soTrue); the process is simply terminated from PyMOL’s thread via C exit() (packages/engine/layer5/main.cpp:221, packages/engine/layer1/P.cpp:359/369/1488), which skips Py_FinalizeEx. Implications: (a) the bridge daemon cannot rely on atexit or finally for cleanup — flush/persist eagerly; (b) any log line the bridge writes must be flushed or written to an unbuffered/line-buffered stream; (c) pymol.finish_launching is the wrong entry point for a long-lived server — use pymol2.PyMOL(), which does not install that teardown path (§5.2 ran to completion and returned control normally).

6.3 _cmd._refresh does not exist

Detailed in §5.3. Use cmd.refresh() / cmd.refresh_now() / cmd.refresh_later().

7. Recommendations for files I do not own

These are reported, not applied.
  1. .git/info/exclude (local, never merged, not a repo file) should gain:
    Without this, every pip install leaves packages/engine/modules/pymol.egg-info/ as untracked and git status is never clean. Whoever owns dev-environment setup should do this in scripts/bootstrap.sh.
  2. scripts/bootstrap.sh (owner: build/tooling agent) should use --config-settings use-msgpackc=c++11 with an out-of-tree vendored mmtf-cpp on PREFIX_PATH, not use-msgpackc=no. The no path silently disables MMTF and BCIF file I/O, which are rows in feature-parity.md.
  3. docs/architecture.md (owner: architecture agent) — if it describes driving _cmd._draw from the bridge, that is unimplementable headless (§6.1) and needs revising.
  4. CI: do not gate on ray-traced image-diff tests on macOS/arm64 (§5.5), and do not use --testing=true on a brew-only mac (needs Catch2 v2).

8. Rebuild cheat-sheet

Logs kept in the scratchpad: build-real-1.log (successful build), build-default-fail.log (the mmtf.hpp failure), testsuite.log, draw-crash.log.