You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Wiki and Obsidian exporters rewrite every page on every run, even when nothing changed (31,417 files / 137 MB per export, ~7 GB/day here) #3060
to_wiki() and to_obsidian() regenerate the full page set on every call, and they write
every page unconditionally:
graphify/wiki.py deletes every *.md in wiki/ up front (for old_article in out.glob("*.md"): old_article.unlink()),
which guarantees that all of them are then written from scratch even when the content is identical;
graphify/export.py::_owned_write calls target.write_text(content, ...) without ever comparing
against what is already on disk.
Nothing in either path asks "did this page actually change?".
Why it hurts
An export is triggered by any graph.json change, so on an always-on setup it runs constantly.
Measured on one vault (26.7k nodes, 2.3k communities), watched by graphify watch --semantic with
a systemd path unit re-running graphify export wiki + graphify export obsidian on every graph.json change:
pages regenerated per export
31,417 (2,354 wiki articles + 29,063 Obsidian notes)
bytes rewritten per export
137 MB
exports in a measured 24 h
51
disk written per day just for the views
~7 GB
Between two consecutive exports the graph moved by ~20 nodes out of 26,500 — and all 31,417 pages
were rewritten anyway. The disk cost is the smaller half of it: every rewrite is also a
modification event, so Obsidian re-indexes the whole vault, sync clients re-upload it, and any
inotify-driven pipeline downstream re-fires, dozens of times a day, for content that did not change.
What fixes it
Compare before writing. Concretely, in a fork of v8:
paths.write_text_if_changed(path, text) -> bool — writes only on a difference and reports
whether it wrote. Compare decoded text, not bytes: on Windows write_text/read_text
translate newlines, so a byte compare reports "changed" for every file.
_owned_write skips identical notes but still records them in _written — that list is
both the ownership manifest and the prune-exclusion set (export obsidian: never prunes notes for removed nodes, so re-export merges old and new graphs #1896), so dropping an unchanged note
from it would disown the note and delete it as stale on the same run. to_canvas() and the
manifest go through the same helper.
In to_wiki, replace the up-front unlink with a sweep at the end that removes only what the
run did not produce. Same orphan guarantee for relabelled communities, and it also closes a
window where a reader sees an empty wiki/ mid-export.
Measured result of that change, on the same vault
Export run twice with no graph change: git-independent mtime comparison across all 31,417
files → 0 files rewritten on the second run. Wall clock 13.2 s → 4.9 s.
A real production export, with a real graph change (26,713 → 26,730 nodes, communities
re-clustered): 2,075 of 2,354 articles and 25,552 of 29,075 notes left untouched — 3,802 files
written instead of 31,429, an 88% cut on an export that genuinely had work to do.
Happy to open a PR against v8 with the implementation and the mtime-based tests
(12 of them: the primitive, both exporters' no-op re-run, the page that really did change, the
manifest/prune interaction, orphan removal after a relabel, and the no-empty-window guarantee) —
they are already written and green in our fork.
The "Why it hurts" section says the vault was "watched by graphify watch --semantic". That flag does not exist upstream — it is my fork's. Checked against v0.9.50 (43d54acbfa9e731f7a592bb582c1f4b9d48ed73e): --semantic appears zero times in graphify/watch.py, graphify/cli.py and graphify/__main__.py, and the watch sub-command parses no options at all. So a reader who copies that command gets a plain watch, and cannot reproduce my setup from it.
What the measurement actually needs, and it is not that flag. The numbers come from any arrangement where graphify export wiki and graphify export obsidian re-run on every graph.json change. Upstream that is a path-triggered or post-commit re-export; the thinking pass is not part of it. The defect reported here is in the exporters and is independent of what triggers them:
wiki.py deletes every article before writing any
export.py::_owned_write writes unconditionally, with no comparison
Both hold at v0.9.50 regardless of how the export is invoked.
The measured figures — 31,417 pages and ~137 MB per export, ~51 exports observed in 24 hours — stand. Only the command line naming the trigger was wrong, and it was wrong by describing my own build as yours.
Activity
Azeem1985 commentedon Aug 27, 2026
Correcting one line of my own report.
The "Why it hurts" section says the vault was "watched by
graphify watch --semantic". That flag does not exist upstream — it is my fork's. Checked againstv0.9.50(43d54acbfa9e731f7a592bb582c1f4b9d48ed73e):--semanticappears zero times ingraphify/watch.py,graphify/cli.pyandgraphify/__main__.py, and thewatchsub-command parses no options at all. So a reader who copies that command gets a plain watch, and cannot reproduce my setup from it.What the measurement actually needs, and it is not that flag. The numbers come from any arrangement where
graphify export wikiandgraphify export obsidianre-run on everygraph.jsonchange. Upstream that is a path-triggered or post-commit re-export; the thinking pass is not part of it. The defect reported here is in the exporters and is independent of what triggers them:wiki.pydeletes every article before writing anyexport.py::_owned_writewrites unconditionally, with no comparisonBoth hold at
v0.9.50regardless of how the export is invoked.The measured figures — 31,417 pages and ~137 MB per export, ~51 exports observed in 24 hours — stand. Only the command line naming the trigger was wrong, and it was wrong by describing my own build as yours.
🤖 Generated with Claude Code
fix(export): stop rewriting unchanged wiki/Obsidian pages on every run (
safishamsi commentedon Sep 23, 2026
Fixed in v0.9.67 via #3760 — export no longer rewrites unchanged wiki/Obsidian pages. Thanks @abhay-codes07.