Pachyderm Global ID — Concept Documentation
Pachyderm
Concept documentation with diagram
Information architecture judgment — knowing what info is worthy of being surfaced, how to structure it, what to write, and what can be distilled through a diagram. A novel distributed systems concept made intuitive through structure that builds a complete mental model without explanation volume.
The situation
Global ID is a single identifier that propagates automatically across every Pachyderm repo and pipeline in a DAG when a commit triggers a run — a concept with no direct analogue elsewhere, and one that's easy to misread as simpler than it is.
The task
Make an abstract, novel distributed systems concept intuitive for users ranging from data scientists to ML engineers, without relying on explanation volume.
What I did
Chose information architecture over prose. Built a dual-scope table distinguishing the global ID perspective from the individual commit perspective. Grounded the CLI output and diagram in Pachyderm's canonical getting-started example — a pipeline users already knew — so the abstract concept lands on familiar terrain. Designed a DAG diagram showing exactly how a put file operation propagates commits across Spec, User, and Meta repo types — with a legend that disambiguates commit types cleanly.
What happened
A concept that would otherwise produce confident confusion becomes an intuitive mental model on first read. The artifact demonstrates what information architecture judgment looks like in practice: knowing what to surface, in what order, and when a diagram does more work than a paragraph ever could.
Conceptual writingInformation architectureMental model constructionDiagram design