Pachyderm Production Deployment System

Pachyderm ran on Kubernetes, backed by cloud object storage and PostgreSQL, with a non-trivial production deployment surface: ingress controllers, load balancers, managed databases, IAM and service account configuration, and persistent volume provisioning — all varying by cloud provider. No prior documentation system existed for this deployment surface. Later, a proxy layer was introduced that consolidated all external traffic to a single port, replacing the multi-port architecture the existing documentation was built around.

Build a complete documentation system that would take operators from architectural understanding to a working production deployment on their cloud provider of choice — then evolve it through its infrastructure change.

Designed a cloud-agnostic high-level architecture diagram organized around the three layers operators needed to understand: clients, Pachyderm containers, and storage. Built the infrastructure and ingress documentation covering load balancer setup, Kubernetes service topology, and identity provider integration. Wrote full production deployment guides for AWS (EKS), GCP (GKE), and Azure (AKS) — each covering cluster creation, managed PostgreSQL setup (RDS, CloudSQL, PostgreSQL Server), IAM and service account configuration, persistent volume provisioning, Helm values, and cluster communication. When the proxy architecture shipped, wrote the concept introduction explaining the shift, designed new topology diagrams with port-level detail organized by Kubernetes layer, and produced updated multi-cloud Helm examples. Every piece of this surface required understanding the infrastructure well enough to make the right recommendations.

Complete deployment documentation system shipped and maintained as sole technical writer across a significant architectural change. The cloud-agnostic architecture diagram was carried forward into HPE's official product documentation after the acquisition and remains live on the HPE Support Center today.