GROVE.md · grove

12.4 KB334 lines

1	# Grove
2
3	Monorepo-first hosting platform powered by Mononoke.
4
5	Grove is a GitHub-like web platform for hosting, browsing, reviewing, and building code. Built on Meta's Mononoke source control server, it supports both standard Git and Sapling clients.
6
7	Live at [grove.host](https://grove.host)
8
9	---
10
11	## Architecture
12
13	```
14	┌──────────────┐
15	│ grove.host │ Caddy: auto-TLS + routing
16	└──────┬───────┘
17	┌───────────────┼───────────────┐
18	▼ ▼ ▼
19	┌─────────────┐ ┌────────────┐ ┌──────────────┐
20	│ Hub API │ │ Grove API │ │ Grove Web │
21	│ (identity) │ │ (repos/CI) │ │ (Next.js) │
22	│ :4000 │ │ :4000 │ │ :3000 │
23	└─────────────┘ └─────┬──────┘ └──────────────┘
24	│
25	┌─────┴──────┐
26	│ Bridge │
27	│ :3100 │
28	└─────┬──────┘
29	┌───────────┴───────────┐
30	▼ ▼
31	┌──────────────┐ ┌──────────────┐
32	│ Mononoke │ │ Mononoke │
33	│ SLAPI :8443 │ │ Git :8080 │
34	└──────────────┘ └──────────────┘
35	```
36
37	### Components
38
39	\| Component \| Tech \| Port \| Purpose \|
40	\|-----------\|------\|------\|---------\|
41	\| Caddy \| Go \| 80/443 \| Reverse proxy, auto-TLS (Let's Encrypt) \|
42	\| Hub API \| Node.js/Fastify \| 4000 \| Users, WebAuthn auth, orgs, JWT issuance \|
43	\| Grove API \| Node.js/Fastify \| 4000 \| Repos, file browsing, diffs, Canopy CI/CD \|
44	\| Grove Web \| Next.js 15/React 19 \| 3000 \| Browser UI \|
45	\| Grove Bridge \| Mononoke binary \| 3100 \| Bridge between API and Mononoke \|
46	\| Mononoke SLAPI \| Rust \| 8443 \| EdenAPI server (Sapling push/pull), mTLS \|
47	\| Mononoke Git \| Rust \| 8080 \| Git HTTP protocol \|
48	\| Docker Registry \| registry:2 \| 5000 \| Local image registry for Canopy CI \|
49
50	### Product Subdomains
51
52	\| Subdomain \| Purpose \|
53	\|-----------\|---------\|
54	\| `grove.host` \| Main app — repos, diffs, settings \|
55	\| `canopy.grove.host` \| CI/CD — pipeline runs, build logs \|
56	\| `ring.grove.host` \| Instance logging and monitoring \|
57
58	---
59
60	## Authentication
61
62	Grove uses WebAuthn passkeys (no passwords). Flow:
63
64	1. User registers/logs in with a passkey (biometric, hardware key, etc.)
65	2. Hub API issues a JWT (7-day expiry) signed with a shared `JWT_SECRET`
66	3. Grove API verifies the JWT on each request
67	4. Personal Access Tokens (PATs) are supported for CLI and API access
68
69	Auth endpoints live in hub-api. Grove API only verifies tokens.
70
71	---
72
73	## Features
74
75	### Code Hosting
76	- Push/pull via Git (`git clone http://grove.host:8080/<repo>`)
77	- Push/pull via Sapling (`sl clone https://grove.host:8443/edenapi/<repo>`)
78	- File browser with directory navigation and syntax highlighting
79	- Commit history with pagination
80	- Blame view
81	- Branch listing (Mononoke bookmarks)
82	- README rendering (Markdown + syntax highlighting)
83	- Private repos with org-based access control
84
85	### Code Review (Diffs)
86	- Create diffs from the web UI (select a branch → opens against default branch)
87	- Conversation view with threaded comments
88	- Changes view with unified diff rendering
89	- Land (merge) diffs via Mononoke pushrebase
90	- Close and reopen diffs
91	- Reviews with approve/request-changes
92
93	### Canopy CI/CD
94	- YAML pipeline definitions (`.canopy/` directory)
95	- Docker-based step execution
96	- Real-time log streaming via SSE
97	- Push-triggered pipelines (polls Mononoke bookmarks)
98	- Manual pipeline triggers
99	- Pipeline secrets management
100	- Build history and status indicators
101
102	### Organizations
103	- Create organizations, add/remove members
104	- Org-owned repositories
105	- Access control for private repos
106
107	### Ring (Instance Logging)
108	- Log ingestion API (JSON + plain text)
109	- Per-repo and global log views
110	- Real-time log polling
111	- Accessible at `ring.grove.host`
112
113	### CLI (`grove`)
114	- `grove auth login` — Browser-based WebAuthn login
115	- `grove clone <repo>` — Clone a repository
116	- `grove init <name>` — Create and initialize a new repo
117	- `grove repo list` — List repositories
118	- `grove repo create <name>` — Create a new repo
119	- Config stored in `~/.grove/config`
120
121	---
122
123	## Quick Start
124
125	### Prerequisites
126
127	- Docker & Docker Compose
128	- Node.js 22+ (for local dev)
129	- Git
130
131	### 1. Build Mononoke Docker Image
132
133	```bash
134	cd sapling
135	docker build -f ../docker/Dockerfile.mononoke -t grove/mononoke .
136	```
137
138	> First build takes 1-2+ hours (compiles Mononoke from source). Subsequent builds are cached.
139
140	### 2. Import a Repository
141
142	```bash
143	./scripts/import-repo.sh https://github.com/your/repo.git myrepo
144	```
145
146	### 3. Start Everything
147
148	```bash
149	cd docker
150	docker compose up
151	```
152
153	### 4. Access
154
155	- Web UI: http://localhost:3000
156	- API: http://localhost:4000
157	- Git clone: `git clone http://localhost:8080/myrepo`
158
159	### Local Development (Without Docker)
160
161	```bash
162	# Hub API
163	cd hub-api && npm install && npm run dev # http://localhost:4001
164
165	# Grove API
166	cd api && npm install && npm run dev # http://localhost:4000
167
168	# Web UI
169	cd web && npm install && npm run dev # http://localhost:3000
170	```
171
172	### Local Subdomain Dev (For Canopy/Ring Auth)
173
174	Add to `/etc/hosts`:
175
176	```text
177	127.0.0.1 grove.test
178	127.0.0.1 canopy.grove.test
179	127.0.0.1 ring.grove.test
180	```
181
182	Then `npm run dev:local` and open:
183	- Grove: `http://grove.test:3000`
184	- Canopy: `http://canopy.grove.test:3000`
185	- Ring: `http://ring.grove.test:3000`
186
187	---
188
189	## Project Structure
190
191	```
192	grove/
193	├── GROVE.md # This documentation
194	├── AUDIT.md # Codebase + server audit
195	├── api/ # Grove API server
196	│ └── src/
197	│ ├── server.ts # Fastify entry point
198	│ ├── auth/middleware.ts # JWT verification
199	│ ├── routes/
200	│ │ ├── repos.ts # Repo browsing, CRUD
201	│ │ ├── diffs.ts # Code review (diffs, comments, reviews)
202	│ │ ├── canopy.ts # CI/CD pipeline management
203	│ │ └── ring.ts # Instance log ingestion + viewing
204	│ └── services/
205	│ ├── database.ts # SQLite schema + migrations
206	│ ├── bridge.ts # Mononoke Bridge client
207	│ ├── canopy-runner.ts # Docker-based CI execution
208	│ ├── canopy-poller.ts # Push event polling
209	│ ├── canopy-events.ts # SSE event bus
210	│ └── mononoke-provisioner.ts # Dynamic repo config
211	├── hub-api/ # Hub API server (identity)
212	│ └── src/
213	│ ├── server.ts # Fastify entry point
214	│ └── routes/
215	│ ├── auth.ts # WebAuthn, JWT, PATs
216	│ ├── instances.ts # Instance management
217	│ └── orgs.ts # Organization CRUD
218	├── web/ # Next.js frontend
219	│ ├── app/
220	│ │ ├── layout.tsx, page.tsx # Root layout + dashboard
221	│ │ ├── login/ # WebAuthn login
222	│ │ ├── new/ # Create repository
223	│ │ ├── [owner]/[repo]/ # Repo pages (files, commits, blame, diffs)
224	│ │ ├── canopy/ # CI/CD pages
225	│ │ ├── ring/ # Instance logging pages
226	│ │ └── components/ # UI components
227	│ ├── lib/
228	│ │ ├── api.ts # Typed API client
229	│ │ ├── auth.tsx # Auth context + hooks
230	│ │ └── theme.tsx # Dark/light mode
231	│ └── middleware.ts # Subdomain routing
232	├── cli/ # Grove CLI
233	│ └── src/commands/ # Command modules
234	├── addons/ # Sapling integrations
235	│ ├── isl/ # Interactive Smartlog UI
236	│ ├── isl-server/ # ISL backend
237	│ ├── vscode/ # VS Code extension
238	│ ├── shared/ # Shared utilities
239	│ └── components/ # ISL components library
240	├── docker/ # Development Docker setup
241	│ ├── docker-compose.yml # Dev stack
242	│ └── Dockerfile.* # Build files
243	├── hub/ # Production setup
244	│ ├── docker-compose.yml # Prod stack (8 services + Caddy)
245	│ └── Caddyfile # Reverse proxy config
246	└── scripts/ # Utility scripts
247	├── import-repo.sh # Git → Mononoke importer
248	└── generate-certs.sh # TLS cert generator
249	```
250
251	---
252
253	## Database
254
255	### Grove API (SQLite: grove.db)
256
257	\| Table \| Purpose \|
258	\|-------\|---------\|
259	\| `users` \| Local user records (synced from hub JWT claims) \|
260	\| `orgs` \| Local org records \|
261	\| `repos` \| Repository metadata (owner, name, branches, visibility) \|
262	\| `diffs` \| Code review diffs (title, commits, status) \|
263	\| `reviews` \| Diff reviews (approved/changes requested) \|
264	\| `comments` \| Diff comments (inline or general) \|
265	\| `pipelines` \| CI/CD pipeline definitions \|
266	\| `pipeline_runs` \| Pipeline execution records \|
267	\| `pipeline_steps` \| Individual step records \|
268	\| `step_logs` \| Step output logs \|
269	\| `canopy_secrets` \| Encrypted CI/CD secrets \|
270	\| `bookmark_state` \| Mononoke bookmark tracking \|
271
272	### Hub API (SQLite: hub.db)
273
274	\| Table \| Purpose \|
275	\|-------\|---------\|
276	\| `users` \| User accounts \|
277	\| `credentials` \| WebAuthn passkey credentials \|
278	\| `orgs` \| Organizations \|
279	\| `org_members` \| Org membership \|
280	\| `instances` \| Grove instance registrations \|
281	\| `repos` \| Hub-level repo metadata \|
282	\| `api_tokens` \| Personal access tokens \|
283
284	---
285
286	## Deployment
287
288	### Production (grove.host)
289
290	```bash
291	# On server at /opt/grove/
292	docker compose up -d
293	```
294
295	Services auto-restart. Caddy handles TLS. Canopy CI rebuilds and deploys on push.
296
297	Backups: Daily at 4am UTC via `/opt/grove/backup.sh`. 7-day retention.
298
299	Firewall: UFW enabled. SSH restricted to admin IP. Ports 80, 443, 8080, 8443 open.
300
301	### CI/CD Pipeline
302
303	The Grove repo itself uses Canopy for CI/CD (self-hosting). Push to `main` triggers:
304	1. Build Docker images for grove-api, grove-web, hub-api
305	2. Push to local registry
306	3. `docker compose pull && up -d`
307
308	---
309
310	## Tech Stack
311
312	\| Layer \| Technology \| Why \|
313	\|-------\|-----------\|-----\|
314	\| Source control \| Mononoke (Rust) \| Production-proven at Meta scale, Git-compatible \|
315	\| API servers \| Fastify (Node.js/TypeScript) \| Fast, typed, same ecosystem as ISL \|
316	\| Web UI \| Next.js 15 + React 19 + Tailwind 4 \| SSR, app router, rapid iteration \|
317	\| Database \| SQLite \| Zero-config, embedded, sufficient for current scale \|
318	\| Auth \| WebAuthn passkeys + JWT \| Passwordless, phishing-resistant \|
319	\| Reverse proxy \| Caddy \| Auto-TLS, simple config \|
320	\| CI/CD \| Docker-based (Canopy) \| Native, no external dependency \|
321	\| Extension \| VS Code Extension API \| Largest IDE market share \|
322
323	---
324
325	## Key Design Decisions
326
327	1. Monorepo-first: One repo per user/org. Mononoke excels at this.
328	2. Git protocol for clients: Standard `git clone`/`git push` works. No custom client needed.
329	3. Git-based API reads: Uses bare Git clones + `git ls-tree`/`git show`/`git log` for file browsing. Avoids Mononoke CBOR protocol complexity.
330	4. Two-API split: Hub API (identity) is separate from Grove API (repos). Hub issues JWTs; Grove verifies them. Designed for multi-instance deployments.
331	5. WebAuthn-only auth: No passwords. Passkeys are phishing-resistant and simpler to implement securely.
332	6. Canopy CI/CD is native: Docker-based pipeline execution, integrated into grove-api. No external CI needed.
333	7. ISL extracted, not forked: ISL components are reusable. Full ISL integration is a future phase.
334