@1 Start: BLAKE3, the log, objects, the model and store, working directories, diffs, and the CLI

by claude for antonky · landed by antonky · 18d ago · files at @1

Files

.groveignoreadded+2 −0

Cargo.lockadded+7 −0

Cargo.tomladded+17 −0

VISION.mdadded+67 −0

docs/model.mdadded+231 −0

src/blake3.rsadded+289 −0

src/cli.rsadded+690 −0

src/diff.rsadded+181 −0

src/hex.rsadded+37 −0

src/ignore.rsadded+127 −0

src/json.rsadded+436 −0

src/lib.rsadded+17 −0

src/log.rsadded+336 −0

src/main.rsadded+7 −0

src/model.rsadded+481 −0

src/objects.rsadded+308 −0

src/random.rsadded+16 −0

src/store.rsadded+442 −0

src/testdir.rsadded+28 −0

src/workdir.rsadded+233 −0

	1	# This file is automatically @generated by Cargo.
	2	# It is not intended for manual editing.
	3	version = 4
	4
	5	[[package]]
	6	name = "grove"
	7	version = "0.1.0"

	1	[package]
	2	name = "grove"
	3	version = "0.1.0"
	4	edition = "2021"
	5	description = "Where agents build software for people."
	6	publish = false
	7
	8	# No dependencies: grove controls its own code. See VISION.md.
	9	[dependencies]
	10
	11	[[bin]]
	12	name = "grove"
	13	path = "src/main.rs"
	14
	15	[profile.release]
	16	lto = true
	17	codegen-units = 1

	1	# grove
	2
	3	grove is where agents build and maintain software for people. The owner of a project
	4	doesn't write its code; agents do, the owner's own and anyone else's. The owner steers
	5	the work and decides what goes in.
	6
	7	## People and agents
	8
	9	- Owner. A person who owns a project. Nothing lands without them. They can redirect
	10	work while it's in progress, not only accept or reject it at the end.
	11	- Agent. Writes code. Every agent acts for a person, and grove records both: which
	12	agent made a change and on whose behalf.
	13	- Contributor. Anyone, through their agents. Anyone can propose a change to a public
	14	project. The owner is under no obligation to take it.
	15
	16	## Changes
	17
	18	The unit of work is a change: something someone wants done to a project. A change can
	19	start as only an intent and a discussion, before any code exists. When an agent builds it,
	20	the same change gains its steps and evidence; the discussion stays attached. One object
	21	holds the idea, the argument about it, the work and the decision.
	22
	23	- Intent. What the change is for, in plain words. Usually the task the agent was given.
	24	- Stack. A change is made of small steps that build on each other and are kept in
	25	order. A step can be revised at any time; every earlier version of it is kept.
	26	- Evidence. Whatever the agent decides shows its change works: a written account,
	27	screenshots, logs, data, a running preview, anything else. grove doesn't prescribe the
	28	kinds; it stores what the agent attaches and shows it well. If the owner wants more or
	29	different evidence, they ask in the discussion.
	30	- Discussion. Anyone can take part: the owner, the proposer, other people, and agents
	31	for any of them. The owner can comment, ask for something different or redirect, and
	32	the building agent answers by revising the stack.
	33	- State. Idea (intent only), draft, proposed, accepted, landed, or declined.
	34
	35	A project's history is the sequence of landed changes, each with its intent and evidence.
	36	That is what people read. The code is there when someone wants it.
	37
	38	## Interfaces
	39
	40	- Agents use an API and an MCP server, and a small CLI to get a project's files into
	41	a working directory and send changes back.
	42	- People use the web, to see what's in flight, judge results and steer. Code views
	43	exist but are secondary.
	44
	45	## What grove builds on
	46
	47	grove is written in Rust with no dependencies beyond the standard library. It controls its
	48	own code: hashing, storage, HTTP and JSON are all grove's. The one planned exception is
	49	signature verification for passkeys, where a small audited library is safer than our own.
	50
	51	- No Git. No Sapling, Mononoke or EdenFS code. grove takes Sapling's ideas (stacks, revising
	52	freely while keeping history, a view of everything in flight) and implements them itself.
	53	- grove's own tools (CI, deploys, agents) get code from grove.
	54	- Storage has to be simple and safe above everything else: content-addressed files and an
	55	append-only log, tested, backed up from the start. Losing work is the one failure grove
	56	can't have.
	57	- grove holds its own code from the first day it can.
	58
	59	## First steps
	60
	61	1. The store (objects and log) with its tests, and a CLI working directly on it.
	62	2. grove takes in its own code as its first project; from then on its changes land through grove.
	63	3. Three-way merge for landing on a moved main; diffs.
	64	4. The HTTP API and MCP server, tokens for agents; grove runs on grove.host.
	65	5. Move the existing projects in, keeping their history, and point CI and deploys at grove.
	66	6. The web, built around changes and results.
	67	7. Retire Mononoke, the bridge and the Sapling fork.

	1	//! Line diffs (Myers' O(ND) algorithm) and unified output.
	2
	3	#[derive(Debug, Clone, Copy, PartialEq, Eq)]
	4	pub enum Op {
	5	Same,
	6	Del,
	7	Add,
	8	}
	9
	10	/// The shortest edit script from `a` to `b`, as (op, line) pairs in order.
	11	pub fn diff<'a>(a: &[&'a str], b: &[&'a str]) -> Vec<(Op, &'a str)> {
	12	// Trim the common prefix and suffix first; most diffs are small edits in large files.
	13	let pre = a.iter().zip(b).take_while(\|(x, y)\| x == y).count();
	14	let suf = a[pre..].iter().rev().zip(b[pre..].iter().rev()).take_while(\|(x, y)\| x == y).count();
	15	let (am, bm) = (&a[pre..a.len() - suf], &b[pre..b.len() - suf]);
	16
	17	let mut out: Vec<(Op, &str)> = a[..pre].iter().map(\|l\| (Op::Same, *l)).collect();
	18	out.extend(myers(am, bm));
	19	out.extend(a[a.len() - suf..].iter().map(\|l\| (Op::Same, *l)));
	20	out
	21	}
	22
	23	fn myers<'a>(a: &[&'a str], b: &[&'a str]) -> Vec<(Op, &'a str)> {
	24	let (n, m) = (a.len() as isize, b.len() as isize);
	25	if n == 0 {
	26	return b.iter().map(\|l\| (Op::Add, *l)).collect();
	27	}
	28	if m == 0 {
	29	return a.iter().map(\|l\| (Op::Del, *l)).collect();
	30	}
	31	let max = (n + m) as usize;
	32	let offset = max as isize;
	33	let mut v = vec![0isize; 2 * max + 2];
	34	let mut trace: Vec<Vec<isize>> = Vec::new();
	35	'outer: for d in 0..=max as isize {
	36	trace.push(v.clone());
	37	let mut k = -d;
	38	while k <= d {
	39	let idx = (k + offset) as usize;
	40	let mut x = if k == -d \|\| (k != d && v[idx - 1] < v[idx + 1]) { v[idx + 1] } else { v[idx - 1] + 1 };
	41	let mut y = x - k;
	42	while x < n && y < m && a[x as usize] == b[y as usize] {
	43	x += 1;
	44	y += 1;
	45	}
	46	v[idx] = x;
	47	if x >= n && y >= m {
	48	break 'outer;
	49	}
	50	k += 2;
	51	}
	52	}
	53	// Walk back through the saved frontiers to recover the path.
	54	let mut out = Vec::new();
	55	let (mut x, mut y) = (n, m);
	56	for d in (0..trace.len() as isize).rev() {
	57	let v = &trace[d as usize];
	58	let k = x - y;
	59	let idx = (k + offset) as usize;
	60	let prev_k = if k == -d \|\| (k != d && v[idx - 1] < v[idx + 1]) { k + 1 } else { k - 1 };
	61	let prev_x = v[(prev_k + offset) as usize];
	62	let prev_y = prev_x - prev_k;
	63	while x > prev_x && y > prev_y {
	64	out.push((Op::Same, a[(x - 1) as usize]));
	65	x -= 1;
	66	y -= 1;
	67	}
	68	if d > 0 {
	69	if x == prev_x {
	70	out.push((Op::Add, b[(y - 1) as usize]));
	71	} else {
	72	out.push((Op::Del, a[(x - 1) as usize]));
	73	}
	74	}
	75	x = prev_x;
	76	y = prev_y;
	77	}
	78	out.reverse();
	79	out
	80	}
	81
	82	/// Split text into lines, keeping track of whether the last one ended with a newline.
	83	pub fn lines(text: &str) -> Vec<&str> {
	84	if text.is_empty() {
	85	return Vec::new();
	86	}
	87	text.strip_suffix('\n').unwrap_or(text).split('\n').collect()
	88	}
	89
	90	/// Unified diff hunks with `context` lines around each change. Empty if equal.
	91	pub fn unified(a: &str, b: &str, context: usize) -> String {
	92	let (al, bl) = (lines(a), lines(b));
	93	let ops = diff(&al, &bl);
	94	if ops.iter().all(\|(op, _)\| *op == Op::Same) {
	95	return String::new();
	96	}
	97	// Line numbers before each op.
	98	let mut pos = Vec::with_capacity(ops.len());
	99	let (mut i, mut j) = (1usize, 1usize);
	100	for (op, _) in &ops {
	101	pos.push((i, j));
	102	match op {
	103	Op::Same => {
	104	i += 1;
	105	j += 1;
	106	}
	107	Op::Del => i += 1,
	108	Op::Add => j += 1,
	109	}
	110	}
	111	let changed: Vec<usize> = ops.iter().enumerate().filter(\|(_, (op, _))\| *op != Op::Same).map(\|(k, _)\| k).collect();
	112	let mut out = String::new();
	113	let mut k = 0;
	114	while k < changed.len() {
	115	let start = changed[k].saturating_sub(context);
	116	let mut end = changed[k];
	117	while k + 1 < changed.len() && changed[k + 1] <= end + 2 * context + 1 {
	118	k += 1;
	119	end = changed[k];
	120	}
	121	let end = (end + context + 1).min(ops.len());
	122	let (a0, b0) = pos[start];
	123	let a_len = ops[start..end].iter().filter(\|(op, _)\| *op != Op::Add).count();
	124	let b_len = ops[start..end].iter().filter(\|(op, _)\| *op != Op::Del).count();
	125	out.push_str(&format!("@@ -{},{} +{},{} @@\n", if a_len == 0 { a0 - 1 } else { a0 }, a_len, if b_len == 0 { b0 - 1 } else { b0 }, b_len));
	126	for (op, line) in &ops[start..end] {
	127	out.push(match op {
	128	Op::Same => ' ',
	129	Op::Del => '-',
	130	Op::Add => '+',
	131	});
	132	out.push_str(line);
	133	out.push('\n');
	134	}
	135	k += 1;
	136	}
	137	out
	138	}
	139
	140	#[cfg(test)]
	141	mod tests {
	142	use super::*;
	143
	144	fn apply(a: &[&str], ops: &[(Op, &str)]) -> (Vec<String>, Vec<String>) {
	145	let from: Vec<String> = ops.iter().filter(\|(o, _)\| *o != Op::Add).map(\|(_, l)\| l.to_string()).collect();
	146	let to: Vec<String> = ops.iter().filter(\|(o, _)\| *o != Op::Del).map(\|(_, l)\| l.to_string()).collect();
	147	let _ = a;
	148	(from, to)
	149	}
	150
	151	#[test]
	152	fn edit_scripts_reproduce_both_sides() {
	153	let cases: &[(&str, &str)] = &[
	154	("", ""),
	155	("a\nb\nc\n", "a\nb\nc\n"),
	156	("", "a\nb\n"),
	157	("a\nb\n", ""),
	158	("a\nb\nc\nd\n", "a\nx\nc\nd\ne\n"),
	159	("a\nb\nc\na\nb\nb\na\n", "c\nb\na\nb\na\nc\n"),
	160	];
	161	for (a, b) in cases {
	162	let (al, bl) = (lines(a), lines(b));
	163	let ops = diff(&al, &bl);
	164	let (from, to) = apply(&al, &ops);
	165	assert_eq!(from, al, "{a:?} -> {b:?}");
	166	assert_eq!(to, bl, "{a:?} -> {b:?}");
	167	}
	168	// The classic example has a shortest edit script of length 5.
	169	let (al, bl) = (lines("a\nb\nc\na\nb\nb\na\n"), lines("c\nb\na\nb\na\nc\n"));
	170	assert_eq!(diff(&al, &bl).iter().filter(\|(o, _)\| *o != Op::Same).count(), 5);
	171	}
	172
	173	#[test]
	174	fn unified_output() {
	175	let a = "1\n2\n3\n4\n5\n6\n7\n8\n9\n";
	176	let b = "1\n2\n3\nfour\n5\n6\n7\n8\n9\n";
	177	assert_eq!(unified(a, b, 1), "@@ -3,3 +3,3 @@\n 3\n-4\n+four\n 5\n");
	178	assert_eq!(unified(a, a, 3), "");
	179	assert_eq!(unified("", "new\n", 3), "@@ -0,0 +1,1 @@\n+new\n");
	180	}
	181	}

	1	//! Lowercase hex, for object ids and tokens.
	2
	3	const DIGITS: &[u8; 16] = b"0123456789abcdef";
	4
	5	pub fn encode(bytes: &[u8]) -> String {
	6	let mut s = String::with_capacity(bytes.len() * 2);
	7	for b in bytes {
	8	s.push(DIGITS[(b >> 4) as usize] as char);
	9	s.push(DIGITS[(b & 0xf) as usize] as char);
	10	}
	11	s
	12	}
	13
	14	pub fn decode(s: &str) -> Option<Vec<u8>> {
	15	if s.len() % 2 != 0 {
	16	return None;
	17	}
	18	let nibble = \|c: u8\| match c {
	19	b'0'..=b'9' => Some(c - b'0'),
	20	b'a'..=b'f' => Some(c - b'a' + 10),
	21	_ => None,
	22	};
	23	s.as_bytes().chunks(2).map(\|p\| Some(nibble(p[0])? << 4 \| nibble(p[1])?)).collect()
	24	}
	25
	26	#[cfg(test)]
	27	mod tests {
	28	#[test]
	29	fn round_trip() {
	30	let bytes = [0u8, 1, 0x7f, 0x80, 0xff];
	31	assert_eq!(super::encode(&bytes), "00017f80ff");
	32	assert_eq!(super::decode("00017f80ff").unwrap(), bytes);
	33	assert!(super::decode("0g").is_none());
	34	assert!(super::decode("abc").is_none());
	35	assert!(super::decode("AB").is_none());
	36	}
	37	}

	1	# grove core model
	2
	3	The objects grove stores, the rules they obey, and the operations people and agents perform.
	4	Read with [VISION.md](../VISION.md). Built in Rust with no dependencies beyond the standard library.
	5
	6	## Overview
	7
	8	```
	9	person ── owns ──> project ── main: point 1 → point 2 → … → point n (landed history)
	10	│ │
	11	└─ agents └─ changes ── discussion (messages, events)
	12	(act for │
	13	the person) └─ revisions ── steps (snapshot + message)
	14	└─ evidence (anything the agent attaches)
	15	```
	16
	17	A project's main line is a numbered sequence of snapshots. Work happens in changes. A change
	18	collects revisions of a stack of steps; the owner lands one revision, and its steps become the
	19	next points on main.
	20
	21	## Identity
	22
	23	Person. A human account: username, display name, passkeys for signing in.
	24
	25	Agent. Belongs to one person and acts only for them. It has a name the person chose
	26	("claude on laptop", "operator: grove"), and its own tokens. Every write an agent makes is
	27	recorded as agent A for person P. A person can revoke an agent, which kills its tokens and
	28	leaves its history intact.
	29
	30	People can also act directly, from the web. Then the actor is just the person.
	31
	32	## Projects
	33
	34	A project has an owner (a person), a name unique within the owner, a description, and a
	35	visibility:
	36
	37	- public: anyone can read it, open changes and join discussions.
	38	- private: only the owner and people they invite.
	39
	40	Only the owner (and people they make co-owners) can land, decline, or change settings, and
	41	only in person: these are decisions, so an agent acting for the owner can't make them.
	42
	43	## Content
	44
	45	Content is stored by hash, so identical files are stored once and nothing can change in place.
	46
	47	- Blob. The bytes of one file. Id: BLAKE3 of `"grove blob\0"` followed by the bytes.
	48	- Tree. One directory: a sorted list of entries, each `name`, `kind` (`file`, `exec`,
	49	`link`, `dir`) and the id of a blob or tree. A `link` is a blob holding the link target.
	50	Id: BLAKE3 of `"grove tree\0"` followed by the tree's canonical encoding.
	51	The prefixes keep a file and a tree with the same bytes from sharing an id.
	52	- Snapshot. The id of a root tree: the whole project at one moment.
	53
	54	Names are UTF-8 and may not contain `/` or be `.` or `..`. Binary and text files are the same
	55	to storage; diffs are computed for text and shown as size changes otherwise.
	56
	57	Ignored paths come from a `.groveignore` file in the project, same syntax as the usual ignore
	58	files. The CLI applies it; the server stores whatever it's given.
	59
	60	## Main
	61
	62	Main is the project's history: a linear sequence of points, numbered from 1.
	63
	64	\| field \| \|
	65	\|---\|---\|
	66	\| `n` \| 1, 2, 3, … with no gaps \|
	67	\| `snapshot` \| the project after this point \|
	68	\| `change`, `step` \| the change and step that produced it \|
	69	\| `message` \| the step's message \|
	70	\| `landed_by` \| who landed it (person), and when \|
	71	\| `authored_by` \| agent-for-person that wrote the step \|
	72
	73	There are no branches and no merges on main. Work in progress lives in changes. Main only
	74	ever grows; a mistake is undone by landing a change that reverts it.
	75
	76	Agents and the web refer to versions as `owner/project@n`, and `@main` for the latest.
	77
	78	## Changes
	79
	80	A change is something someone wants done to a project. It is numbered within the project
	81	(`owner/project#12`).
	82
	83	\| field \| \|
	84	\|---\|---\|
	85	\| `number` \| per project \|
	86	\| `intent` \| what it's for, in plain words; editable, every version kept \|
	87	\| `proposer` \| person or agent-for-person who opened it \|
	88	\| `state` \| see below \|
	89	\| `revisions` \| zero or more, in order \|
	90	\| `discussion` \| messages and events \|
	91
	92	### States
	93
	94	```
	95	idea ──> draft ──> proposed ──> accepted ──> landed
	96	│ │ │ │
	97	└────────┴───────────┴──> declined / withdrawn
	98	```
	99
	100	- idea: intent and discussion only, no revisions.
	101	- draft: has revisions; the author is still working. Not asking for a decision.
	102	- proposed: the author asks the owner to decide on the latest revision.
	103	- accepted: the owner approved a specific revision. grove lands it straight away; this
	104	state is visible only while landing runs, or if landing failed (see Landing).
	105	- landed: its steps are on main.
	106	- declined: the owner said no. withdrawn: the proposer gave up.
	107
	108	Declined and withdrawn changes can be reopened as draft. Nothing is ever deleted.
	109
	110	### Revisions and steps
	111
	112	A revision is one complete version of the work, immutable once created:
	113
	114	\| field \| \|
	115	\|---\|---\|
	116	\| `number` \| r1, r2, … within the change \|
	117	\| `base` \| the main point it was built on \|
	118	\| `steps` \| ordered list; each step builds on the one before \|
	119	\| `note` \| what changed since the previous revision, optional \|
	120	\| `created_by` \| agent-for-person \|
	121
	122	A step within a revision is a snapshot plus a message. Each step has a stable id inside
	123	the change, so the same step can be followed across revisions ("step 2 in r1, r2 and r4"),
	124	even as it's edited, reordered, split or dropped. Revising one step means creating a new
	125	revision; later steps carry forward, rebuilt on top of it if needed.
	126
	127	This keeps Sapling's useful property, that amending is free and nothing is lost, without
	128	tracking each amend as its own object. The diff between two revisions (or between two
	129	versions of one step) shows exactly what the agent did in response to feedback.
	130
	131	### Evidence
	132
	133	Evidence is attached to a revision. The agent decides what to attach.
	134
	135	\| field \| \|
	136	\|---\|---\|
	137	\| `title` \| short label \|
	138	\| `type` \| a media type (`text/markdown`, `image/png`, `application/json`, `text/uri-list`, …) \|
	139	\| `content` \| a blob, or a URL for something running elsewhere (a preview) \|
	140	\| `step` \| optional: the step it's about \|
	141	\| `created_by` \| agent-for-person \|
	142
	143	grove stores evidence as given and renders what it knows how to: markdown, images, video,
	144	logs, tables, links to previews. Anything else downloads.
	145
	146	### Discussion
	147
	148	A change's discussion is one timeline of messages and events.
	149
	150	A message has an author (person, or agent-for-person), a markdown body, optional attachments
	151	(stored like evidence), an optional reply-to, and an optional anchor: a revision, step, file
	152	or line it's about. Messages can be edited; every version is kept.
	153
	154	Events are written by grove: state changes, new revisions, evidence added, landing results.
	155
	156	Steering is a message. The owner (or their agent) writes in the discussion; the building
	157	agent reads it and responds with a new revision, a reply, or both. grove doesn't interpret
	158	the request.
	159
	160	## Landing
	161
	162	When the owner accepts revision R of change C:
	163
	164	1. If `R.base` is the current tip of main, each step becomes the next point, in order.
	165	2. Otherwise grove rebuilds the steps on the current tip with a three-way merge of each file
	166	(base snapshot, step snapshot, current tip).
	167	3. If every file merges cleanly, the rebuilt steps land as above, and the change records
	168	which revision landed and what it was rebased onto.
	169	4. If anything conflicts, nothing lands. The change returns to proposed with an event
	170	listing the conflicting files, and the agent makes a new revision on the current tip.
	171
	172	Landing is one database transaction: either all of a change's steps land or none do. Points
	173	are only ever appended.
	174
	175	## Agents' workflow
	176
	177	From an agent's side, through the CLI or the API:
	178
	179	1. `checkout owner/project`: files at `@main` in a working directory, with `.grove/`
	180	holding local state.
	181	2. Edit files. `step "message"` records the working directory as a new step; `amend` replaces
	182	the current step; `stack` shows the local steps.
	183	3. `propose` (or `revise` for an existing change) uploads the local steps as a new revision,
	184	sending only blobs the server doesn't have.
	185	4. `attach` adds evidence. `say` posts to the discussion. `watch` streams the discussion so
	186	the agent can respond to steering.
	187	5. `sync` brings the working directory up to the new tip and rebuilds local steps on it.
	188
	189	The MCP server exposes the same operations to agents that speak MCP.
	190
	191	## Storage
	192
	193	grove has no database. Two kinds of files hold everything:
	194
	195	- Objects (blobs and trees) live under `objects/<first 2 hex>/<rest of id>`. An object is
	196	written to a temporary file, flushed to disk, renamed into place, and the directory flushed,
	197	before any record refers to it. Objects are never rewritten.
	198	- The log holds every other fact as an append-only sequence of records: person and agent
	199	created, project created, change opened, revision added, message posted, state changed,
	200	point landed. Each record is `length, checksum, payload` (payload is JSON; checksum is the
	201	first 8 bytes of its BLAKE3). A write is acknowledged only after the record is flushed to
	202	disk. On start, grove replays the log into memory and answers every query from memory.
	203	- Torn writes. If the process dies mid-append, the last record fails its length or
	204	checksum. On start, grove cuts the file back to the last whole record. Damage anywhere
	205	but the tail stops grove with an error instead of guessing.
	206	- One writer. The store takes an exclusive file lock while open.
	207	- Backups are a copy of the log and the objects directory, both append-only, so backups
	208	are incremental. An integrity check confirms every object a record refers to exists and
	209	hashes to its id; it runs after every backup and on demand.
	210	- Fast start. When the log grows large, grove writes a checkpoint (the in-memory state at
	211	record N, checksummed) and replays only the records after it. The log itself is kept whole.
	212
	213	## Moving existing projects in
	214
	215	Each repository's current history on Mononoke is read commit by commit, oldest first, and
	216	written as points on main with the original messages, authors and times. Each imported point
	217	records that it came from an import and the original commit hash. Open diffs on the old system
	218	become changes in proposed state with one revision each. Nothing on the old system is removed
	219	until the import of every project has been checked against it.
	220
	221	## Open decisions
	222
	223	1. Shared ownership. Today there are orgs. Should a project be owned by a group of
	224	people, or by one person with co-owners?
	225	2. Agents acting for someone else. Should a person be able to lend their agent to
	226	another project's owner (an agent working for the owner, run by someone else)?
	227	3. Who runs previews. Evidence can link to a running preview. Does grove host previews
	228	itself (as Pages does today), or only link to what the agent runs?
	229	4. Notifications. How you hear that something needs your decision: web only, email,
	230	or through your own agent.
	231	5. Size limits. A maximum file and revision size, so an agent can't fill the disk by mistake.

	1	//! BLAKE3, the hash behind every object id.
	2	//!
	3	//! A direct implementation of the BLAKE3 specification (portable, single-threaded), checked
	4	//! against the official test vectors below. Only the default hash mode is needed.
	5
	6	const OUT_LEN: usize = 32;
	7	const BLOCK_LEN: usize = 64;
	8	const CHUNK_LEN: usize = 1024;
	9
	10	const CHUNK_START: u32 = 1 << 0;
	11	const CHUNK_END: u32 = 1 << 1;
	12	const PARENT: u32 = 1 << 2;
	13	const ROOT: u32 = 1 << 3;
	14
	15	const IV: [u32; 8] = [
	16	0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A, 0x510E527F, 0x9B05688C, 0x1F83D9AB, 0x5BE0CD19,
	17	];
	18
	19	const MSG_PERMUTATION: [usize; 16] = [2, 6, 3, 10, 7, 0, 4, 13, 1, 11, 12, 5, 9, 14, 15, 8];
	20
	21	#[inline(always)]
	22	fn g(state: &mut [u32; 16], a: usize, b: usize, c: usize, d: usize, mx: u32, my: u32) {
	23	state[a] = state[a].wrapping_add(state[b]).wrapping_add(mx);
	24	state[d] = (state[d] ^ state[a]).rotate_right(16);
	25	state[c] = state[c].wrapping_add(state[d]);
	26	state[b] = (state[b] ^ state[c]).rotate_right(12);
	27	state[a] = state[a].wrapping_add(state[b]).wrapping_add(my);
	28	state[d] = (state[d] ^ state[a]).rotate_right(8);
	29	state[c] = state[c].wrapping_add(state[d]);
	30	state[b] = (state[b] ^ state[c]).rotate_right(7);
	31	}
	32
	33	#[inline(always)]
	34	fn round(state: &mut [u32; 16], m: &[u32; 16]) {
	35	// Columns.
	36	g(state, 0, 4, 8, 12, m[0], m[1]);
	37	g(state, 1, 5, 9, 13, m[2], m[3]);
	38	g(state, 2, 6, 10, 14, m[4], m[5]);
	39	g(state, 3, 7, 11, 15, m[6], m[7]);
	40	// Diagonals.
	41	g(state, 0, 5, 10, 15, m[8], m[9]);
	42	g(state, 1, 6, 11, 12, m[10], m[11]);
	43	g(state, 2, 7, 8, 13, m[12], m[13]);
	44	g(state, 3, 4, 9, 14, m[14], m[15]);
	45	}
	46
	47	fn permute(m: &mut [u32; 16]) {
	48	let mut permuted = [0; 16];
	49	for i in 0..16 {
	50	permuted[i] = m[MSG_PERMUTATION[i]];
	51	}
	52	*m = permuted;
	53	}
	54
	55	fn compress(cv: &[u32; 8], block_words: &[u32; 16], counter: u64, block_len: u32, flags: u32) -> [u32; 16] {
	56	let mut state = [
	57	cv[0], cv[1], cv[2], cv[3], cv[4], cv[5], cv[6], cv[7],
	58	IV[0], IV[1], IV[2], IV[3],
	59	counter as u32, (counter >> 32) as u32, block_len, flags,
	60	];
	61	let mut block = *block_words;
	62	for r in 0..7 {
	63	round(&mut state, &block);
	64	if r < 6 {
	65	permute(&mut block);
	66	}
	67	}
	68	for i in 0..8 {
	69	state[i] ^= state[i + 8];
	70	state[i + 8] ^= cv[i];
	71	}
	72	state
	73	}
	74
	75	fn first_8_words(w: [u32; 16]) -> [u32; 8] {
	76	let mut out = [0; 8];
	77	out.copy_from_slice(&w[..8]);
	78	out
	79	}
	80
	81	fn words_from_le_bytes(bytes: &[u8; BLOCK_LEN]) -> [u32; 16] {
	82	let mut words = [0; 16];
	83	for (word, chunk) in words.iter_mut().zip(bytes.chunks_exact(4)) {
	84	*word = u32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]);
	85	}
	86	words
	87	}
	88
	89	struct Output {
	90	input_cv: [u32; 8],
	91	block_words: [u32; 16],
	92	counter: u64,
	93	block_len: u32,
	94	flags: u32,
	95	}
	96
	97	impl Output {
	98	fn chaining_value(&self) -> [u32; 8] {
	99	first_8_words(compress(&self.input_cv, &self.block_words, self.counter, self.block_len, self.flags))
	100	}
	101
	102	fn root_hash(&self) -> [u8; OUT_LEN] {
	103	let words = compress(&self.input_cv, &self.block_words, 0, self.block_len, self.flags \| ROOT);
	104	let mut out = [0u8; OUT_LEN];
	105	for (i, word) in words[..8].iter().enumerate() {
	106	out[i * 4..i * 4 + 4].copy_from_slice(&word.to_le_bytes());
	107	}
	108	out
	109	}
	110	}
	111
	112	struct ChunkState {
	113	cv: [u32; 8],
	114	chunk_counter: u64,
	115	block: [u8; BLOCK_LEN],
	116	block_len: usize,
	117	blocks_compressed: usize,
	118	}
	119
	120	impl ChunkState {
	121	fn new(chunk_counter: u64) -> Self {
	122	ChunkState { cv: IV, chunk_counter, block: [0; BLOCK_LEN], block_len: 0, blocks_compressed: 0 }
	123	}
	124
	125	fn len(&self) -> usize {
	126	BLOCK_LEN * self.blocks_compressed + self.block_len
	127	}
	128
	129	fn start_flag(&self) -> u32 {
	130	if self.blocks_compressed == 0 { CHUNK_START } else { 0 }
	131	}
	132
	133	fn update(&mut self, mut input: &[u8]) {
	134	while !input.is_empty() {
	135	// A full block is only compressed once more input arrives, so the last block of the
	136	// chunk is still here when output() needs it.
	137	if self.block_len == BLOCK_LEN {
	138	let words = words_from_le_bytes(&self.block);
	139	self.cv = first_8_words(compress(&self.cv, &words, self.chunk_counter, BLOCK_LEN as u32, self.start_flag()));
	140	self.blocks_compressed += 1;
	141	self.block = [0; BLOCK_LEN];
	142	self.block_len = 0;
	143	}
	144	let take = (BLOCK_LEN - self.block_len).min(input.len());
	145	self.block[self.block_len..self.block_len + take].copy_from_slice(&input[..take]);
	146	self.block_len += take;
	147	input = &input[take..];
	148	}
	149	}
	150
	151	fn output(&self) -> Output {
	152	Output {
	153	input_cv: self.cv,
	154	block_words: words_from_le_bytes(&self.block),
	155	counter: self.chunk_counter,
	156	block_len: self.block_len as u32,
	157	flags: self.start_flag() \| CHUNK_END,
	158	}
	159	}
	160	}
	161
	162	fn parent_output(left: [u32; 8], right: [u32; 8]) -> Output {
	163	let mut block_words = [0; 16];
	164	block_words[..8].copy_from_slice(&left);
	165	block_words[8..].copy_from_slice(&right);
	166	Output { input_cv: IV, block_words, counter: 0, block_len: BLOCK_LEN as u32, flags: PARENT }
	167	}
	168
	169	/// Incremental BLAKE3 hasher.
	170	pub struct Hasher {
	171	chunk: ChunkState,
	172	cv_stack: [[u32; 8]; 54], // enough for 2^54 chunks
	173	cv_stack_len: usize,
	174	}
	175
	176	impl Default for Hasher {
	177	fn default() -> Self {
	178	Self::new()
	179	}
	180	}
	181
	182	impl Hasher {
	183	pub fn new() -> Self {
	184	Hasher { chunk: ChunkState::new(0), cv_stack: [[0; 8]; 54], cv_stack_len: 0 }
	185	}
	186
	187	fn push(&mut self, cv: [u32; 8]) {
	188	self.cv_stack[self.cv_stack_len] = cv;
	189	self.cv_stack_len += 1;
	190	}
	191
	192	fn pop(&mut self) -> [u32; 8] {
	193	self.cv_stack_len -= 1;
	194	self.cv_stack[self.cv_stack_len]
	195	}
	196
	197	// Merge completed subtrees: one merge per trailing zero bit of the chunk count.
	198	fn add_chunk_cv(&mut self, mut cv: [u32; 8], mut total_chunks: u64) {
	199	while total_chunks & 1 == 0 {
	200	cv = parent_output(self.pop(), cv).chaining_value();
	201	total_chunks >>= 1;
	202	}
	203	self.push(cv);
	204	}
	205
	206	pub fn update(&mut self, mut input: &[u8]) -> &mut Self {
	207	while !input.is_empty() {
	208	if self.chunk.len() == CHUNK_LEN {
	209	let cv = self.chunk.output().chaining_value();
	210	let total = self.chunk.chunk_counter + 1;
	211	self.add_chunk_cv(cv, total);
	212	self.chunk = ChunkState::new(total);
	213	}
	214	let take = (CHUNK_LEN - self.chunk.len()).min(input.len());
	215	self.chunk.update(&input[..take]);
	216	input = &input[take..];
	217	}
	218	self
	219	}
	220
	221	pub fn finalize(&self) -> [u8; OUT_LEN] {
	222	let mut output = self.chunk.output();
	223	let mut i = self.cv_stack_len;
	224	while i > 0 {
	225	i -= 1;
	226	output = parent_output(self.cv_stack[i], output.chaining_value());
	227	}
	228	output.root_hash()
	229	}
	230	}
	231
	232	/// The BLAKE3 hash of `input`.
	233	pub fn hash(input: &[u8]) -> [u8; OUT_LEN] {
	234	let mut h = Hasher::new();
	235	h.update(input);
	236	h.finalize()
	237	}
	238
	239	#[cfg(test)]
	240	mod tests {
	241	use super::*;
	242	use crate::hex;
	243
	244	// From the official test_vectors.json: input is bytes 0, 1, 2, … 250, 0, 1, … of each length.
	245	const VECTORS: &[(usize, &str)] = &[
	246	(0, "af1349b9f5f9a1a6a0404dea36dcc9499bcb25c9adc112b7cc9a93cae41f3262"),
	247	(1, "2d3adedff11b61f14c886e35afa036736dcd87a74d27b5c1510225d0f592e213"),
	248	(63, "e9bc37a594daad83be9470df7f7b3798297c3d834ce80ba85d6e207627b7db7b"),
	249	(64, "4eed7141ea4a5cd4b788606bd23f46e212af9cacebacdc7d1f4c6dc7f2511b98"),
	250	(65, "de1e5fa0be70df6d2be8fffd0e99ceaa8eb6e8c93a63f2d8d1c30ecb6b263dee"),
	251	(1023, "10108970eeda3eb932baac1428c7a2163b0e924c9a9e25b35bba72b28f70bd11"),
	252	(1024, "42214739f095a406f3fc83deb889744ac00df831c10daa55189b5d121c855af7"),
	253	(1025, "d00278ae47eb27b34faecf67b4fe263f82d5412916c1ffd97c8cb7fb814b8444"),
	254	(2048, "e776b6028c7cd22a4d0ba182a8bf62205d2ef576467e838ed6f2529b85fba24a"),
	255	(2049, "5f4d72f40d7a5f82b15ca2b2e44b1de3c2ef86c426c95c1af0b6879522563030"),
	256	(3072, "b98cb0ff3623be03326b373de6b9095218513e64f1ee2edd2525c7ad1e5cffd2"),
	257	(3073, "7124b49501012f81cc7f11ca069ec9226cecb8a2c850cfe644e327d22d3e1cd3"),
	258	(4096, "015094013f57a5277b59d8475c0501042c0b642e531b0a1c8f58d2163229e969"),
	259	(4097, "9b4052b38f1c5fc8b1f9ff7ac7b27cd242487b3d890d15c96a1c25b8aa0fb995"),
	260	(8192, "aae792484c8efe4f19e2ca7d371d8c467ffb10748d8a5a1ae579948f718a2a63"),
	261	(8193, "bab6c09cb8ce8cf459261398d2e7aef35700bf488116ceb94a36d0f5f1b7bc3b"),
	262	(16384, "f875d6646de28985646f34ee13be9a576fd515f76b5b0a26bb324735041ddde4"),
	263	(31744, "62b6960e1a44bcc1eb1a611a8d6235b6b4b78f32e7abc4fb4c6cdcce94895c47"),
	264	(102400, "bc3e3d41a1146b069abffad3c0d44860cf664390afce4d9661f7902e7943e085"),
	265	];
	266
	267	fn input(len: usize) -> Vec<u8> {
	268	(0..len).map(\|i\| (i % 251) as u8).collect()
	269	}
	270
	271	#[test]
	272	fn official_vectors() {
	273	for &(len, expected) in VECTORS {
	274	assert_eq!(hex::encode(&hash(&input(len))), expected, "length {len}");
	275	}
	276	}
	277
	278	#[test]
	279	fn incremental_matches_one_shot() {
	280	let data = input(102400);
	281	for split in [1, 63, 64, 65, 1023, 1024, 1025, 4097, 50000] {
	282	let mut h = Hasher::new();
	283	for piece in data.chunks(split) {
	284	h.update(piece);
	285	}
	286	assert_eq!(h.finalize(), hash(&data), "pieces of {split}");
	287	}
	288	}
	289	}

	1	//! `.groveignore`: which paths in a working directory aren't part of the project.
	2	//!
	3	//! One pattern per line; `#` starts a comment. Like the familiar ignore files:
	4	//! - `name` matches that name at any depth; `dir/name` (or `/name`) is anchored at the root
	5	//! - a trailing `/` matches directories only
	6	//! - `` matches within one path segment, `?` one character, `*` any number of segments
	7	//! - `!pattern` re-includes something an earlier pattern excluded; the last match wins
	8	//!
	9	//! `.grove/` (the working directory's own state) is always ignored.
	10
	11	#[derive(Debug)]
	12	struct Pattern {
	13	glob: Vec<u8>,
	14	anchored: bool,
	15	dir_only: bool,
	16	negate: bool,
	17	}
	18
	19	#[derive(Debug, Default)]
	20	pub struct Ignore {
	21	patterns: Vec<Pattern>,
	22	}
	23
	24	impl Ignore {
	25	pub fn parse(text: &str) -> Ignore {
	26	let mut patterns = Vec::new();
	27	for line in text.lines() {
	28	let line = line.trim_end();
	29	if line.is_empty() \|\| line.starts_with('#') {
	30	continue;
	31	}
	32	let (negate, line) = match line.strip_prefix('!') {
	33	Some(rest) => (true, rest),
	34	None => (false, line),
	35	};
	36	let (dir_only, line) = match line.strip_suffix('/') {
	37	Some(rest) => (true, rest),
	38	None => (false, line),
	39	};
	40	let anchored = line.contains('/');
	41	let line = line.strip_prefix('/').unwrap_or(line);
	42	if line.is_empty() {
	43	continue;
	44	}
	45	patterns.push(Pattern { glob: line.as_bytes().to_vec(), anchored, dir_only, negate });
	46	}
	47	Ignore { patterns }
	48	}
	49
	50	/// Is `path` (relative, `/`-separated) ignored? `is_dir` says whether it's a directory.
	51	pub fn ignored(&self, path: &str, is_dir: bool) -> bool {
	52	if path == ".grove" \|\| path.starts_with(".grove/") {
	53	return true;
	54	}
	55	let name = path.rsplit('/').next().unwrap_or(path);
	56	let mut result = false;
	57	for p in &self.patterns {
	58	if p.dir_only && !is_dir {
	59	continue;
	60	}
	61	let subject = if p.anchored { path } else { name };
	62	if glob(&p.glob, subject.as_bytes()) {
	63	result = !p.negate;
	64	}
	65	}
	66	result
	67	}
	68	}
	69
	70	/// Glob match where `` and `?` stay within a segment and `*` spans segments.
	71	fn glob(p: &[u8], s: &[u8]) -> bool {
	72	if p.is_empty() {
	73	return s.is_empty();
	74	}
	75	if p.starts_with(b"**") {
	76	let rest = p[2..].strip_prefix(b"/").unwrap_or(&p[2..]);
	77	// `**` matches zero or more whole segments.
	78	if glob(rest, s) {
	79	return true;
	80	}
	81	for (i, &c) in s.iter().enumerate() {
	82	if c == b'/' && glob(rest, &s[i + 1..]) {
	83	return true;
	84	}
	85	}
	86	return rest.is_empty();
	87	}
	88	match p[0] {
	89	b'*' => {
	90	for i in 0..=s.len() {
	91	if glob(&p[1..], &s[i..]) {
	92	return true;
	93	}
	94	if i < s.len() && s[i] == b'/' {
	95	break;
	96	}
	97	}
	98	false
	99	}
	100	b'?' => !s.is_empty() && s[0] != b'/' && glob(&p[1..], &s[1..]),
	101	c => !s.is_empty() && s[0] == c && glob(&p[1..], &s[1..]),
	102	}
	103	}
	104
	105	#[cfg(test)]
	106	mod tests {
	107	use super::*;
	108
	109	#[test]
	110	fn patterns() {
	111	let ig = Ignore::parse("# build\ntarget/\n.log\n/root-only\ndocs/.tmp\n**/cache\n!keep.log\n");
	112	assert!(ig.ignored("target", true));
	113	assert!(ig.ignored("sub/target", true));
	114	assert!(!ig.ignored("target", false), "dir-only pattern");
	115	assert!(ig.ignored("a.log", false));
	116	assert!(ig.ignored("deep/b.log", false));
	117	assert!(!ig.ignored("keep.log", false), "negation");
	118	assert!(ig.ignored("root-only", false));
	119	assert!(!ig.ignored("sub/root-only", false), "anchored");
	120	assert!(ig.ignored("docs/x.tmp", false));
	121	assert!(!ig.ignored("docs/sub/x.tmp", false), "* stays in one segment");
	122	assert!(ig.ignored("cache", true));
	123	assert!(ig.ignored("a/b/cache", true));
	124	assert!(ig.ignored(".grove", true));
	125	assert!(!ig.ignored("src/main.rs", false));
	126	}
	127	}

	1	//! JSON (RFC 8259): log record payloads, and later the API.
	2	//!
	3	//! Objects keep their key order, so encoding is deterministic. Integers are kept exact as i64;
	4	//! other numbers are f64.
	5
	6	use std::fmt::Write as _;
	7
	8	#[derive(Clone, Debug, PartialEq)]
	9	pub enum Value {
	10	Null,
	11	Bool(bool),
	12	Int(i64),
	13	Float(f64),
	14	Str(String),
	15	Array(Vec<Value>),
	16	Object(Vec<(String, Value)>),
	17	}
	18
	19	impl Value {
	20	pub fn get(&self, key: &str) -> Option<&Value> {
	21	match self {
	22	Value::Object(fields) => fields.iter().find(\|(k, _)\| k == key).map(\|(_, v)\| v),
	23	_ => None,
	24	}
	25	}
	26	pub fn as_str(&self) -> Option<&str> {
	27	match self {
	28	Value::Str(s) => Some(s),
	29	_ => None,
	30	}
	31	}
	32	pub fn as_i64(&self) -> Option<i64> {
	33	match self {
	34	Value::Int(n) => Some(*n),
	35	_ => None,
	36	}
	37	}
	38	pub fn as_array(&self) -> Option<&[Value]> {
	39	match self {
	40	Value::Array(a) => Some(a),
	41	_ => None,
	42	}
	43	}
	44	pub fn is_null(&self) -> bool {
	45	matches!(self, Value::Null)
	46	}
	47
	48	pub fn encode(&self) -> String {
	49	let mut out = String::new();
	50	self.write(&mut out);
	51	out
	52	}
	53
	54	fn write(&self, out: &mut String) {
	55	match self {
	56	Value::Null => out.push_str("null"),
	57	Value::Bool(b) => out.push_str(if *b { "true" } else { "false" }),
	58	Value::Int(n) => {
	59	let _ = write!(out, "{n}");
	60	}
	61	Value::Float(f) => {
	62	if f.is_finite() {
	63	let _ = write!(out, "{f:?}");
	64	} else {
	65	out.push_str("null"); // JSON has no NaN or infinity
	66	}
	67	}
	68	Value::Str(s) => write_str(s, out),
	69	Value::Array(items) => {
	70	out.push('[');
	71	for (i, v) in items.iter().enumerate() {
	72	if i > 0 {
	73	out.push(',');
	74	}
	75	v.write(out);
	76	}
	77	out.push(']');
	78	}
	79	Value::Object(fields) => {
	80	out.push('{');
	81	for (i, (k, v)) in fields.iter().enumerate() {
	82	if i > 0 {
	83	out.push(',');
	84	}
	85	write_str(k, out);
	86	out.push(':');
	87	v.write(out);
	88	}
	89	out.push('}');
	90	}
	91	}
	92	}
	93	}
	94
	95	fn write_str(s: &str, out: &mut String) {
	96	out.push('"');
	97	for c in s.chars() {
	98	match c {
	99	'"' => out.push_str("\\\""),
	100	'\\' => out.push_str("\\\\"),
	101	'\n' => out.push_str("\\n"),
	102	'\r' => out.push_str("\\r"),
	103	'\t' => out.push_str("\\t"),
	104	c if (c as u32) < 0x20 => {
	105	let _ = write!(out, "\\u{:04x}", c as u32);
	106	}
	107	c => out.push(c),
	108	}
	109	}
	110	out.push('"');
	111	}
	112
	113	/// Build an object: `obj([("a", 1.into()), ("b", "x".into())])`.
	114	pub fn obj<const N: usize>(fields: [(&str, Value); N]) -> Value {
	115	Value::Object(fields.into_iter().map(\|(k, v)\| (k.to_string(), v)).collect())
	116	}
	117
	118	impl From<&str> for Value {
	119	fn from(s: &str) -> Self {
	120	Value::Str(s.to_string())
	121	}
	122	}
	123	impl From<String> for Value {
	124	fn from(s: String) -> Self {
	125	Value::Str(s)
	126	}
	127	}
	128	impl From<i64> for Value {
	129	fn from(n: i64) -> Self {
	130	Value::Int(n)
	131	}
	132	}
	133	impl From<u64> for Value {
	134	fn from(n: u64) -> Self {
	135	Value::Int(n as i64)
	136	}
	137	}
	138	impl From<bool> for Value {
	139	fn from(b: bool) -> Self {
	140	Value::Bool(b)
	141	}
	142	}
	143	impl<T: Into<Value>> From<Option<T>> for Value {
	144	fn from(o: Option<T>) -> Self {
	145	o.map_or(Value::Null, Into::into)
	146	}
	147	}
	148	impl From<Vec<Value>> for Value {
	149	fn from(v: Vec<Value>) -> Self {
	150	Value::Array(v)
	151	}
	152	}
	153
	154	#[derive(Debug, PartialEq)]
	155	pub struct ParseError {
	156	pub at: usize,
	157	pub msg: &'static str,
	158	}
	159
	160	impl std::fmt::Display for ParseError {
	161	fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
	162	write!(f, "invalid JSON at byte {}: {}", self.at, self.msg)
	163	}
	164	}
	165
	166	/// Parse a complete JSON document. Trailing content other than whitespace is an error.
	167	pub fn parse(src: &str) -> Result<Value, ParseError> {
	168	let mut p = Parser { s: src.as_bytes(), i: 0, depth: 0 };
	169	p.ws();
	170	let v = p.value()?;
	171	p.ws();
	172	if p.i != p.s.len() {
	173	return Err(p.err("trailing characters"));
	174	}
	175	Ok(v)
	176	}
	177
	178	const MAX_DEPTH: usize = 128;
	179
	180	struct Parser<'a> {
	181	s: &'a [u8],
	182	i: usize,
	183	depth: usize,
	184	}
	185
	186	impl Parser<'_> {
	187	fn err(&self, msg: &'static str) -> ParseError {
	188	ParseError { at: self.i, msg }
	189	}
	190	fn ws(&mut self) {
	191	while let Some(b' ' \| b'\n' \| b'\r' \| b'\t') = self.s.get(self.i) {
	192	self.i += 1;
	193	}
	194	}
	195	fn eat(&mut self, lit: &[u8]) -> bool {
	196	if self.s[self.i..].starts_with(lit) {
	197	self.i += lit.len();
	198	true
	199	} else {
	200	false
	201	}
	202	}
	203
	204	fn value(&mut self) -> Result<Value, ParseError> {
	205	match self.s.get(self.i) {
	206	None => Err(self.err("unexpected end")),
	207	Some(b'n') if self.eat(b"null") => Ok(Value::Null),
	208	Some(b't') if self.eat(b"true") => Ok(Value::Bool(true)),
	209	Some(b'f') if self.eat(b"false") => Ok(Value::Bool(false)),
	210	Some(b'"') => Ok(Value::Str(self.string()?)),
	211	Some(b'[') => self.array(),
	212	Some(b'{') => self.object(),
	213	Some(b'-' \| b'0'..=b'9') => self.number(),
	214	_ => Err(self.err("unexpected character")),
	215	}
	216	}
	217
	218	fn nest(&mut self) -> Result<(), ParseError> {
	219	self.depth += 1;
	220	if self.depth > MAX_DEPTH {
	221	return Err(self.err("nested too deeply"));
	222	}
	223	Ok(())
	224	}
	225
	226	fn array(&mut self) -> Result<Value, ParseError> {
	227	self.nest()?;
	228	self.i += 1;
	229	let mut items = Vec::new();
	230	self.ws();
	231	if self.eat(b"]") {
	232	self.depth -= 1;
	233	return Ok(Value::Array(items));
	234	}
	235	loop {
	236	self.ws();
	237	items.push(self.value()?);
	238	self.ws();
	239	if self.eat(b",") {
	240	continue;
	241	}
	242	if self.eat(b"]") {
	243	self.depth -= 1;
	244	return Ok(Value::Array(items));
	245	}
	246	return Err(self.err("expected , or ]"));
	247	}
	248	}
	249
	250	fn object(&mut self) -> Result<Value, ParseError> {
	251	self.nest()?;
	252	self.i += 1;
	253	let mut fields: Vec<(String, Value)> = Vec::new();
	254	self.ws();
	255	if self.eat(b"}") {
	256	self.depth -= 1;
	257	return Ok(Value::Object(fields));
	258	}
	259	loop {
	260	self.ws();
	261	if self.s.get(self.i) != Some(&b'"') {
	262	return Err(self.err("expected a key"));
	263	}
	264	let key = self.string()?;
	265	if fields.iter().any(\|(k, _)\| *k == key) {
	266	return Err(self.err("duplicate key"));
	267	}
	268	self.ws();
	269	if !self.eat(b":") {
	270	return Err(self.err("expected :"));
	271	}
	272	self.ws();
	273	let v = self.value()?;
	274	fields.push((key, v));
	275	self.ws();
	276	if self.eat(b",") {
	277	continue;
	278	}
	279	if self.eat(b"}") {
	280	self.depth -= 1;
	281	return Ok(Value::Object(fields));
	282	}
	283	return Err(self.err("expected , or }"));
	284	}
	285	}
	286
	287	fn number(&mut self) -> Result<Value, ParseError> {
	288	let start = self.i;
	289	let digits = \|p: &mut Self\| {
	290	let s = p.i;
	291	while let Some(b'0'..=b'9') = p.s.get(p.i) {
	292	p.i += 1;
	293	}
	294	p.i - s
	295	};
	296	self.eat(b"-");
	297	match self.s.get(self.i) {
	298	Some(b'0') => self.i += 1,
	299	Some(b'1'..=b'9') => {
	300	digits(self);
	301	}
	302	_ => return Err(self.err("expected a digit")),
	303	}
	304	let mut float = false;
	305	if self.eat(b".") {
	306	float = true;
	307	if digits(self) == 0 {
	308	return Err(self.err("expected a digit after ."));
	309	}
	310	}
	311	if let Some(b'e' \| b'E') = self.s.get(self.i) {
	312	float = true;
	313	self.i += 1;
	314	if let Some(b'+' \| b'-') = self.s.get(self.i) {
	315	self.i += 1;
	316	}
	317	if digits(self) == 0 {
	318	return Err(self.err("expected a digit in exponent"));
	319	}
	320	}
	321	let text = std::str::from_utf8(&self.s[start..self.i]).expect("ascii");
	322	if !float {
	323	if let Ok(n) = text.parse::<i64>() {
	324	return Ok(Value::Int(n));
	325	}
	326	}
	327	text.parse::<f64>().map(Value::Float).map_err(\|_\| self.err("bad number"))
	328	}
	329
	330	fn string(&mut self) -> Result<String, ParseError> {
	331	self.i += 1; // opening quote
	332	let mut out = String::new();
	333	loop {
	334	let start = self.i;
	335	while let Some(&b) = self.s.get(self.i) {
	336	if b == b'"' \|\| b == b'\\' \|\| b < 0x20 {
	337	break;
	338	}
	339	self.i += 1;
	340	}
	341	// The input is a &str, so any run between ASCII delimiters is valid UTF-8.
	342	out.push_str(std::str::from_utf8(&self.s[start..self.i]).map_err(\|_\| self.err("bad UTF-8"))?);
	343	match self.s.get(self.i) {
	344	None => return Err(self.err("unterminated string")),
	345	Some(b'"') => {
	346	self.i += 1;
	347	return Ok(out);
	348	}
	349	Some(b'\\') => {
	350	self.i += 1;
	351	let c = match self.s.get(self.i) {
	352	Some(b'"') => '"',
	353	Some(b'\\') => '\\',
	354	Some(b'/') => '/',
	355	Some(b'b') => '\u{8}',
	356	Some(b'f') => '\u{c}',
	357	Some(b'n') => '\n',
	358	Some(b'r') => '\r',
	359	Some(b't') => '\t',
	360	Some(b'u') => {
	361	self.i += 1;
	362	let hi = self.hex4()?;
	363	let code = if (0xD800..0xDC00).contains(&hi) {
	364	if !self.eat(b"\\u") {
	365	return Err(self.err("lone surrogate"));
	366	}
	367	let lo = self.hex4()?;
	368	if !(0xDC00..0xE000).contains(&lo) {
	369	return Err(self.err("bad surrogate pair"));
	370	}
	371	0x10000 + ((hi - 0xD800) << 10) + (lo - 0xDC00)
	372	} else if (0xDC00..0xE000).contains(&hi) {
	373	return Err(self.err("lone surrogate"));
	374	} else {
	375	hi
	376	};
	377	out.push(char::from_u32(code).ok_or_else(\|\| self.err("bad escape"))?);
	378	continue;
	379	}
	380	_ => return Err(self.err("bad escape")),
	381	};
	382	self.i += 1;
	383	out.push(c);
	384	}
	385	Some(_) => return Err(self.err("control character in string")),
	386	}
	387	}
	388	}
	389
	390	fn hex4(&mut self) -> Result<u32, ParseError> {
	391	let h = self.s.get(self.i..self.i + 4).ok_or_else(\|\| self.err("short \\u escape"))?;
	392	let s = std::str::from_utf8(h).map_err(\|_\| self.err("bad \\u escape"))?;
	393	let n = u32::from_str_radix(s, 16).map_err(\|_\| self.err("bad \\u escape"))?;
	394	self.i += 4;
	395	Ok(n)
	396	}
	397	}
	398
	399	#[cfg(test)]
	400	mod tests {
	401	use super::*;
	402
	403	#[test]
	404	fn round_trips() {
	405	let docs = [
	406	r#"null"#,
	407	r#"[true,false,0,-12,9007199254740993,1.5,"a"]"#,
	408	r#"{"b":1,"a":{"nested":[]},"s":"quote \" backslash \\ newline \n tab \t"}"#,
	409	r#""unicode: é 中文 🌱""#,
	410	];
	411	for d in docs {
	412	let v = parse(d).unwrap();
	413	assert_eq!(parse(&v.encode()).unwrap(), v, "{d}");
	414	}
	415	// Key order and exact integers survive.
	416	assert_eq!(parse(r#"{"z":1,"a":9007199254740993}"#).unwrap().encode(), r#"{"z":1,"a":9007199254740993}"#);
	417	}
	418
	419	#[test]
	420	fn escapes() {
	421	assert_eq!(parse(r#""\u00e9\ud83c\udf31\/""#).unwrap(), Value::Str("é🌱/".into()));
	422	assert_eq!(Value::Str("\u{1}".into()).encode(), r#""\u0001""#);
	423	}
	424
	425	#[test]
	426	fn rejects_invalid() {
	427	for bad in [
	428	"", "[", "[1,]", "{\"a\":1,}", "01", "1.", "-", "\"\\x\"", "\"\\ud800\"", "\"a\nb\"",
	429	"{\"a\":1,\"a\":2}", "nul", "[1] 2", "{a:1}",
	430	] {
	431	assert!(parse(bad).is_err(), "should reject {bad:?}");
	432	}
	433	let deep = "[".repeat(200) + &"]".repeat(200);
	434	assert!(parse(&deep).is_err());
	435	}
	436	}

	1	//! grove: where agents build software for people. See VISION.md and docs/model.md.
	2
	3	pub mod blake3;
	4	pub mod cli;
	5	pub mod diff;
	6	pub mod hex;
	7	pub mod ignore;
	8	pub mod json;
	9	pub mod log;
	10	pub mod model;
	11	pub mod objects;
	12	pub mod random;
	13	pub mod store;
	14	pub mod workdir;
	15
	16	#[cfg(test)]
	17	mod testdir;

	1	//! The append-only log: every fact grove knows, in order.
	2	//!
	3	//! File layout: an 8-byte header `grovelg1`, then records. Each record is
	4	//!
	5	//! ```text
	6	//! length: u32 little-endian (payload bytes)
	7	//! length check: 4 bytes (first 4 bytes of BLAKE3(length))
	8	//! payload check: 8 bytes (first 8 bytes of BLAKE3(payload))
	9	//! payload
	10	//! ```
	11	//!
	12	//! An append is acknowledged only once flushed to disk. A crash mid-append can leave a torn
	13	//! last record: a prefix of it, or (on some filesystems) zeros where its bytes should be.
	14	//! Opening the log recognises exactly those cases and cuts the file back to the last whole
	15	//! record. Anything else that fails a check is damage, and opening refuses rather than guess;
	16	//! the separate length check means a damaged length can't pass for a torn tail and take the
	17	//! records after it with it. While open, the log holds an exclusive lock on its file.
	18
	19	use crate::blake3;
	20	use std::fs::{File, OpenOptions, TryLockError};
	21	use std::io::{self, Read, Seek, SeekFrom, Write};
	22	use std::path::Path;
	23
	24	const HEADER: &[u8; 8] = b"grovelg1";
	25	const FRAME: usize = 4 + 4 + 8;
	26	/// Largest record grove will write or read: 64 MiB.
	27	pub const MAX_RECORD: usize = 64 << 20;
	28
	29	pub struct Log {
	30	file: File,
	31	len: u64,
	32	count: u64,
	33	}
	34
	35	#[derive(Debug)]
	36	pub enum OpenError {
	37	Io(io::Error),
	38	Locked,
	39	NotALog,
	40	Damaged { offset: u64, reason: &'static str },
	41	}
	42
	43	impl std::fmt::Display for OpenError {
	44	fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
	45	match self {
	46	OpenError::Io(e) => write!(f, "{e}"),
	47	OpenError::Locked => write!(f, "another grove process has this store open"),
	48	OpenError::NotALog => write!(f, "not a grove log"),
	49	OpenError::Damaged { offset, reason } => write!(f, "log damaged at byte {offset}: {reason}"),
	50	}
	51	}
	52	}
	53
	54	impl From<io::Error> for OpenError {
	55	fn from(e: io::Error) -> Self {
	56	OpenError::Io(e)
	57	}
	58	}
	59
	60	/// What opening found, besides the records.
	61	#[derive(Debug, Default, PartialEq)]
	62	pub struct Recovery {
	63	/// Bytes cut from the end because the last record was torn.
	64	pub truncated: u64,
	65	}
	66
	67	fn check(payload: &[u8]) -> [u8; 8] {
	68	let h = blake3::hash(payload);
	69	let mut c = [0; 8];
	70	c.copy_from_slice(&h[..8]);
	71	c
	72	}
	73
	74	fn len_check(len: [u8; 4]) -> [u8; 4] {
	75	let h = blake3::hash(&len);
	76	[h[0], h[1], h[2], h[3]]
	77	}
	78
	79	/// True if a crash could have left these bytes at the end of the file: nothing but zeros.
	80	fn zeros(bytes: &[u8]) -> bool {
	81	bytes.iter().all(\|&b\| b == 0)
	82	}
	83
	84	impl Log {
	85	/// Open (creating if needed) and read every record.
	86	pub fn open(path: &Path) -> Result<(Log, Vec<Vec<u8>>, Recovery), OpenError> {
	87	let mut file = OpenOptions::new().read(true).write(true).create(true).truncate(false).open(path)?;
	88	match file.try_lock() {
	89	Ok(()) => {}
	90	Err(TryLockError::WouldBlock) => return Err(OpenError::Locked),
	91	Err(TryLockError::Error(e)) => return Err(e.into()),
	92	}
	93
	94	let mut bytes = Vec::new();
	95	file.read_to_end(&mut bytes)?;
	96	if bytes.is_empty() {
	97	file.write_all(HEADER)?;
	98	file.sync_all()?;
	99	sync_parent(path)?;
	100	return Ok((Log { file, len: HEADER.len() as u64, count: 0 }, Vec::new(), Recovery::default()));
	101	}
	102	if bytes.len() < HEADER.len() \|\| &bytes[..HEADER.len()] != HEADER {
	103	return Err(OpenError::NotALog);
	104	}
	105
	106	let mut records = Vec::new();
	107	let mut at = HEADER.len();
	108	let mut recovery = Recovery::default();
	109	while at < bytes.len() {
	110	let rest = &bytes[at..];
	111	let damaged = \|reason\| Err(OpenError::Damaged { offset: at as u64, reason });
	112	if rest.len() < FRAME {
	113	// A frame cut short: the crash came while writing it.
	114	recovery.truncated = rest.len() as u64;
	115	break;
	116	}
	117	let len_bytes: [u8; 4] = rest[..4].try_into().unwrap();
	118	if len_check(len_bytes) != rest[4..8] {
	119	if zeros(rest) {
	120	recovery.truncated = rest.len() as u64;
	121	break;
	122	}
	123	return damaged("record header fails its check");
	124	}
	125	let len = u32::from_le_bytes(len_bytes) as usize;
	126	if len > MAX_RECORD {
	127	return damaged("record length out of range");
	128	}
	129	if rest.len() < FRAME + len {
	130	// A sound header with the payload cut short.
	131	recovery.truncated = rest.len() as u64;
	132	break;
	133	}
	134	let payload = &rest[FRAME..FRAME + len];
	135	if check(payload) != rest[8..FRAME] {
	136	// Torn only if this is the last record; anything after it means real damage.
	137	if FRAME + len == rest.len() \|\| zeros(&rest[FRAME + len..]) {
	138	recovery.truncated = rest.len() as u64;
	139	break;
	140	}
	141	return damaged("record fails its check");
	142	}
	143	records.push(payload.to_vec());
	144	at += FRAME + len;
	145	}
	146
	147	if recovery.truncated > 0 {
	148	file.set_len(at as u64)?;
	149	file.sync_all()?;
	150	}
	151	file.seek(SeekFrom::Start(at as u64))?;
	152	let count = records.len() as u64;
	153	Ok((Log { file, len: at as u64, count }, records, recovery))
	154	}
	155
	156	/// Append one record and flush it to disk. Returns the record's index.
	157	pub fn append(&mut self, payload: &[u8]) -> io::Result<u64> {
	158	if payload.len() > MAX_RECORD {
	159	return Err(io::Error::new(io::ErrorKind::InvalidInput, "record too large"));
	160	}
	161	let mut frame = Vec::with_capacity(FRAME + payload.len());
	162	let len = (payload.len() as u32).to_le_bytes();
	163	frame.extend_from_slice(&len);
	164	frame.extend_from_slice(&len_check(len));
	165	frame.extend_from_slice(&check(payload));
	166	frame.extend_from_slice(payload);
	167	let result = self.file.write_all(&frame).and_then(\|_\| self.file.sync_data());
	168	if let Err(e) = result {
	169	// Leave the file as it was before this append, so the next one starts clean.
	170	let _ = self.file.set_len(self.len);
	171	let _ = self.file.seek(SeekFrom::Start(self.len));
	172	return Err(e);
	173	}
	174	self.len += frame.len() as u64;
	175	self.count += 1;
	176	Ok(self.count - 1)
	177	}
	178
	179	pub fn len(&self) -> u64 {
	180	self.len
	181	}
	182
	183	pub fn count(&self) -> u64 {
	184	self.count
	185	}
	186
	187	pub fn is_empty(&self) -> bool {
	188	self.count == 0
	189	}
	190	}
	191
	192	/// Flush a directory entry (a newly created or renamed file) to disk.
	193	pub fn sync_parent(path: &Path) -> io::Result<()> {
	194	match path.parent() {
	195	Some(dir) if !dir.as_os_str().is_empty() => File::open(dir)?.sync_all(),
	196	_ => Ok(()),
	197	}
	198	}
	199
	200	#[cfg(test)]
	201	mod tests {
	202	use super::*;
	203	use crate::testdir::TestDir;
	204
	205	fn reopen(path: &Path) -> (Log, Vec<Vec<u8>>, Recovery) {
	206	Log::open(path).unwrap()
	207	}
	208
	209	#[test]
	210	fn append_and_replay() {
	211	let dir = TestDir::new("log-replay");
	212	let path = dir.path().join("log");
	213	{
	214	let (mut log, recs, _) = reopen(&path);
	215	assert!(recs.is_empty());
	216	assert_eq!(log.append(b"one").unwrap(), 0);
	217	assert_eq!(log.append(b"").unwrap(), 1);
	218	assert_eq!(log.append(&vec![7u8; 100_000]).unwrap(), 2);
	219	}
	220	let (log, recs, rec) = reopen(&path);
	221	assert_eq!(recs, vec![b"one".to_vec(), Vec::new(), vec![7u8; 100_000]]);
	222	assert_eq!(log.count(), 3);
	223	assert_eq!(rec, Recovery::default());
	224	}
	225
	226	#[test]
	227	fn second_open_is_locked_out() {
	228	let dir = TestDir::new("log-lock");
	229	let path = dir.path().join("log");
	230	let _held = reopen(&path);
	231	assert!(matches!(Log::open(&path), Err(OpenError::Locked)));
	232	}
	233
	234	#[test]
	235	fn torn_tail_is_cut_at_every_byte() {
	236	// Simulate a crash at every possible point while writing the third record.
	237	let dir = TestDir::new("log-torn");
	238	let full = dir.path().join("full");
	239	{
	240	let (mut log, _, _) = reopen(&full);
	241	log.append(b"first").unwrap();
	242	log.append(b"second").unwrap();
	243	log.append(b"third record").unwrap();
	244	}
	245	let bytes = std::fs::read(&full).unwrap();
	246	let two_records = HEADER.len() + 2 * FRAME + b"first".len() + b"second".len();
	247	for cut in two_records..bytes.len() {
	248	let path = dir.path().join(format!("cut{cut}"));
	249	std::fs::write(&path, &bytes[..cut]).unwrap();
	250	let (mut log, recs, rec) = reopen(&path);
	251	assert_eq!(recs, vec![b"first".to_vec(), b"second".to_vec()], "cut at {cut}");
	252	assert_eq!(rec.truncated, (cut - two_records) as u64);
	253	// The log is usable again afterwards.
	254	log.append(b"after").unwrap();
	255	drop(log);
	256	assert_eq!(reopen(&path).1.len(), 3);
	257	}
	258	}
	259
	260	#[test]
	261	fn garbage_in_last_record_is_cut() {
	262	let dir = TestDir::new("log-garbage-tail");
	263	let path = dir.path().join("log");
	264	{
	265	let (mut log, _, _) = reopen(&path);
	266	log.append(b"keep").unwrap();
	267	log.append(b"lose").unwrap();
	268	}
	269	let mut bytes = std::fs::read(&path).unwrap();
	270	let n = bytes.len();
	271	bytes[n - 1] ^= 0xff; // the tail record's payload no longer matches its check
	272	std::fs::write(&path, &bytes).unwrap();
	273	let (_, recs, rec) = reopen(&path);
	274	assert_eq!(recs, vec![b"keep".to_vec()]);
	275	assert!(rec.truncated > 0);
	276	}
	277
	278	#[test]
	279	fn damage_before_the_tail_is_refused() {
	280	let dir = TestDir::new("log-damage");
	281	let path = dir.path().join("log");
	282	{
	283	let (mut log, _, _) = reopen(&path);
	284	log.append(b"first").unwrap();
	285	log.append(b"second").unwrap();
	286	}
	287	let mut bytes = std::fs::read(&path).unwrap();
	288	bytes[HEADER.len() + FRAME] ^= 0xff; // inside the first payload
	289	std::fs::write(&path, &bytes).unwrap();
	290	assert!(matches!(Log::open(&path), Err(OpenError::Damaged { .. })));
	291	// And nothing was cut.
	292	assert_eq!(std::fs::read(&path).unwrap(), bytes);
	293	}
	294
	295	#[test]
	296	fn damaged_length_does_not_eat_later_records() {
	297	let dir = TestDir::new("log-bad-length");
	298	let path = dir.path().join("log");
	299	{
	300	let (mut log, _, _) = reopen(&path);
	301	log.append(b"first").unwrap();
	302	log.append(b"second").unwrap();
	303	log.append(b"third").unwrap();
	304	}
	305	let mut bytes = std::fs::read(&path).unwrap();
	306	bytes[HEADER.len() + 1] = 0x40; // first record now claims to be 16 MiB long
	307	std::fs::write(&path, &bytes).unwrap();
	308	assert!(matches!(Log::open(&path), Err(OpenError::Damaged { .. })));
	309	assert_eq!(std::fs::read(&path).unwrap(), bytes, "nothing cut");
	310	}
	311
	312	#[test]
	313	fn zero_filled_tail_is_cut() {
	314	// Some filesystems extend the file before the data lands; a crash leaves zeros.
	315	let dir = TestDir::new("log-zeros");
	316	let path = dir.path().join("log");
	317	{
	318	let (mut log, _, _) = reopen(&path);
	319	log.append(b"kept").unwrap();
	320	}
	321	let mut bytes = std::fs::read(&path).unwrap();
	322	bytes.extend_from_slice(&[0u8; 300]);
	323	std::fs::write(&path, &bytes).unwrap();
	324	let (_, recs, rec) = reopen(&path);
	325	assert_eq!(recs, vec![b"kept".to_vec()]);
	326	assert_eq!(rec.truncated, 300);
	327	}
	328
	329	#[test]
	330	fn not_a_log() {
	331	let dir = TestDir::new("log-foreign");
	332	let path = dir.path().join("log");
	333	std::fs::write(&path, b"hello world").unwrap();
	334	assert!(matches!(Log::open(&path), Err(OpenError::NotALog)));
	335	}
	336	}

	1	fn main() {
	2	let argv: Vec<String> = std::env::args().collect();
	3	if let Err(msg) = grove::cli::run(argv) {
	4	eprintln!("grove: {msg}");
	5	std::process::exit(1);
	6	}
	7	}

	1	//! grove's state: what the log's records add up to.
	2	//!
	3	//! Every record is a JSON object `{"t": type, "at": ms, "by": actor, …}`. `apply` folds one
	4	//! record into the state. Replaying the log from the start rebuilds the state exactly, so
	5	//! `apply` must be deterministic and must accept only records the store would have written:
	6	//! a record that doesn't fit means the log is damaged, and the store refuses to open.
	7	//!
	8	//! `apply` is all or nothing: it checks a record completely before changing anything, so the
	9	//! store can use it to validate a write before appending it.
	10
	11	use crate::hex;
	12	use crate::json::{obj, Value};
	13	use crate::objects::Id;
	14
	15	#[derive(Clone, Debug, PartialEq, Eq)]
	16	pub struct Actor {
	17	pub person: String,
	18	/// The agent acting for the person, if it wasn't the person themselves.
	19	pub agent: Option<String>,
	20	}
	21
	22	impl Actor {
	23	pub fn person(name: &str) -> Actor {
	24	Actor { person: name.into(), agent: None }
	25	}
	26	pub fn agent(person: &str, agent: &str) -> Actor {
	27	Actor { person: person.into(), agent: Some(agent.into()) }
	28	}
	29	pub fn to_json(&self) -> Value {
	30	obj([("person", self.person.clone().into()), ("agent", self.agent.clone().into())])
	31	}
	32	fn from_json(v: &Value) -> Result<Actor, String> {
	33	Ok(Actor {
	34	person: str_field(v, "person")?,
	35	agent: match v.get("agent") {
	36	None \| Some(Value::Null) => None,
	37	Some(a) => Some(a.as_str().ok_or("agent must be a string")?.to_string()),
	38	},
	39	})
	40	}
	41	}
	42
	43	impl std::fmt::Display for Actor {
	44	fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
	45	match &self.agent {
	46	Some(a) => write!(f, "{a} for {}", self.person),
	47	None => write!(f, "{}", self.person),
	48	}
	49	}
	50	}
	51
	52	#[derive(Clone, Copy, Debug, PartialEq, Eq)]
	53	pub enum ChangeState {
	54	Idea,
	55	Draft,
	56	Proposed,
	57	Landed,
	58	Declined,
	59	Withdrawn,
	60	}
	61
	62	impl ChangeState {
	63	pub fn name(self) -> &'static str {
	64	match self {
	65	ChangeState::Idea => "idea",
	66	ChangeState::Draft => "draft",
	67	ChangeState::Proposed => "proposed",
	68	ChangeState::Landed => "landed",
	69	ChangeState::Declined => "declined",
	70	ChangeState::Withdrawn => "withdrawn",
	71	}
	72	}
	73	pub fn parse(s: &str) -> Option<ChangeState> {
	74	Some(match s {
	75	"idea" => ChangeState::Idea,
	76	"draft" => ChangeState::Draft,
	77	"proposed" => ChangeState::Proposed,
	78	"landed" => ChangeState::Landed,
	79	"declined" => ChangeState::Declined,
	80	"withdrawn" => ChangeState::Withdrawn,
	81	_ => return None,
	82	})
	83	}
	84	pub fn is_open(self) -> bool {
	85	matches!(self, ChangeState::Idea \| ChangeState::Draft \| ChangeState::Proposed)
	86	}
	87	}
	88
	89	#[derive(Clone, Debug)]
	90	pub struct Person {
	91	pub username: String,
	92	pub display_name: String,
	93	pub at: i64,
	94	}
	95
	96	#[derive(Clone, Debug)]
	97	pub struct Agent {
	98	pub id: String,
	99	pub person: String,
	100	pub name: String,
	101	pub revoked: bool,
	102	pub at: i64,
	103	}
	104
	105	#[derive(Clone, Debug)]
	106	pub struct Point {
	107	pub n: u64,
	108	pub snapshot: Id,
	109	pub message: String,
	110	pub author: Actor,
	111	pub landed_by: Actor,
	112	pub change: Option<u64>,
	113	pub step: Option<String>,
	114	pub at: i64,
	115	}
	116
	117	#[derive(Clone, Debug, PartialEq)]
	118	pub struct Step {
	119	/// Stable within a change, across revisions.
	120	pub id: String,
	121	pub snapshot: Id,
	122	pub message: String,
	123	}
	124
	125	#[derive(Clone, Debug)]
	126	pub struct Revision {
	127	pub number: u32,
	128	pub base: u64,
	129	pub steps: Vec<Step>,
	130	pub note: Option<String>,
	131	pub by: Actor,
	132	pub at: i64,
	133	}
	134
	135	#[derive(Clone, Debug)]
	136	pub struct Message {
	137	pub id: u32,
	138	pub body: String,
	139	pub by: Actor,
	140	pub at: i64,
	141	}
	142
	143	#[derive(Clone, Debug)]
	144	pub struct Evidence {
	145	pub revision: u32,
	146	pub title: String,
	147	pub media_type: String,
	148	pub blob: Option<Id>,
	149	pub url: Option<String>,
	150	pub by: Actor,
	151	pub at: i64,
	152	}
	153
	154	/// Something that happened to a change, for its timeline.
	155	#[derive(Clone, Debug)]
	156	pub enum Event {
	157	Opened,
	158	Revised(u32),
	159	State(ChangeState),
	160	Said(u32),
	161	Attached(usize),
	162	Landed { revision: u32, points: (u64, u64) },
	163	}
	164
	165	#[derive(Clone, Debug)]
	166	pub struct Change {
	167	pub number: u64,
	168	pub intent: String,
	169	pub proposer: Actor,
	170	pub state: ChangeState,
	171	pub revisions: Vec<Revision>,
	172	pub messages: Vec<Message>,
	173	pub evidence: Vec<Evidence>,
	174	pub timeline: Vec<(i64, Actor, Event)>,
	175	pub at: i64,
	176	}
	177
	178	impl Change {
	179	pub fn latest(&self) -> Option<&Revision> {
	180	self.revisions.last()
	181	}
	182	}
	183
	184	#[derive(Clone, Debug)]
	185	pub struct Project {
	186	pub owner: String,
	187	pub name: String,
	188	pub public: bool,
	189	pub points: Vec<Point>,
	190	pub changes: Vec<Change>,
	191	pub at: i64,
	192	}
	193
	194	impl Project {
	195	pub fn tip(&self) -> u64 {
	196	self.points.len() as u64
	197	}
	198	pub fn point(&self, n: u64) -> Option<&Point> {
	199	if n == 0 { None } else { self.points.get(n as usize - 1) }
	200	}
	201	pub fn change(&self, number: u64) -> Option<&Change> {
	202	if number == 0 { None } else { self.changes.get(number as usize - 1) }
	203	}
	204	fn change_mut(&mut self, number: u64) -> Option<&mut Change> {
	205	if number == 0 { None } else { self.changes.get_mut(number as usize - 1) }
	206	}
	207	}
	208
	209	#[derive(Default)]
	210	pub struct State {
	211	pub people: Vec<Person>,
	212	pub agents: Vec<Agent>,
	213	pub projects: Vec<Project>,
	214	pub records: u64,
	215	}
	216
	217	// ── Reading fields ─────────────────────────────────────────
	218
	219	fn str_field(v: &Value, key: &str) -> Result<String, String> {
	220	v.get(key).and_then(Value::as_str).map(str::to_string).ok_or_else(\|\| format!("missing {key}"))
	221	}
	222	fn opt_str(v: &Value, key: &str) -> Result<Option<String>, String> {
	223	match v.get(key) {
	224	None \| Some(Value::Null) => Ok(None),
	225	Some(Value::Str(s)) => Ok(Some(s.clone())),
	226	_ => Err(format!("{key} must be a string")),
	227	}
	228	}
	229	fn int_field(v: &Value, key: &str) -> Result<i64, String> {
	230	v.get(key).and_then(Value::as_i64).ok_or_else(\|\| format!("missing {key}"))
	231	}
	232	fn uint(v: &Value, key: &str) -> Result<u64, String> {
	233	let n = int_field(v, key)?;
	234	u64::try_from(n).map_err(\|_\| format!("{key} must not be negative"))
	235	}
	236	pub fn id_to_json(id: &Id) -> Value {
	237	Value::Str(hex::encode(id))
	238	}
	239	fn id_value(v: &Value) -> Result<Id, String> {
	240	let s = v.as_str().ok_or("an id must be a string")?;
	241	let bytes = hex::decode(s).ok_or("an id must be hex")?;
	242	bytes.try_into().map_err(\|_\| "an id must be 32 bytes".to_string())
	243	}
	244	fn id_field(v: &Value, key: &str) -> Result<Id, String> {
	245	id_value(v.get(key).ok_or_else(\|\| format!("missing {key}"))?).map_err(\|e\| format!("{key}: {e}"))
	246	}
	247	fn opt_id(v: &Value, key: &str) -> Result<Option<Id>, String> {
	248	match v.get(key) {
	249	None \| Some(Value::Null) => Ok(None),
	250	_ => id_field(v, key).map(Some),
	251	}
	252	}
	253
	254	pub fn step_to_json(s: &Step) -> Value {
	255	obj([("id", s.id.clone().into()), ("snapshot", id_to_json(&s.snapshot)), ("message", s.message.clone().into())])
	256	}
	257	fn step_from_json(v: &Value) -> Result<Step, String> {
	258	Ok(Step { id: str_field(v, "id")?, snapshot: id_field(v, "snapshot")?, message: str_field(v, "message")? })
	259	}
	260
	261	// ── Applying records ───────────────────────────────────────
	262
	263	impl State {
	264	pub fn person(&self, username: &str) -> Option<&Person> {
	265	self.people.iter().find(\|p\| p.username == username)
	266	}
	267	pub fn agent(&self, id: &str) -> Option<&Agent> {
	268	self.agents.iter().find(\|a\| a.id == id)
	269	}
	270	pub fn project(&self, owner: &str, name: &str) -> Option<&Project> {
	271	self.projects.iter().find(\|p\| p.owner == owner && p.name == name)
	272	}
	273	fn project_mut(&mut self, owner: &str, name: &str) -> Result<&mut Project, String> {
	274	self.projects.iter_mut().find(\|p\| p.owner == owner && p.name == name).ok_or_else(\|\| format!("no project {owner}/{name}"))
	275	}
	276
	277	/// Fold one record into the state.
	278	pub fn apply(&mut self, rec: &Value) -> Result<(), String> {
	279	let t = str_field(rec, "t")?;
	280	let at = int_field(rec, "at")?;
	281	let by = Actor::from_json(rec.get("by").ok_or("missing by")?)?;
	282	if t != "person" && self.person(&by.person).is_none() {
	283	return Err(format!("unknown person {}", by.person));
	284	}
	285	if let Some(agent) = &by.agent {
	286	match self.agent(agent) {
	287	Some(a) if a.person == by.person && !a.revoked => {}
	288	_ => return Err(format!("agent {agent} can't act for {}", by.person)),
	289	}
	290	}
	291	match t.as_str() {
	292	"person" => {
	293	let username = str_field(rec, "username")?;
	294	if username != by.person {
	295	return Err("a person record is made by that person".into());
	296	}
	297	if self.person(&username).is_some() {
	298	return Err(format!("person {username} exists"));
	299	}
	300	self.people.push(Person { display_name: str_field(rec, "display_name")?, username, at });
	301	}
	302	"agent" => {
	303	let id = str_field(rec, "id")?;
	304	if self.agent(&id).is_some() {
	305	return Err(format!("agent {id} exists"));
	306	}
	307	self.agents.push(Agent { id, person: by.person.clone(), name: str_field(rec, "name")?, revoked: false, at });
	308	}
	309	"agent.revoke" => {
	310	let id = str_field(rec, "id")?;
	311	let a = self.agents.iter_mut().find(\|a\| a.id == id && a.person == by.person).ok_or("no such agent")?;
	312	a.revoked = true;
	313	}
	314	"project" => {
	315	let name = str_field(rec, "name")?;
	316	if self.project(&by.person, &name).is_some() {
	317	return Err(format!("project {}/{name} exists", by.person));
	318	}
	319	let public = matches!(rec.get("public"), Some(Value::Bool(true)));
	320	self.projects.push(Project { owner: by.person.clone(), name, public, points: Vec::new(), changes: Vec::new(), at });
	321	}
	322	"point" => {
	323	// A point recorded directly (the first snapshot of a project, or imported history).
	324	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	325	if by.person != p.owner {
	326	return Err("only the owner records points".into());
	327	}
	328	let n = uint(rec, "n")?;
	329	if n != p.tip() + 1 {
	330	return Err(format!("point {n} doesn't follow {}", p.tip()));
	331	}
	332	let author = match rec.get("author") {
	333	Some(a) => Actor::from_json(a)?,
	334	None => by.clone(),
	335	};
	336	p.points.push(Point { n, snapshot: id_field(rec, "snapshot")?, message: str_field(rec, "message")?, author, landed_by: by, change: None, step: None, at });
	337	}
	338	"change" => {
	339	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	340	let number = uint(rec, "number")?;
	341	if number != p.changes.len() as u64 + 1 {
	342	return Err(format!("change {number} out of sequence"));
	343	}
	344	let state = ChangeState::parse(&str_field(rec, "state")?).ok_or("bad state")?;
	345	if !matches!(state, ChangeState::Idea \| ChangeState::Draft) {
	346	return Err("a change opens as an idea or a draft".into());
	347	}
	348	p.changes.push(Change {
	349	number,
	350	intent: str_field(rec, "intent")?,
	351	proposer: by.clone(),
	352	state,
	353	revisions: Vec::new(),
	354	messages: Vec::new(),
	355	evidence: Vec::new(),
	356	timeline: vec![(at, by, Event::Opened)],
	357	at,
	358	});
	359	}
	360	"revision" => {
	361	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	362	let tip = p.tip();
	363	let c = p.change_mut(uint(rec, "change")?).ok_or("no such change")?;
	364	if !c.state.is_open() {
	365	return Err("change is closed".into());
	366	}
	367	let number = uint(rec, "number")? as u32;
	368	if number as usize != c.revisions.len() + 1 {
	369	return Err("revision out of sequence".into());
	370	}
	371	let base = uint(rec, "base")?;
	372	if base == 0 \|\| base > tip {
	373	return Err(format!("base {base} isn't a point"));
	374	}
	375	let steps = rec.get("steps").and_then(Value::as_array).ok_or("missing steps")?;
	376	let steps = steps.iter().map(step_from_json).collect::<Result<Vec<_>, _>>()?;
	377	if steps.is_empty() {
	378	return Err("a revision needs at least one step".into());
	379	}
	380	c.revisions.push(Revision { number, base, steps, note: opt_str(rec, "note")?, by: by.clone(), at });
	381	if c.state == ChangeState::Idea {
	382	c.state = ChangeState::Draft;
	383	}
	384	c.timeline.push((at, by, Event::Revised(number)));
	385	}
	386	"state" => {
	387	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	388	let owner = p.owner.clone();
	389	let c = p.change_mut(uint(rec, "change")?).ok_or("no such change")?;
	390	let to = ChangeState::parse(&str_field(rec, "state")?).ok_or("bad state")?;
	391	check_transition(c, to, &by, &owner)?;
	392	c.state = to;
	393	c.timeline.push((at, by, Event::State(to)));
	394	}
	395	"land" => {
	396	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	397	if by.person != p.owner \|\| by.agent.is_some() {
	398	return Err("only the owner, in person, lands".into());
	399	}
	400	let number = uint(rec, "change")?;
	401	let revision = uint(rec, "revision")? as u32;
	402	let first = uint(rec, "first")?;
	403	if first != p.tip() + 1 {
	404	return Err("landing doesn't follow the tip".into());
	405	}
	406	let c = p.change(number).ok_or("no such change")?;
	407	if c.state != ChangeState::Proposed {
	408	return Err("only proposed changes land".into());
	409	}
	410	let rev = c.revisions.get((revision as usize).wrapping_sub(1)).ok_or("no such revision")?.clone();
	411	let snapshots = rec.get("snapshots").and_then(Value::as_array).ok_or("missing snapshots")?;
	412	let snapshots = snapshots.iter().map(id_value).collect::<Result<Vec<_>, _>>()?;
	413	if snapshots.len() != rev.steps.len() {
	414	return Err("one snapshot per step".into());
	415	}
	416	// Everything is checked; from here on nothing can fail.
	417	for (i, (step, snapshot)) in rev.steps.iter().zip(snapshots).enumerate() {
	418	p.points.push(Point {
	419	n: first + i as u64,
	420	snapshot,
	421	message: step.message.clone(),
	422	author: rev.by.clone(),
	423	landed_by: by.clone(),
	424	change: Some(number),
	425	step: Some(step.id.clone()),
	426	at,
	427	});
	428	}
	429	let last = p.tip();
	430	let c = p.change_mut(number).unwrap();
	431	c.state = ChangeState::Landed;
	432	c.timeline.push((at, by, Event::Landed { revision, points: (first, last) }));
	433	}
	434	"say" => {
	435	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	436	let c = p.change_mut(uint(rec, "change")?).ok_or("no such change")?;
	437	let id = c.messages.len() as u32 + 1;
	438	c.messages.push(Message { id, body: str_field(rec, "body")?, by: by.clone(), at });
	439	c.timeline.push((at, by, Event::Said(id)));
	440	}
	441	"evidence" => {
	442	let p = self.project_mut(&str_field(rec, "owner")?, &str_field(rec, "project")?)?;
	443	let c = p.change_mut(uint(rec, "change")?).ok_or("no such change")?;
	444	let revision = uint(rec, "revision")? as u32;
	445	if revision == 0 \|\| revision as usize > c.revisions.len() {
	446	return Err("no such revision".into());
	447	}
	448	let blob = opt_id(rec, "blob")?;
	449	let url = opt_str(rec, "url")?;
	450	if blob.is_none() == url.is_none() {
	451	return Err("evidence is a blob or a url".into());
	452	}
	453	c.evidence.push(Evidence { revision, title: str_field(rec, "title")?, media_type: str_field(rec, "type")?, blob, url, by: by.clone(), at });
	454	let i = c.evidence.len() - 1;
	455	c.timeline.push((at, by, Event::Attached(i)));
	456	}
	457	other => return Err(format!("unknown record type {other}")),
	458	}
	459	self.records += 1;
	460	Ok(())
	461	}
	462	}
	463
	464	/// Who may move a change between which states.
	465	fn check_transition(c: &Change, to: ChangeState, by: &Actor, owner: &str) -> Result<(), String> {
	466	use ChangeState::*;
	467	let is_owner = by.person == owner;
	468	// Decisions are the owner's own: an agent acting for them can't make them.
	469	let owner_in_person = is_owner && by.agent.is_none();
	470	let is_proposer = by.person == c.proposer.person;
	471	match (c.state, to) {
	472	(Draft, Proposed) if is_proposer \|\| is_owner => Ok(()),
	473	(Proposed, Draft) if is_proposer \|\| is_owner => Ok(()),
	474	(Idea \| Draft \| Proposed, Declined) if owner_in_person => Ok(()),
	475	(Idea \| Draft \| Proposed, Withdrawn) if is_proposer => Ok(()),
	476	(Declined \| Withdrawn, Draft) if (is_proposer \|\| is_owner) && !c.revisions.is_empty() => Ok(()),
	477	(Declined \| Withdrawn, Idea) if (is_proposer \|\| is_owner) && c.revisions.is_empty() => Ok(()),
	478	(from, to) if from == to => Err(format!("already {}", to.name())),
	479	(from, to) => Err(format!("{} can't move a change from {} to {}", by, from.name(), to.name())),
	480	}
	481	}

	1	//! Objects: file contents (blobs) and directories (trees), stored by the hash of their bytes.
	2	//!
	3	//! An object's file holds exactly the bytes its id is the hash of: a type prefix
	4	//! (`grove blob\0` or `grove tree\0`) and the content. So any object can be checked by hashing
	5	//! its file, and a blob and a tree can never share an id.
	6	//!
	7	//! Writing: temporary file, flush, rename into place, flush the directory. An object either
	8	//! exists completely or not at all, and is never rewritten.
	9
	10	use crate::blake3::Hasher;
	11	use crate::hex;
	12	use crate::log::sync_parent;
	13	use std::fs::{self, File};
	14	use std::io::{self, Write};
	15	use std::path::{Path, PathBuf};
	16
	17	pub type Id = [u8; 32];
	18
	19	const BLOB: &[u8] = b"grove blob\0";
	20	const TREE: &[u8] = b"grove tree\0";
	21
	22	#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
	23	pub enum Kind {
	24	File,
	25	Exec,
	26	Link,
	27	Dir,
	28	}
	29
	30	impl Kind {
	31	fn byte(self) -> u8 {
	32	match self {
	33	Kind::File => b'f',
	34	Kind::Exec => b'x',
	35	Kind::Link => b'l',
	36	Kind::Dir => b'd',
	37	}
	38	}
	39	fn from_byte(b: u8) -> Option<Kind> {
	40	Some(match b {
	41	b'f' => Kind::File,
	42	b'x' => Kind::Exec,
	43	b'l' => Kind::Link,
	44	b'd' => Kind::Dir,
	45	_ => return None,
	46	})
	47	}
	48	}
	49
	50	#[derive(Clone, Debug, PartialEq, Eq)]
	51	pub struct Entry {
	52	pub name: String,
	53	pub kind: Kind,
	54	pub id: Id,
	55	}
	56
	57	#[derive(Debug)]
	58	pub enum Error {
	59	Io(io::Error),
	60	Missing(Id),
	61	Corrupt(Id),
	62	BadName(String),
	63	}
	64
	65	impl std::fmt::Display for Error {
	66	fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
	67	match self {
	68	Error::Io(e) => write!(f, "{e}"),
	69	Error::Missing(id) => write!(f, "object {} is missing", hex::encode(id)),
	70	Error::Corrupt(id) => write!(f, "object {} is corrupt", hex::encode(id)),
	71	Error::BadName(n) => write!(f, "invalid file name {n:?}"),
	72	}
	73	}
	74	}
	75
	76	impl From<io::Error> for Error {
	77	fn from(e: io::Error) -> Self {
	78	Error::Io(e)
	79	}
	80	}
	81
	82	pub type Result<T> = std::result::Result<T, Error>;
	83
	84	/// Names must be usable as one path component everywhere.
	85	pub fn valid_name(name: &str) -> bool {
	86	!name.is_empty() && name != "." && name != ".." && !name.contains('/') && !name.contains('\0')
	87	}
	88
	89	fn id_of(prefix: &[u8], content: &[u8]) -> Id {
	90	let mut h = Hasher::new();
	91	h.update(prefix).update(content);
	92	h.finalize()
	93	}
	94
	95	pub fn blob_id(content: &[u8]) -> Id {
	96	id_of(BLOB, content)
	97	}
	98
	99	/// Canonical tree bytes: entries sorted by name, each `kind, name length (u32 LE), name, id`.
	100	pub fn encode_tree(entries: &[Entry]) -> Result<Vec<u8>> {
	101	let mut sorted: Vec<&Entry> = entries.iter().collect();
	102	sorted.sort_by(\|a, b\| a.name.as_bytes().cmp(b.name.as_bytes()));
	103	let mut out = Vec::new();
	104	for (i, e) in sorted.iter().enumerate() {
	105	if !valid_name(&e.name) {
	106	return Err(Error::BadName(e.name.clone()));
	107	}
	108	if i > 0 && sorted[i - 1].name == e.name {
	109	return Err(Error::BadName(format!("{} (twice)", e.name)));
	110	}
	111	out.push(e.kind.byte());
	112	out.extend_from_slice(&(e.name.len() as u32).to_le_bytes());
	113	out.extend_from_slice(e.name.as_bytes());
	114	out.extend_from_slice(&e.id);
	115	}
	116	Ok(out)
	117	}
	118
	119	fn decode_tree(id: &Id, mut bytes: &[u8]) -> Result<Vec<Entry>> {
	120	let corrupt = \|\| Error::Corrupt(*id);
	121	let mut entries = Vec::new();
	122	while !bytes.is_empty() {
	123	let kind = Kind::from_byte(bytes[0]).ok_or_else(corrupt)?;
	124	let len = u32::from_le_bytes(bytes.get(1..5).ok_or_else(corrupt)?.try_into().unwrap()) as usize;
	125	let name = std::str::from_utf8(bytes.get(5..5 + len).ok_or_else(corrupt)?).map_err(\|_\| corrupt())?;
	126	let id_bytes = bytes.get(5 + len..5 + len + 32).ok_or_else(corrupt)?;
	127	entries.push(Entry { name: name.to_string(), kind, id: id_bytes.try_into().unwrap() });
	128	bytes = &bytes[5 + len + 32..];
	129	}
	130	Ok(entries)
	131	}
	132
	133	pub struct Objects {
	134	root: PathBuf,
	135	}
	136
	137	impl Objects {
	138	pub fn open(root: &Path) -> Result<Objects> {
	139	fs::create_dir_all(root.join("tmp"))?;
	140	Ok(Objects { root: root.to_path_buf() })
	141	}
	142
	143	fn path(&self, id: &Id) -> PathBuf {
	144	let h = hex::encode(id);
	145	self.root.join(&h[..2]).join(&h[2..])
	146	}
	147
	148	pub fn has(&self, id: &Id) -> bool {
	149	self.path(id).is_file()
	150	}
	151
	152	fn put(&self, prefix: &[u8], content: &[u8]) -> Result<Id> {
	153	let id = id_of(prefix, content);
	154	let path = self.path(&id);
	155	if path.is_file() {
	156	return Ok(id);
	157	}
	158	fs::create_dir_all(path.parent().unwrap())?;
	159	let tmp = self.root.join("tmp").join(format!("{}-{}", hex::encode(&id[..8]), crate::random::hex(8)));
	160	{
	161	let mut f = File::create(&tmp)?;
	162	f.write_all(prefix)?;
	163	f.write_all(content)?;
	164	f.sync_all()?;
	165	}
	166	fs::rename(&tmp, &path)?;
	167	sync_parent(&path)?;
	168	Ok(id)
	169	}
	170
	171	/// Read an object's content, checking that it hashes to its id.
	172	fn get(&self, prefix: &[u8], id: &Id) -> Result<Vec<u8>> {
	173	let bytes = match fs::read(self.path(id)) {
	174	Ok(b) => b,
	175	Err(e) if e.kind() == io::ErrorKind::NotFound => return Err(Error::Missing(*id)),
	176	Err(e) => return Err(e.into()),
	177	};
	178	if !bytes.starts_with(prefix) \|\| crate::blake3::hash(&bytes) != *id {
	179	return Err(Error::Corrupt(*id));
	180	}
	181	Ok(bytes[prefix.len()..].to_vec())
	182	}
	183
	184	pub fn put_blob(&self, content: &[u8]) -> Result<Id> {
	185	self.put(BLOB, content)
	186	}
	187
	188	pub fn get_blob(&self, id: &Id) -> Result<Vec<u8>> {
	189	self.get(BLOB, id)
	190	}
	191
	192	pub fn put_tree(&self, entries: &[Entry]) -> Result<Id> {
	193	self.put(TREE, &encode_tree(entries)?)
	194	}
	195
	196	/// Entries sorted by name.
	197	pub fn get_tree(&self, id: &Id) -> Result<Vec<Entry>> {
	198	let bytes = self.get(TREE, id)?;
	199	decode_tree(id, &bytes)
	200	}
	201
	202	/// The id of an empty tree, stored.
	203	pub fn empty_tree(&self) -> Result<Id> {
	204	self.put_tree(&[])
	205	}
	206
	207	/// Every file under a tree, as (path, kind, blob id), sorted by path.
	208	pub fn flatten(&self, tree: &Id) -> Result<Vec<(String, Kind, Id)>> {
	209	let mut out = Vec::new();
	210	self.flatten_into(tree, "", &mut out)?;
	211	Ok(out)
	212	}
	213
	214	fn flatten_into(&self, tree: &Id, prefix: &str, out: &mut Vec<(String, Kind, Id)>) -> Result<()> {
	215	for e in self.get_tree(tree)? {
	216	let path = format!("{prefix}{}", e.name);
	217	if e.kind == Kind::Dir {
	218	self.flatten_into(&e.id, &format!("{path}/"), out)?;
	219	} else {
	220	out.push((path, e.kind, e.id));
	221	}
	222	}
	223	Ok(())
	224	}
	225
	226	/// Confirm every object under a tree exists and matches its id.
	227	pub fn verify_tree(&self, tree: &Id) -> Result<usize> {
	228	let mut count = 1;
	229	for e in self.get_tree(tree)? {
	230	count += if e.kind == Kind::Dir { self.verify_tree(&e.id)? } else { self.get_blob(&e.id).map(\|_\| 1)? };
	231	}
	232	Ok(count)
	233	}
	234	}
	235
	236	#[cfg(test)]
	237	mod tests {
	238	use super::*;
	239	use crate::testdir::TestDir;
	240
	241	#[test]
	242	fn blobs_round_trip_and_dedupe() {
	243	let dir = TestDir::new("objects-blobs");
	244	let objects = Objects::open(dir.path()).unwrap();
	245	let a = objects.put_blob(b"hello").unwrap();
	246	assert_eq!(objects.put_blob(b"hello").unwrap(), a);
	247	assert_ne!(objects.put_blob(b"hello!").unwrap(), a);
	248	assert_eq!(objects.get_blob(&a).unwrap(), b"hello");
	249	assert_eq!(a, blob_id(b"hello"));
	250	assert!(objects.has(&a));
	251	}
	252
	253	#[test]
	254	fn trees_are_canonical() {
	255	let dir = TestDir::new("objects-trees");
	256	let objects = Objects::open(dir.path()).unwrap();
	257	let f = objects.put_blob(b"x").unwrap();
	258	let e = \|name: &str\| Entry { name: name.into(), kind: Kind::File, id: f };
	259	let t1 = objects.put_tree(&[e("b"), e("a")]).unwrap();
	260	let t2 = objects.put_tree(&[e("a"), e("b")]).unwrap();
	261	assert_eq!(t1, t2, "order of input doesn't matter");
	262	let entries = objects.get_tree(&t1).unwrap();
	263	assert_eq!(entries.iter().map(\|e\| e.name.as_str()).collect::<Vec<_>>(), ["a", "b"]);
	264	// A tree and a blob with identical bytes still differ.
	265	assert_ne!(objects.empty_tree().unwrap(), objects.put_blob(b"").unwrap());
	266	}
	267
	268	#[test]
	269	fn rejects_bad_names() {
	270	let dir = TestDir::new("objects-names");
	271	let objects = Objects::open(dir.path()).unwrap();
	272	let f = objects.put_blob(b"x").unwrap();
	273	for bad in ["", ".", "..", "a/b", "nul\0"] {
	274	let e = Entry { name: bad.into(), kind: Kind::File, id: f };
	275	assert!(matches!(objects.put_tree(&[e]), Err(Error::BadName(_))), "{bad:?}");
	276	}
	277	let e = Entry { name: "same".into(), kind: Kind::File, id: f };
	278	assert!(objects.put_tree(&[e.clone(), e]).is_err());
	279	}
	280
	281	#[test]
	282	fn corruption_and_absence_are_detected() {
	283	let dir = TestDir::new("objects-corrupt");
	284	let objects = Objects::open(dir.path()).unwrap();
	285	let id = objects.put_blob(b"precious").unwrap();
	286	std::fs::write(objects.path(&id), b"grove blob\0tampered").unwrap();
	287	assert!(matches!(objects.get_blob(&id), Err(Error::Corrupt(_))));
	288	assert!(matches!(objects.get_blob(&blob_id(b"never stored")), Err(Error::Missing(_))));
	289	}
	290
	291	#[test]
	292	fn flatten_and_verify() {
	293	let dir = TestDir::new("objects-flatten");
	294	let objects = Objects::open(dir.path()).unwrap();
	295	let a = objects.put_blob(b"a").unwrap();
	296	let b = objects.put_blob(b"b").unwrap();
	297	let sub = objects.put_tree(&[Entry { name: "inner.txt".into(), kind: Kind::File, id: b }]).unwrap();
	298	let root = objects
	299	.put_tree(&[
	300	Entry { name: "run.sh".into(), kind: Kind::Exec, id: a },
	301	Entry { name: "dir".into(), kind: Kind::Dir, id: sub },
	302	])
	303	.unwrap();
	304	let files = objects.flatten(&root).unwrap();
	305	assert_eq!(files, vec![("dir/inner.txt".into(), Kind::File, b), ("run.sh".into(), Kind::Exec, a)]);
	306	assert_eq!(objects.verify_tree(&root).unwrap(), 4);
	307	}
	308	}

	1	//! Randomness from the operating system, for ids and tokens.
	2
	3	use std::io::Read;
	4
	5	pub fn bytes(n: usize) -> Vec<u8> {
	6	let mut buf = vec![0u8; n];
	7	std::fs::File::open("/dev/urandom")
	8	.and_then(\|mut f\| f.read_exact(&mut buf))
	9	.expect("the operating system's random source must be readable");
	10	buf
	11	}
	12
	13	/// `n` random bytes as hex.
	14	pub fn hex(n: usize) -> String {
	15	crate::hex::encode(&bytes(n))
	16	}

	1	//! The `grove` command. For now it works directly on a store on this machine; the same
	2	//! operations will go over HTTP once grove runs as a server.
	3	//!
	4	//! Who's acting: `~/.grove/config.json` names a person and, usually, the agent acting for them.
	5	//! Every command acts as that agent, except `land` and `decline`, which are decisions and act
	6	//! as the person. Locally this rests on trust; on a server, tokens will enforce it.
	7
	8	use crate::diff;
	9	use crate::json::{self, obj, Value};
	10	use crate::model::{Actor, Change, ChangeState, Event, Project, Step};
	11	use crate::objects::Id;
	12	use crate::store::{self, Content, Store};
	13	use crate::workdir::{self, FileChange};
	14	use crate::{hex, random};
	15	use std::fs;
	16	use std::path::{Path, PathBuf};
	17
	18	type Result<T> = std::result::Result<T, String>;
	19
	20	macro_rules! e {
	21	($($arg:tt)) => { Err(format!($($arg))) };
	22	}
	23
	24	// ── Arguments ──────────────────────────────────────────────
	25
	26	struct Args {
	27	positional: Vec<String>,
	28	flags: Vec<(String, Option<String>)>,
	29	}
	30
	31	impl Args {
	32	fn parse(raw: &[String], with_values: &[&str]) -> Args {
	33	let mut positional = Vec::new();
	34	let mut flags = Vec::new();
	35	let mut i = 0;
	36	while i < raw.len() {
	37	let a = &raw[i];
	38	if a.starts_with('-') && a.len() > 1 {
	39	if with_values.contains(&a.as_str()) && i + 1 < raw.len() {
	40	flags.push((a.clone(), Some(raw[i + 1].clone())));

	1	//! A fresh directory per test, removed afterwards.
	2
	3	use std::path::{Path, PathBuf};
	4	use std::sync::atomic::{AtomicU64, Ordering};
	5
	6	static NEXT: AtomicU64 = AtomicU64::new(0);
	7
	8	pub struct TestDir(PathBuf);
	9
	10	impl TestDir {
	11	pub fn new(name: &str) -> TestDir {
	12	let n = NEXT.fetch_add(1, Ordering::Relaxed);
	13	let path = std::env::temp_dir().join(format!("grove-test-{}-{name}-{n}", std::process::id()));
	14	let _ = std::fs::remove_dir_all(&path);
	15	std::fs::create_dir_all(&path).unwrap();
	16	TestDir(path)
	17	}
	18
	19	pub fn path(&self) -> &Path {
	20	&self.0
	21	}
	22	}
	23
	24	impl Drop for TestDir {
	25	fn drop(&mut self) {
	26	let _ = std::fs::remove_dir_all(&self.0);
	27	}
	28	}

	1	//! The store: objects, the log, and the state the log adds up to.
	2	//!
	3	//! A write is: check the referenced objects exist, apply the record to the in-memory state
	4	//! (which validates it and changes nothing if it's invalid), then append it to the log. If
	5	//! the append fails, the state is rebuilt from the log on disk, so memory never claims
	6	//! something the disk doesn't hold.
	7
	8	use crate::json::{self, Value};
	9	use crate::log::{Log, OpenError, Recovery};
	10	use crate::model::{id_to_json, step_to_json, Actor, ChangeState, Project, State, Step};
	11	use crate::objects::{self, Id, Objects};
	12	use std::path::{Path, PathBuf};
	13	use std::time::{SystemTime, UNIX_EPOCH};
	14
	15	#[derive(Debug)]
	16	pub enum Error {
	17	Open(OpenError),
	18	Objects(objects::Error),
	19	Io(std::io::Error),
	20	/// The log holds a record that doesn't fit the state before it.
	21	Damaged { record: u64, reason: String },
	22	/// The request isn't allowed or doesn't make sense.
	23	Refused(String),
	24	/// Landing needs the change rebuilt on the current tip first.
	25	Behind { base: u64, tip: u64 },
	26	}
	27
	28	impl std::fmt::Display for Error {
	29	fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
	30	match self {
	31	Error::Open(e) => write!(f, "{e}"),
	32	Error::Objects(e) => write!(f, "{e}"),
	33	Error::Io(e) => write!(f, "{e}"),
	34	Error::Damaged { record, reason } => write!(f, "log record {record} is damaged: {reason}"),
	35	Error::Refused(r) => write!(f, "{r}"),
	36	Error::Behind { base, tip } => write!(f, "built on point {base}, but main is at {tip}; rebuild on the tip first"),
	37	}
	38	}
	39	}
	40
	41	impl From<OpenError> for Error {
	42	fn from(e: OpenError) -> Self {
	43	Error::Open(e)
	44	}
	45	}
	46	impl From<objects::Error> for Error {
	47	fn from(e: objects::Error) -> Self {
	48	Error::Objects(e)
	49	}
	50	}
	51	impl From<std::io::Error> for Error {
	52	fn from(e: std::io::Error) -> Self {
	53	Error::Io(e)
	54	}
	55	}
	56
	57	pub type Result<T> = std::result::Result<T, Error>;
	58
	59	fn refused<T>(msg: impl Into<String>) -> Result<T> {
	60	Err(Error::Refused(msg.into()))
	61	}
	62
	63	/// What a piece of evidence holds.
	64	pub enum Content<'a> {
	65	Bytes(&'a [u8]),
	66	Url(&'a str),
	67	}
	68
	69	pub fn now_ms() -> i64 {
	70	SystemTime::now().duration_since(UNIX_EPOCH).map(\|d\| d.as_millis() as i64).unwrap_or(0)
	71	}
	72
	73	pub struct Store {
	74	dir: PathBuf,
	75	log: Log,
	76	pub objects: Objects,
	77	state: State,
	78	pub recovery: Recovery,
	79	}
	80
	81	fn replay(records: &[Vec<u8>]) -> Result<State> {
	82	let mut state = State::default();
	83	for (i, bytes) in records.iter().enumerate() {
	84	let damaged = \|reason: String\| Error::Damaged { record: i as u64, reason };
	85	let text = std::str::from_utf8(bytes).map_err(\|_\| damaged("not UTF-8".into()))?;
	86	let rec = json::parse(text).map_err(\|e\| damaged(e.to_string()))?;
	87	state.apply(&rec).map_err(damaged)?;
	88	}
	89	Ok(state)
	90	}
	91
	92	impl Store {
	93	pub fn open(dir: &Path) -> Result<Store> {
	94	std::fs::create_dir_all(dir)?;
	95	let objects = Objects::open(&dir.join("objects"))?;
	96	let (log, records, recovery) = Log::open(&dir.join("log"))?;
	97	let state = replay(&records)?;
	98	Ok(Store { dir: dir.to_path_buf(), log, objects, state, recovery })
	99	}
	100
	101	pub fn dir(&self) -> &Path {
	102	&self.dir
	103	}
	104
	105	pub fn state(&self) -> &State {
	106	&self.state
	107	}
	108
	109	/// Validate, apply, append. `rec` gets its time and actor here.
	110	fn write(&mut self, by: &Actor, mut rec: Vec<(String, Value)>) -> Result<()> {
	111	rec.insert(1, ("at".into(), now_ms().into()));
	112	rec.insert(2, ("by".into(), by.to_json()));
	113	let rec = Value::Object(rec);
	114	self.state.apply(&rec).map_err(Error::Refused)?;
	115	if let Err(e) = self.log.append(rec.encode().as_bytes()) {
	116	// The state now holds a record the log doesn't. Rebuild it from the disk.
	117	let (log, records, _) = Log::open(&self.dir.join("log")).map_err(\|_\| Error::Io(e))?;
	118	self.log = log;
	119	self.state = replay(&records)?;
	120	return Err(Error::Refused("couldn't write to the log".into()));
	121	}
	122	Ok(())
	123	}
	124
	125	fn rec(t: &str, fields: Vec<(&str, Value)>) -> Vec<(String, Value)> {
	126	let mut out = vec![("t".to_string(), Value::from(t))];
	127	out.extend(fields.into_iter().map(\|(k, v)\| (k.to_string(), v)));
	128	out
	129	}
	130
	131	fn require_tree(&self, id: &Id) -> Result<()> {
	132	if !self.objects.has(id) {
	133	return refused(format!("snapshot {} isn't stored", crate::hex::encode(id)));
	134	}
	135	Ok(())
	136	}
	137
	138	pub fn project(&self, owner: &str, name: &str) -> Result<&Project> {
	139	self.state.project(owner, name).ok_or_else(\|\| Error::Refused(format!("no project {owner}/{name}")))
	140	}
	141
	142	// ── People and agents ──────────────────────────────────
	143
	144	pub fn add_person(&mut self, username: &str, display_name: &str) -> Result<()> {
	145	if !username.chars().all(\|c\| c.is_ascii_alphanumeric() \|\| c == '-' \|\| c == '_') \|\| username.is_empty() {
	146	return refused("usernames are letters, digits, - and _");
	147	}
	148	self.write(&Actor::person(username), Self::rec("person", vec![("username", username.into()), ("display_name", display_name.into())]))
	149	}
	150
	151	/// Register an agent acting for `person`. Returns its id.
	152	pub fn add_agent(&mut self, person: &str, name: &str) -> Result<String> {
	153	let id = crate::random::hex(6);
	154	self.write(&Actor::person(person), Self::rec("agent", vec![("id", id.clone().into()), ("name", name.into())]))?;
	155	Ok(id)
	156	}
	157
	158	pub fn revoke_agent(&mut self, person: &str, id: &str) -> Result<()> {
	159	self.write(&Actor::person(person), Self::rec("agent.revoke", vec![("id", id.into())]))
	160	}
	161
	162	// ── Projects and main ──────────────────────────────────
	163
	164	pub fn create_project(&mut self, by: &Actor, name: &str, public: bool) -> Result<()> {
	165	if !objects::valid_name(name) \|\| name.contains(char::is_whitespace) {
	166	return refused(format!("{name:?} can't be a project name"));
	167	}
	168	self.write(by, Self::rec("project", vec![("name", name.into()), ("public", public.into())]))
	169	}
	170
	171	/// Record a point directly on main: a project's first snapshot, or imported history.
	172	pub fn record_point(&mut self, by: &Actor, owner: &str, name: &str, snapshot: &Id, message: &str, author: Option<&Actor>) -> Result<u64> {
	173	self.require_tree(snapshot)?;
	174	let n = self.project(owner, name)?.tip() + 1;
	175	let mut fields = vec![
	176	("owner", owner.into()),
	177	("project", name.into()),
	178	("n", n.into()),
	179	("snapshot", id_to_json(snapshot)),
	180	("message", message.into()),
	181	];
	182	if let Some(a) = author {
	183	fields.push(("author", a.to_json()));
	184	}
	185	self.write(by, Self::rec("point", fields))?;
	186	Ok(n)
	187	}
	188
	189	// ── Changes ────────────────────────────────────────────
	190
	191	/// Open a change as an idea: an intent, no code yet. Its first revision makes it a draft.
	192	pub fn open_change(&mut self, by: &Actor, owner: &str, name: &str, intent: &str) -> Result<u64> {
	193	let p = self.project(owner, name)?;
	194	if !p.public && by.person != p.owner {
	195	return refused(format!("{owner}/{name} is private"));
	196	}
	197	let number = p.changes.len() as u64 + 1;
	198	self.write(by, Self::rec("change", vec![("owner", owner.into()), ("project", name.into()), ("number", number.into()), ("intent", intent.into()), ("state", "idea".into())]))?;
	199	Ok(number)
	200	}
	201
	202	/// Add a revision: the whole stack of steps, built on main point `base`.
	203	pub fn revise(&mut self, by: &Actor, owner: &str, name: &str, change: u64, base: u64, steps: &[Step], note: Option<&str>) -> Result<u32> {
	204	for s in steps {
	205	self.require_tree(&s.snapshot)?;
	206	}
	207	let p = self.project(owner, name)?;
	208	let c = p.change(change).ok_or_else(\|\| Error::Refused(format!("no change #{change}")))?;
	209	if by.person != c.proposer.person && by.person != p.owner {
	210	return refused("only the proposer or the owner can revise a change");
	211	}
	212	let number = c.revisions.len() as u32 + 1;
	213	let steps: Vec<Value> = steps.iter().map(step_to_json).collect();
	214	self.write(
	215	by,
	216	Self::rec("revision", vec![
	217	("owner", owner.into()),
	218	("project", name.into()),
	219	("change", change.into()),
	220	("number", (number as u64).into()),
	221	("base", base.into()),
	222	("steps", steps.into()),
	223	("note", note.map(str::to_string).into()),
	224	]),
	225	)?;
	226	Ok(number)
	227	}
	228
	229	pub fn set_state(&mut self, by: &Actor, owner: &str, name: &str, change: u64, to: ChangeState) -> Result<()> {
	230	if to == ChangeState::Landed {
	231	return refused("changes land through land()");
	232	}
	233	self.write(by, Self::rec("state", vec![("owner", owner.into()), ("project", name.into()), ("change", change.into()), ("state", to.name().into())]))
	234	}
	235
	236	/// Land a change's latest revision. Its steps become the next points on main, all at once.
	237	pub fn land(&mut self, by: &Actor, owner: &str, name: &str, change: u64) -> Result<(u64, u64)> {
	238	let p = self.project(owner, name)?;
	239	if by.person != p.owner \|\| by.agent.is_some() {
	240	return refused("only the owner, in person, lands changes");
	241	}
	242	let c = p.change(change).ok_or_else(\|\| Error::Refused(format!("no change #{change}")))?;
	243	let rev = c.latest().ok_or_else(\|\| Error::Refused(format!("#{change} has no revisions")))?;
	244	if rev.base != p.tip() {
	245	return Err(Error::Behind { base: rev.base, tip: p.tip() });
	246	}
	247	let first = p.tip() + 1;
	248	let count = rev.steps.len() as u64;
	249	let snapshots: Vec<Value> = rev.steps.iter().map(\|s\| id_to_json(&s.snapshot)).collect();
	250	let revision = rev.number as u64;
	251	self.write(
	252	by,
	253	Self::rec("land", vec![
	254	("owner", owner.into()),
	255	("project", name.into()),
	256	("change", change.into()),
	257	("revision", revision.into()),
	258	("first", first.into()),
	259	("snapshots", snapshots.into()),
	260	]),
	261	)?;
	262	Ok((first, first + count - 1))
	263	}
	264
	265	pub fn say(&mut self, by: &Actor, owner: &str, name: &str, change: u64, body: &str) -> Result<()> {
	266	self.write(by, Self::rec("say", vec![("owner", owner.into()), ("project", name.into()), ("change", change.into()), ("body", body.into())]))
	267	}
	268
	269	/// Attach evidence to a revision: stored bytes of a media type, or a URL.
	270	pub fn attach(&mut self, by: &Actor, owner: &str, name: &str, change: u64, revision: u32, title: &str, media_type: &str, content: Content) -> Result<()> {
	271	let (blob, url) = match content {
	272	Content::Bytes(bytes) => (Value::from(crate::hex::encode(&self.objects.put_blob(bytes)?)), Value::Null),
	273	Content::Url(url) => (Value::Null, Value::from(url)),
	274	};
	275	self.write(
	276	by,
	277	Self::rec("evidence", vec![
	278	("owner", owner.into()),
	279	("project", name.into()),
	280	("change", change.into()),
	281	("revision", (revision as u64).into()),
	282	("title", title.into()),
	283	("type", media_type.into()),
	284	("blob", blob),
	285	("url", url),
	286	]),
	287	)
	288	}
	289
	290	/// Check that every object the state refers to exists and matches its id.
	291	pub fn verify(&self) -> Result<usize> {
	292	let mut count = 0;
	293	let mut seen = std::collections::HashSet::new();
	294	let mut check_tree = \|id: &Id, count: &mut usize\| -> Result<()> {
	295	if seen.insert(*id) {
	296	*count += self.objects.verify_tree(id)?;
	297	}
	298	Ok(())
	299	};
	300	for p in &self.state.projects {
	301	for pt in &p.points {
	302	check_tree(&pt.snapshot, &mut count)?;
	303	}
	304	for c in &p.changes {
	305	for r in &c.revisions {
	306	for s in &r.steps {
	307	check_tree(&s.snapshot, &mut count)?;
	308	}
	309	}
	310	for e in &c.evidence {
	311	if let Some(b) = &e.blob {
	312	self.objects.get_blob(b)?;
	313	count += 1;
	314	}
	315	}
	316	}
	317	}
	318	Ok(count)
	319	}
	320	}
	321
	322	#[cfg(test)]
	323	mod tests {
	324	use super::*;
	325	use crate::objects::{Entry, Kind};
	326	use crate::testdir::TestDir;
	327
	328	fn tree(store: &Store, files: &[(&str, &str)]) -> Id {
	329	let entries: Vec<Entry> = files
	330	.iter()
	331	.map(\|(n, c)\| Entry { name: n.to_string(), kind: Kind::File, id: store.objects.put_blob(c.as_bytes()).unwrap() })
	332	.collect();
	333	store.objects.put_tree(&entries).unwrap()
	334	}
	335
	336	fn setup(dir: &TestDir) -> (Store, Actor, String) {
	337	let mut store = Store::open(dir.path()).unwrap();
	338	store.add_person("anton", "Anton").unwrap();
	339	let agent = store.add_agent("anton", "claude").unwrap();
	340	let me = Actor::agent("anton", &agent);
	341	store.create_project(&Actor::person("anton"), "grove", true).unwrap();
	342	let t = tree(&store, &[("README", "hello")]);
	343	store.record_point(&Actor::person("anton"), "anton", "grove", &t, "Start", None).unwrap();
	344	(store, me, agent)
	345	}
	346
	347	#[test]
	348	fn change_lifecycle_and_replay() {
	349	let dir = TestDir::new("store-lifecycle");
	350	{
	351	let (mut store, me, _) = setup(&dir);
	352	let owner = Actor::person("anton");
	353	let c = store.open_change(&me, "anton", "grove", "Say more in the README").unwrap();
	354	assert_eq!(store.project("anton", "grove").unwrap().change(c).unwrap().state, ChangeState::Idea);
	355
	356	let t1 = tree(&store, &[("README", "hello there")]);
	357	let t2 = tree(&store, &[("README", "hello there"), ("NOTES", "x")]);
	358	let steps = vec![
	359	Step { id: "s1".into(), snapshot: t1, message: "Longer greeting".into() },
	360	Step { id: "s2".into(), snapshot: t2, message: "Add notes".into() },
	361	];
	362	assert_eq!(store.revise(&me, "anton", "grove", c, 1, &steps, None).unwrap(), 1);
	363	store.set_state(&me, "anton", "grove", c, ChangeState::Proposed).unwrap();
	364	store.say(&owner, "anton", "grove", c, "Looks good").unwrap();
	365	store.attach(&me, "anton", "grove", c, 1, "notes", "text/plain", Content::Bytes(b"checked by hand")).unwrap();
	366
	367	// Only the owner lands.
	368	assert!(matches!(store.land(&me, "anton", "grove", c), Err(Error::Refused(_))));
	369	assert_eq!(store.land(&owner, "anton", "grove", c).unwrap(), (2, 3));
	370	let p = store.project("anton", "grove").unwrap();
	371	assert_eq!(p.tip(), 3);
	372	assert_eq!(p.point(3).unwrap().snapshot, t2);
	373	assert_eq!(p.change(c).unwrap().state, ChangeState::Landed);
	374	assert_eq!(store.verify().unwrap() > 0, true);
	375	}
	376	// Everything comes back from the log.
	377	let store = Store::open(dir.path()).unwrap();
	378	let p = store.project("anton", "grove").unwrap();
	379	assert_eq!(p.tip(), 3);
	380	let c = p.change(1).unwrap();
	381	assert_eq!(c.state, ChangeState::Landed);
	382	assert_eq!(c.messages[0].body, "Looks good");
	383	assert_eq!(c.evidence.len(), 1);
	384	assert_eq!(p.point(2).unwrap().message, "Longer greeting");
	385	assert!(p.point(2).unwrap().author.agent.is_some());
	386	}
	387
	388	#[test]
	389	fn landing_behind_the_tip_is_refused() {
	390	let dir = TestDir::new("store-behind");
	391	let (mut store, me, _) = setup(&dir);
	392	let owner = Actor::person("anton");
	393	let t = tree(&store, &[("README", "a")]);
	394	let step = \|id: &str\| vec![Step { id: id.into(), snapshot: t, message: "m".into() }];
	395	let a = store.open_change(&me, "anton", "grove", "a").unwrap();
	396	let b = store.open_change(&me, "anton", "grove", "b").unwrap();
	397	store.revise(&me, "anton", "grove", a, 1, &step("x"), None).unwrap();
	398	store.revise(&me, "anton", "grove", b, 1, &step("y"), None).unwrap();
	399	store.set_state(&me, "anton", "grove", a, ChangeState::Proposed).unwrap();
	400	store.set_state(&me, "anton", "grove", b, ChangeState::Proposed).unwrap();
	401	store.land(&owner, "anton", "grove", a).unwrap();
	402	assert!(matches!(store.land(&owner, "anton", "grove", b), Err(Error::Behind { base: 1, tip: 2 })));
	403	// Nothing changed for b.
	404	assert_eq!(store.project("anton", "grove").unwrap().change(b).unwrap().state, ChangeState::Proposed);
	405	}
	406
	407	#[test]
	408	fn refusals_leave_no_trace() {
	409	let dir = TestDir::new("store-refusals");
	410	let (mut store, me, agent) = setup(&dir);
	411	let before = store.state().records;
	412	// Unstored snapshot, unknown project, bad transition, revoked agent.
	413	let ghost = crate::objects::blob_id(b"not stored");
	414	assert!(store.record_point(&Actor::person("anton"), "anton", "grove", &ghost, "x", None).is_err());
	415	assert!(store.open_change(&me, "anton", "nope", "x").is_err());
	416	let c = store.open_change(&me, "anton", "grove", "idea").unwrap();
	417	assert!(store.set_state(&me, "anton", "grove", c, ChangeState::Proposed).is_err(), "an idea can't be proposed");
	418	store.revoke_agent("anton", &agent).unwrap();
	419	assert!(store.say(&me, "anton", "grove", c, "still here?").is_err());
	420	assert_eq!(store.state().records, before + 2, "only the change and the revocation were written");
	421	drop(store);
	422	assert_eq!(Store::open(dir.path()).unwrap().state().records, before + 2);
	423	}
	424
	425	#[test]
	426	fn strangers_and_private_projects() {
	427	let dir = TestDir::new("store-strangers");
	428	let (mut store, _, _) = setup(&dir);
	429	store.add_person("visitor", "Visitor").unwrap();
	430	let visitor = Actor::person("visitor");
	431	// Anyone can propose to a public project, but not land.
	432	let c = store.open_change(&visitor, "anton", "grove", "An idea from outside").unwrap();
	433	let t = tree(&store, &[("README", "outside")]);
	434	store.revise(&visitor, "anton", "grove", c, 1, &[Step { id: "s".into(), snapshot: t, message: "m".into() }], None).unwrap();
	435	store.set_state(&visitor, "anton", "grove", c, ChangeState::Proposed).unwrap();
	436	assert!(store.land(&visitor, "anton", "grove", c).is_err());
	437	store.set_state(&Actor::person("anton"), "anton", "grove", c, ChangeState::Declined).unwrap();
	438	// Private projects take changes only from the owner.
	439	store.create_project(&Actor::person("anton"), "secret", false).unwrap();
	440	assert!(store.open_change(&visitor, "anton", "secret", "let me in").is_err());
	441	}
	442	}

	1	//! Working directories: turning a folder into a snapshot and a snapshot back into a folder.
	2
	3	use crate::ignore::Ignore;
	4	use crate::objects::{self, Entry, Id, Kind, Objects};
	5	use std::collections::BTreeMap;
	6	use std::fs;
	7	use std::io;
	8	use std::os::unix::fs::PermissionsExt;
	9	use std::path::Path;
	10
	11	#[derive(Debug)]
	12	pub enum Error {
	13	Io(io::Error),
	14	Objects(objects::Error),
	15	/// A file name grove can't store (not UTF-8, or reserved).
	16	BadName(String),
	17	}
	18
	19	impl std::fmt::Display for Error {
	20	fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
	21	match self {
	22	Error::Io(e) => write!(f, "{e}"),
	23	Error::Objects(e) => write!(f, "{e}"),
	24	Error::BadName(n) => write!(f, "can't store a file named {n}"),
	25	}
	26	}
	27	}
	28
	29	impl From<io::Error> for Error {
	30	fn from(e: io::Error) -> Self {
	31	Error::Io(e)
	32	}
	33	}
	34	impl From<objects::Error> for Error {
	35	fn from(e: objects::Error) -> Self {
	36	Error::Objects(e)
	37	}
	38	}
	39
	40	pub type Result<T> = std::result::Result<T, Error>;
	41
	42	pub fn load_ignore(root: &Path) -> Ignore {
	43	Ignore::parse(&fs::read_to_string(root.join(".groveignore")).unwrap_or_default())
	44	}
	45
	46	/// Store everything in `root` that isn't ignored; return the snapshot (root tree) id.
	47	pub fn snapshot(objects: &Objects, root: &Path) -> Result<Id> {
	48	let ignore = load_ignore(root);
	49	Ok(snapshot_dir(objects, root, "", &ignore)?.unwrap_or(objects.empty_tree()?))
	50	}
	51
	52	/// None for a directory with nothing to store (empty directories aren't kept).
	53	fn snapshot_dir(objects: &Objects, dir: &Path, rel: &str, ignore: &Ignore) -> Result<Option<Id>> {
	54	let mut entries = Vec::new();
	55	let mut names: Vec<_> = fs::read_dir(dir)?.collect::<io::Result<Vec<_>>>()?;
	56	names.sort_by_key(\|e\| e.file_name());
	57	for e in names {
	58	let name = e.file_name().into_string().map_err(\|n\| Error::BadName(format!("{n:?}")))?;
	59	if !objects::valid_name(&name) {
	60	return Err(Error::BadName(name));
	61	}
	62	let path = if rel.is_empty() { name.clone() } else { format!("{rel}/{name}") };
	63	let meta = fs::symlink_metadata(e.path())?;
	64	let ft = meta.file_type();
	65	if ignore.ignored(&path, ft.is_dir()) {
	66	continue;
	67	}
	68	if ft.is_dir() {
	69	if let Some(id) = snapshot_dir(objects, &e.path(), &path, ignore)? {
	70	entries.push(Entry { name, kind: Kind::Dir, id });
	71	}
	72	} else if ft.is_symlink() {
	73	let target = fs::read_link(e.path())?;
	74	let target = target.to_str().ok_or_else(\|\| Error::BadName(format!("link target of {path}")))?;
	75	entries.push(Entry { name, kind: Kind::Link, id: objects.put_blob(target.as_bytes())? });
	76	} else if ft.is_file() {
	77	let kind = if meta.permissions().mode() & 0o111 != 0 { Kind::Exec } else { Kind::File };
	78	entries.push(Entry { name, kind, id: objects.put_blob(&fs::read(e.path())?)? });
	79	}
	80	// Sockets, devices and pipes aren't project content.
	81	}
	82	if entries.is_empty() {
	83	return Ok(None);
	84	}
	85	Ok(Some(objects.put_tree(&entries)?))
	86	}
	87
	88	/// Write a snapshot into `root`. Files that were in `previous` but aren't in `tree` are
	89	/// removed; anything grove never tracked is left alone.
	90	pub fn materialize(objects: &Objects, tree: &Id, root: &Path, previous: Option<&Id>) -> Result<()> {
	91	let want: BTreeMap<String, (Kind, Id)> = objects.flatten(tree)?.into_iter().map(\|(p, k, i)\| (p, (k, i))).collect();
	92	if let Some(prev) = previous {
	93	for (path, _, _) in objects.flatten(prev)? {
	94	if !want.contains_key(&path) {
	95	let full = root.join(&path);
	96	match fs::remove_file(&full) {
	97	Ok(()) => {}
	98	Err(e) if e.kind() == io::ErrorKind::NotFound => {}
	99	Err(e) => return Err(e.into()),
	100	}
	101	remove_empty_parents(root, &full);
	102	}
	103	}
	104	}
	105	for (path, (kind, id)) in &want {
	106	let full = root.join(path);
	107	if let Some(dir) = full.parent() {
	108	fs::create_dir_all(dir)?;
	109	}
	110	let content = objects.get_blob(id)?;
	111	// Replace whatever is there (a file, or a link of a different kind).
	112	if fs::symlink_metadata(&full).is_ok() {
	113	fs::remove_file(&full)?;
	114	}
	115	match kind {
	116	Kind::Link => {
	117	let target = String::from_utf8(content).map_err(\|_\| Error::BadName(path.clone()))?;
	118	std::os::unix::fs::symlink(target, &full)?;
	119	}
	120	Kind::File \| Kind::Exec => {
	121	fs::write(&full, &content)?;
	122	let mode = if *kind == Kind::Exec { 0o755 } else { 0o644 };
	123	fs::set_permissions(&full, fs::Permissions::from_mode(mode))?;
	124	}
	125	Kind::Dir => unreachable!("flatten yields files only"),
	126	}
	127	}
	128	Ok(())
	129	}
	130
	131	fn remove_empty_parents(root: &Path, file: &Path) {
	132	let mut dir = file.parent();
	133	while let Some(d) = dir {
	134	if d == root \|\| fs::remove_dir(d).is_err() {
	135	break;
	136	}
	137	dir = d.parent();
	138	}
	139	}
	140
	141	#[derive(Debug, PartialEq, Eq, Clone, Copy)]
	142	pub enum FileChange {
	143	Added,
	144	Removed,
	145	Modified,
	146	}
	147
	148	/// Paths that differ between two snapshots, sorted. A kind change (file to exec) counts as modified.
	149	pub fn compare(objects: &Objects, a: &Id, b: &Id) -> Result<Vec<(String, FileChange)>> {
	150	if a == b {
	151	return Ok(Vec::new());
	152	}
	153	let a: BTreeMap<_, _> = objects.flatten(a)?.into_iter().map(\|(p, k, i)\| (p, (k, i))).collect();
	154	let b: BTreeMap<_, _> = objects.flatten(b)?.into_iter().map(\|(p, k, i)\| (p, (k, i))).collect();
	155	let mut out = Vec::new();
	156	for (path, old) in &a {
	157	match b.get(path) {
	158	None => out.push((path.clone(), FileChange::Removed)),
	159	Some(new) if new != old => out.push((path.clone(), FileChange::Modified)),
	160	_ => {}
	161	}
	162	}
	163	for path in b.keys() {
	164	if !a.contains_key(path) {
	165	out.push((path.clone(), FileChange::Added));
	166	}
	167	}
	168	out.sort_by(\|x, y\| x.0.cmp(&y.0));
	169	Ok(out)
	170	}
	171
	172	#[cfg(test)]
	173	mod tests {
	174	use super::*;
	175	use crate::testdir::TestDir;
	176
	177	#[test]
	178	fn snapshot_and_materialize_round_trip() {
	179	let src = TestDir::new("wd-src");
	180	let store = TestDir::new("wd-store");
	181	let objects = Objects::open(store.path()).unwrap();
	182	let root = src.path();
	183	fs::create_dir_all(root.join("src/deep")).unwrap();
	184	fs::create_dir_all(root.join("target")).unwrap();
	185	fs::create_dir_all(root.join("empty")).unwrap();
	186	fs::write(root.join("README"), "hi").unwrap();
	187	fs::write(root.join("src/deep/a.rs"), "fn a() {}").unwrap();
	188	fs::write(root.join("run.sh"), "#!/bin/sh").unwrap();
	189	fs::set_permissions(root.join("run.sh"), fs::Permissions::from_mode(0o755)).unwrap();
	190	std::os::unix::fs::symlink("README", root.join("link")).unwrap();
	191	fs::write(root.join("target/out.bin"), "built").unwrap();
	192	fs::write(root.join(".groveignore"), "target/\n").unwrap();
	193
	194	let snap = snapshot(&objects, root).unwrap();
	195	let files: Vec<String> = objects.flatten(&snap).unwrap().into_iter().map(\|(p, _, _)\| p).collect();
	196	assert_eq!(files, [".groveignore", "README", "link", "run.sh", "src/deep/a.rs"]);
	197
	198	let dst = TestDir::new("wd-dst");
	199	materialize(&objects, &snap, dst.path(), None).unwrap();
	200	assert_eq!(snapshot(&objects, dst.path()).unwrap(), snap, "identical round trip");
	201	assert_eq!(fs::read_link(dst.path().join("link")).unwrap().to_str(), Some("README"));
	202	assert!(fs::metadata(dst.path().join("run.sh")).unwrap().permissions().mode() & 0o111 != 0);
	203	}
	204
	205	#[test]
	206	fn moving_between_snapshots() {
	207	let store = TestDir::new("wd-move-store");
	208	let objects = Objects::open(store.path()).unwrap();
	209	let wd = TestDir::new("wd-move");
	210	let root = wd.path();
	211	fs::create_dir_all(root.join("a/b")).unwrap();
	212	fs::write(root.join("a/b/old.txt"), "old").unwrap();
	213	fs::write(root.join("keep.txt"), "1").unwrap();
	214	let first = snapshot(&objects, root).unwrap();
	215
	216	fs::remove_dir_all(root.join("a")).unwrap();
	217	fs::write(root.join("keep.txt"), "2").unwrap();
	218	fs::write(root.join("new.txt"), "new").unwrap();
	219	let second = snapshot(&objects, root).unwrap();
	220	assert_eq!(
	221	compare(&objects, &first, &second).unwrap(),
	222	vec![("a/b/old.txt".into(), FileChange::Removed), ("keep.txt".into(), FileChange::Modified), ("new.txt".into(), FileChange::Added)]
	223	);
	224
	225	// Going back restores the old files and removes the new one, leaving untracked files.
	226	fs::write(root.join("untracked.tmp"), "mine").unwrap();
	227	materialize(&objects, &first, root, Some(&second)).unwrap();
	228	assert!(root.join("a/b/old.txt").exists());
	229	assert!(!root.join("new.txt").exists());
	230	assert_eq!(fs::read_to_string(root.join("keep.txt")).unwrap(), "1");
	231	assert!(root.join("untracked.tmp").exists());
	232	}
	233	}