# NetBird Reconciler Implementation Plan > **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. **Goal:** Build a Deno-based HTTP service that reconciles NetBird VPN configuration from a declarative `netbird.json` state file, with event-driven peer enrollment detection and Gitea Actions CI integration. **Architecture:** Three-layer design. A NetBird API client wraps all management API calls. A reconciliation engine diffs desired vs actual state and produces an ordered operation plan. An HTTP server exposes `/reconcile`, `/sync-events`, and `/health` endpoints. A background event poller detects peer enrollments and commits state updates via Gitea API. **Tech Stack:** Deno 2.x, Zod (schema validation), Deno standard library (HTTP server), Docker --- ## Task 0: Scaffold project structure **Files:** - Create: `deno.json` - Create: `src/main.ts` - Create: `src/config.ts` - Create: `.gitignore` - Create: `Dockerfile` **Step 1: Create `deno.json`** ```json { "name": "@blastpilot/netbird-reconciler", "version": "0.1.0", "tasks": { "dev": "deno run --allow-net --allow-read --allow-write --allow-env --watch src/main.ts", "start": "deno run --allow-net --allow-read --allow-write --allow-env src/main.ts", "test": "deno test --allow-net --allow-read --allow-write --allow-env", "check": "deno check src/main.ts", "lint": "deno lint", "fmt": "deno fmt" }, "imports": { "zod": "npm:zod@^3.23.0" }, "compilerOptions": { "strict": true } } ``` **Step 2: Create `src/config.ts`** ```typescript import { z } from "zod"; const ConfigSchema = z.object({ netbirdApiUrl: z.string().url(), netbirdApiToken: z.string().min(1), giteaUrl: z.string().url(), giteaToken: z.string().min(1), giteaRepo: z.string().regex(/^[^/]+\/[^/]+$/), // owner/repo reconcilerToken: z.string().min(1), pollIntervalSeconds: z.coerce.number().int().positive().default(30), port: z.coerce.number().int().positive().default(8080), dataDir: z.string().default("/data"), }); export type Config = z.infer; export function loadConfig(): Config { return ConfigSchema.parse({ netbirdApiUrl: Deno.env.get("NETBIRD_API_URL"), netbirdApiToken: Deno.env.get("NETBIRD_API_TOKEN"), giteaUrl: Deno.env.get("GITEA_URL"), giteaToken: Deno.env.get("GITEA_TOKEN"), giteaRepo: Deno.env.get("GITEA_REPO"), reconcilerToken: Deno.env.get("RECONCILER_TOKEN"), pollIntervalSeconds: Deno.env.get("POLL_INTERVAL_SECONDS"), port: Deno.env.get("PORT"), dataDir: Deno.env.get("DATA_DIR"), }); } ``` **Step 3: Create `src/main.ts`** (minimal placeholder) ```typescript import { loadConfig } from "./config.ts"; const config = loadConfig(); console.log(JSON.stringify({ msg: "starting", port: config.port })); ``` **Step 4: Create `.gitignore`** ``` /data/ *.log ``` **Step 5: Create `Dockerfile`** ```dockerfile FROM denoland/deno:2.2.2 AS builder WORKDIR /app COPY deno.json . COPY src/ src/ RUN deno compile --allow-net --allow-read --allow-write --allow-env --output reconciler src/main.ts FROM gcr.io/distroless/cc-debian12 COPY --from=builder /app/reconciler /usr/local/bin/reconciler ENTRYPOINT ["reconciler"] ``` **Step 6: Verify project compiles** Run: `deno check src/main.ts` Expected: no errors **Step 7: Commit** ``` feat: scaffold netbird-reconciler project ``` --- ## Task 1: NetBird API client — types and base client **Files:** - Create: `src/netbird/types.ts` - Create: `src/netbird/client.ts` - Create: `src/netbird/client.test.ts` **Step 1: Define NetBird API response types in `src/netbird/types.ts`** These types model the NetBird Management API responses. Only the fields we need for reconciliation are included. ```typescript /** Group as returned by GET /api/groups */ export interface NbGroup { id: string; name: string; peers_count: number; peers: Array<{ id: string; name: string }>; issued: "api" | "jwt" | "integration"; } /** Setup key as returned by GET /api/setup-keys */ export interface NbSetupKey { id: number; name: string; type: "one-off" | "reusable"; key: string; expires: string; valid: boolean; revoked: boolean; used_times: number; state: "valid" | "expired" | "revoked" | "overused"; auto_groups: string[]; usage_limit: number; } /** Peer as returned by GET /api/peers */ export interface NbPeer { id: string; name: string; ip: string; connected: boolean; hostname: string; os: string; version: string; groups: Array<{ id: string; name: string }>; last_seen: string; dns_label: string; login_expiration_enabled: boolean; ssh_enabled: boolean; inactivity_expiration_enabled: boolean; } /** Policy as returned by GET /api/policies */ export interface NbPolicy { id: string; name: string; description: string; enabled: boolean; rules: NbPolicyRule[]; } export interface NbPolicyRule { id?: string; name: string; description: string; enabled: boolean; action: "accept" | "drop"; bidirectional: boolean; protocol: "tcp" | "udp" | "icmp" | "all"; ports?: string[]; sources: Array; destinations: Array; } /** Route as returned by GET /api/routes */ export interface NbRoute { id: string; description: string; network_id: string; enabled: boolean; peer?: string; peer_groups?: string[]; network?: string; domains?: string[]; metric: number; masquerade: boolean; groups: string[]; keep_route: boolean; } /** DNS nameserver group as returned by GET /api/dns/nameservers */ export interface NbDnsNameserverGroup { id: string; name: string; description: string; nameservers: Array<{ ip: string; ns_type: string; port: number; }>; enabled: boolean; groups: string[]; primary: boolean; domains: string[]; search_domains_enabled: boolean; } /** Audit event as returned by GET /api/events/audit */ export interface NbEvent { id: number; timestamp: string; activity: string; activity_code: string; initiator_id: string; initiator_name: string; target_id: string; meta: Record; } ``` **Step 2: Write test for base HTTP client in `src/netbird/client.test.ts`** Test the client can be constructed and makes authenticated requests. Use a mock fetch pattern: inject a fetch function so tests don't hit a real API. ```typescript import { assertEquals } from "jsr:@std/assert"; import { NetbirdClient } from "./client.ts"; function mockFetch( responses: Map, ): typeof fetch { return (input: string | URL | Request, init?: RequestInit) => { const url = typeof input === "string" ? input : input instanceof URL ? input.toString() : input.url; const method = init?.method ?? "GET"; const key = `${method} ${url}`; const resp = responses.get(key); if (!resp) throw new Error(`Unmocked request: ${key}`); return Promise.resolve( new Response(JSON.stringify(resp.body), { status: resp.status, headers: { "Content-Type": "application/json" }, }), ); }; } Deno.test("NetbirdClient.listGroups sends auth header and parses response", async () => { const groups = [{ id: "g1", name: "pilots", peers_count: 1, peers: [], issued: "api" }]; const client = new NetbirdClient( "https://nb.example.com/api", "test-token", mockFetch(new Map([ ["GET https://nb.example.com/api/groups", { status: 200, body: groups }], ])), ); const result = await client.listGroups(); assertEquals(result.length, 1); assertEquals(result[0].name, "pilots"); }); Deno.test("NetbirdClient throws on non-2xx response", async () => { const client = new NetbirdClient( "https://nb.example.com/api", "test-token", mockFetch(new Map([ ["GET https://nb.example.com/api/groups", { status: 401, body: { message: "unauthorized" } }], ])), ); try { await client.listGroups(); throw new Error("Should have thrown"); } catch (e) { assertEquals((e as Error).message.includes("401"), true); } }); ``` **Step 3: Run tests to verify they fail** Run: `deno test src/netbird/client.test.ts` Expected: FAIL — `NetbirdClient` not found **Step 4: Implement base client in `src/netbird/client.ts`** ```typescript import type { NbDnsNameserverGroup, NbEvent, NbGroup, NbPeer, NbPolicy, NbRoute, NbSetupKey, } from "./types.ts"; export class NetbirdApiError extends Error { constructor( public readonly status: number, public readonly method: string, public readonly path: string, public readonly body: unknown, ) { super(`NetBird API ${method} ${path} returned ${status}`); this.name = "NetbirdApiError"; } } type FetchFn = typeof fetch; export class NetbirdClient { constructor( private readonly baseUrl: string, private readonly token: string, private readonly fetchFn: FetchFn = fetch, ) {} private async request(method: string, path: string, body?: unknown): Promise { const url = `${this.baseUrl}${path}`; const headers: Record = { "Authorization": `Token ${this.token}`, "Accept": "application/json", }; if (body !== undefined) { headers["Content-Type"] = "application/json"; } const resp = await this.fetchFn(url, { method, headers, body: body !== undefined ? JSON.stringify(body) : undefined, }); if (!resp.ok) { const text = await resp.text().catch(() => ""); throw new NetbirdApiError(resp.status, method, path, text); } if (resp.status === 204) return undefined as T; return resp.json() as Promise; } // --- Groups --- listGroups(): Promise { return this.request("GET", "/groups"); } createGroup(name: string, peers: string[] = []): Promise { return this.request("POST", "/groups", { name, peers }); } updateGroup(id: string, name: string, peers: string[] = []): Promise { return this.request("PUT", `/groups/${id}`, { name, peers }); } deleteGroup(id: string): Promise { return this.request("DELETE", `/groups/${id}`); } // --- Setup Keys --- listSetupKeys(): Promise { return this.request("GET", "/setup-keys"); } createSetupKey(params: { name: string; type: "one-off" | "reusable"; expires_in: number; auto_groups: string[]; usage_limit: number; }): Promise { return this.request("POST", "/setup-keys", params); } deleteSetupKey(id: number): Promise { return this.request("DELETE", `/setup-keys/${id}`); } // --- Peers --- listPeers(): Promise { return this.request("GET", "/peers"); } updatePeer(id: string, params: { name: string; ssh_enabled: boolean; login_expiration_enabled: boolean; inactivity_expiration_enabled: boolean; }): Promise { return this.request("PUT", `/peers/${id}`, params); } deletePeer(id: string): Promise { return this.request("DELETE", `/peers/${id}`); } // --- Policies --- listPolicies(): Promise { return this.request("GET", "/policies"); } createPolicy(params: { name: string; description: string; enabled: boolean; rules: Array<{ name: string; description: string; enabled: boolean; action: "accept" | "drop"; bidirectional: boolean; protocol: string; ports?: string[]; sources: string[]; destinations: string[]; }>; }): Promise { return this.request("POST", "/policies", params); } updatePolicy(id: string, params: { name: string; description: string; enabled: boolean; rules: Array<{ name: string; description: string; enabled: boolean; action: "accept" | "drop"; bidirectional: boolean; protocol: string; ports?: string[]; sources: string[]; destinations: string[]; }>; }): Promise { return this.request("PUT", `/policies/${id}`, params); } deletePolicy(id: string): Promise { return this.request("DELETE", `/policies/${id}`); } // --- Routes --- listRoutes(): Promise { return this.request("GET", "/routes"); } createRoute(params: { description: string; network_id: string; enabled: boolean; peer_groups?: string[]; network?: string; domains?: string[]; metric: number; masquerade: boolean; groups: string[]; keep_route: boolean; }): Promise { return this.request("POST", "/routes", params); } updateRoute(id: string, params: { description: string; network_id: string; enabled: boolean; peer_groups?: string[]; network?: string; domains?: string[]; metric: number; masquerade: boolean; groups: string[]; keep_route: boolean; }): Promise { return this.request("PUT", `/routes/${id}`, params); } deleteRoute(id: string): Promise { return this.request("DELETE", `/routes/${id}`); } // --- DNS --- listDnsNameserverGroups(): Promise { return this.request("GET", "/dns/nameservers"); } createDnsNameserverGroup(params: { name: string; description: string; nameservers: Array<{ ip: string; ns_type: string; port: number }>; enabled: boolean; groups: string[]; primary: boolean; domains: string[]; search_domains_enabled: boolean; }): Promise { return this.request("POST", "/dns/nameservers", params); } updateDnsNameserverGroup(id: string, params: { name: string; description: string; nameservers: Array<{ ip: string; ns_type: string; port: number }>; enabled: boolean; groups: string[]; primary: boolean; domains: string[]; search_domains_enabled: boolean; }): Promise { return this.request("PUT", `/dns/nameservers/${id}`, params); } deleteDnsNameserverGroup(id: string): Promise { return this.request("DELETE", `/dns/nameservers/${id}`); } // --- Events --- listEvents(): Promise { return this.request("GET", "/events/audit"); } } ``` **Step 5: Run tests** Run: `deno test src/netbird/client.test.ts` Expected: PASS **Step 6: Commit** ``` feat: add NetBird API client with types and tests ``` --- ## Task 2: State file schema and validation **Files:** - Create: `src/state/schema.ts` - Create: `src/state/schema.test.ts` **Step 1: Define the `netbird.json` schema using Zod in `src/state/schema.ts`** ```typescript import { z } from "zod"; const SetupKeySchema = z.object({ type: z.enum(["one-off", "reusable"]), expires_in: z.number().int().positive(), usage_limit: z.number().int().nonnegative(), auto_groups: z.array(z.string()), enrolled: z.boolean(), }); const GroupSchema = z.object({ peers: z.array(z.string()), }); const PolicySchema = z.object({ description: z.string().default(""), enabled: z.boolean(), sources: z.array(z.string()), destinations: z.array(z.string()), bidirectional: z.boolean(), protocol: z.enum(["tcp", "udp", "icmp", "all"]).default("all"), action: z.enum(["accept", "drop"]).default("accept"), ports: z.array(z.string()).optional(), }); const RouteSchema = z.object({ description: z.string().default(""), network: z.string().optional(), domains: z.array(z.string()).optional(), peer_groups: z.array(z.string()), metric: z.number().int().min(1).max(9999).default(9999), masquerade: z.boolean().default(true), distribution_groups: z.array(z.string()), enabled: z.boolean(), keep_route: z.boolean().default(true), }); const NameserverSchema = z.object({ ip: z.string(), ns_type: z.string().default("udp"), port: z.number().int().default(53), }); const DnsNameserverGroupSchema = z.object({ description: z.string().default(""), nameservers: z.array(NameserverSchema).min(1).max(3), enabled: z.boolean(), groups: z.array(z.string()), primary: z.boolean(), domains: z.array(z.string()), search_domains_enabled: z.boolean().default(false), }); export const DesiredStateSchema = z.object({ groups: z.record(z.string(), GroupSchema), setup_keys: z.record(z.string(), SetupKeySchema), policies: z.record(z.string(), PolicySchema).default({}), routes: z.record(z.string(), RouteSchema).default({}), dns: z .object({ nameserver_groups: z.record(z.string(), DnsNameserverGroupSchema).default({}), }) .default({}), }); export type DesiredState = z.infer; export type SetupKeyConfig = z.infer; export type GroupConfig = z.infer; export type PolicyConfig = z.infer; export type RouteConfig = z.infer; export type DnsNameserverGroupConfig = z.infer; /** * Validates cross-references within the desired state: * - Groups referenced in policies must exist * - Groups referenced in setup key auto_groups must exist * - Peers referenced in groups must have a corresponding setup key * - Groups referenced in routes must exist * - Groups referenced in DNS nameserver groups must exist * * Returns an array of validation error strings. Empty means valid. */ export function validateCrossReferences(state: DesiredState): string[] { const errors: string[] = []; const groupNames = new Set(Object.keys(state.groups)); const keyNames = new Set(Object.keys(state.setup_keys)); // Check peers in groups reference existing setup keys for (const [groupName, group] of Object.entries(state.groups)) { for (const peer of group.peers) { if (!keyNames.has(peer)) { errors.push( `Group "${groupName}" references peer "${peer}" which has no setup key`, ); } } } // Check auto_groups in setup keys reference existing groups for (const [keyName, key] of Object.entries(state.setup_keys)) { for (const groupRef of key.auto_groups) { if (!groupNames.has(groupRef)) { errors.push( `Setup key "${keyName}" references auto_group "${groupRef}" which does not exist`, ); } } } // Check groups in policies reference existing groups for (const [policyName, policy] of Object.entries(state.policies)) { for (const src of policy.sources) { if (!groupNames.has(src)) { errors.push( `Policy "${policyName}" source "${src}" does not exist as a group`, ); } } for (const dst of policy.destinations) { if (!groupNames.has(dst)) { errors.push( `Policy "${policyName}" destination "${dst}" does not exist as a group`, ); } } } // Check groups in routes for (const [routeName, route] of Object.entries(state.routes)) { for (const g of route.peer_groups) { if (!groupNames.has(g)) { errors.push( `Route "${routeName}" peer_group "${g}" does not exist`, ); } } for (const g of route.distribution_groups) { if (!groupNames.has(g)) { errors.push( `Route "${routeName}" distribution_group "${g}" does not exist`, ); } } } // Check groups in DNS for (const [dnsName, dns] of Object.entries(state.dns.nameserver_groups)) { for (const g of dns.groups) { if (!groupNames.has(g)) { errors.push( `DNS nameserver group "${dnsName}" group "${g}" does not exist`, ); } } } return errors; } ``` **Step 2: Write tests in `src/state/schema.test.ts`** ```typescript import { assertEquals, assertThrows } from "jsr:@std/assert"; import { DesiredStateSchema, validateCrossReferences } from "./schema.ts"; const VALID_STATE = { groups: { pilots: { peers: ["Pilot-hawk-72"] }, "ground-stations": { peers: ["GS-hawk-72"] }, }, setup_keys: { "GS-hawk-72": { type: "one-off" as const, expires_in: 604800, usage_limit: 1, auto_groups: ["ground-stations"], enrolled: true, }, "Pilot-hawk-72": { type: "one-off" as const, expires_in: 604800, usage_limit: 1, auto_groups: ["pilots"], enrolled: false, }, }, policies: { "pilots-to-gs": { description: "Allow pilots to reach ground stations", enabled: true, sources: ["pilots"], destinations: ["ground-stations"], bidirectional: true, protocol: "all" as const, }, }, routes: {}, dns: { nameserver_groups: {} }, }; Deno.test("DesiredStateSchema parses valid state", () => { const result = DesiredStateSchema.parse(VALID_STATE); assertEquals(Object.keys(result.groups).length, 2); assertEquals(Object.keys(result.setup_keys).length, 2); }); Deno.test("DesiredStateSchema rejects invalid setup key type", () => { const invalid = structuredClone(VALID_STATE); (invalid.setup_keys["GS-hawk-72"] as Record).type = "invalid"; assertThrows(() => DesiredStateSchema.parse(invalid)); }); Deno.test("validateCrossReferences returns empty for valid state", () => { const state = DesiredStateSchema.parse(VALID_STATE); const errors = validateCrossReferences(state); assertEquals(errors.length, 0); }); Deno.test("validateCrossReferences catches missing group in policy", () => { const bad = structuredClone(VALID_STATE); bad.policies["pilots-to-gs"].sources = ["nonexistent"]; const state = DesiredStateSchema.parse(bad); const errors = validateCrossReferences(state); assertEquals(errors.length, 1); assertEquals(errors[0].includes("nonexistent"), true); }); Deno.test("validateCrossReferences catches peer without setup key", () => { const bad = structuredClone(VALID_STATE); bad.groups.pilots.peers.push("Unknown-peer"); const state = DesiredStateSchema.parse(bad); const errors = validateCrossReferences(state); assertEquals(errors.length, 1); assertEquals(errors[0].includes("Unknown-peer"), true); }); ``` **Step 3: Run tests** Run: `deno test src/state/schema.test.ts` Expected: PASS **Step 4: Commit** ``` feat: add desired state schema with Zod validation and cross-reference checks ``` --- ## Task 3: Actual state fetcher **Files:** - Create: `src/state/actual.ts` - Create: `src/state/actual.test.ts` The actual state fetcher calls all NetBird list endpoints and normalizes the results into a structure that can be compared with the desired state. The key job is building bidirectional name<->ID mappings. **Step 1: Write test in `src/state/actual.test.ts`** ```typescript import { assertEquals } from "jsr:@std/assert"; import { fetchActualState } from "./actual.ts"; import type { NbGroup, NbPeer, NbPolicy, NbRoute, NbSetupKey, NbDnsNameserverGroup, NbEvent } from "../netbird/types.ts"; /** Minimal mock NetBird client that returns predetermined data */ function mockClient(data: { groups?: NbGroup[]; setupKeys?: NbSetupKey[]; peers?: NbPeer[]; policies?: NbPolicy[]; routes?: NbRoute[]; dns?: NbDnsNameserverGroup[]; }) { return { listGroups: () => Promise.resolve(data.groups ?? []), listSetupKeys: () => Promise.resolve(data.setupKeys ?? []), listPeers: () => Promise.resolve(data.peers ?? []), listPolicies: () => Promise.resolve(data.policies ?? []), listRoutes: () => Promise.resolve(data.routes ?? []), listDnsNameserverGroups: () => Promise.resolve(data.dns ?? []), listEvents: () => Promise.resolve([] as NbEvent[]), }; } Deno.test("fetchActualState builds name-to-id maps", async () => { const actual = await fetchActualState(mockClient({ groups: [ { id: "g1", name: "pilots", peers_count: 0, peers: [], issued: "api" }, ], setupKeys: [ { id: 1, name: "Pilot-hawk-72", type: "one-off", key: "masked", expires: "2026-04-01T00:00:00Z", valid: true, revoked: false, used_times: 0, state: "valid", auto_groups: ["g1"], usage_limit: 1, }, ], })); assertEquals(actual.groupsByName.get("pilots")?.id, "g1"); assertEquals(actual.setupKeysByName.get("Pilot-hawk-72")?.id, 1); }); ``` **Step 2: Run test — expect FAIL** Run: `deno test src/state/actual.test.ts` **Step 3: Implement `src/state/actual.ts`** ```typescript import type { NetbirdClient } from "../netbird/client.ts"; import type { NbDnsNameserverGroup, NbGroup, NbPeer, NbPolicy, NbRoute, NbSetupKey, } from "../netbird/types.ts"; /** Indexed view of all current NetBird state */ export interface ActualState { groups: NbGroup[]; groupsByName: Map; groupsById: Map; setupKeys: NbSetupKey[]; setupKeysByName: Map; peers: NbPeer[]; peersByName: Map; peersById: Map; policies: NbPolicy[]; policiesByName: Map; routes: NbRoute[]; routesByNetworkId: Map; dns: NbDnsNameserverGroup[]; dnsByName: Map; } type ClientLike = Pick< NetbirdClient, | "listGroups" | "listSetupKeys" | "listPeers" | "listPolicies" | "listRoutes" | "listDnsNameserverGroups" >; export async function fetchActualState(client: ClientLike): Promise { const [groups, setupKeys, peers, policies, routes, dns] = await Promise.all([ client.listGroups(), client.listSetupKeys(), client.listPeers(), client.listPolicies(), client.listRoutes(), client.listDnsNameserverGroups(), ]); return { groups, groupsByName: new Map(groups.map((g) => [g.name, g])), groupsById: new Map(groups.map((g) => [g.id, g])), setupKeys, setupKeysByName: new Map(setupKeys.map((k) => [k.name, k])), peers, peersByName: new Map(peers.map((p) => [p.name, p])), peersById: new Map(peers.map((p) => [p.id, p])), policies, policiesByName: new Map(policies.map((p) => [p.name, p])), routes, routesByNetworkId: new Map(routes.map((r) => [r.network_id, r])), dns, dnsByName: new Map(dns.map((d) => [d.name, d])), }; } ``` **Step 4: Run tests** Run: `deno test src/state/actual.test.ts` Expected: PASS **Step 5: Commit** ``` feat: add actual state fetcher with name/ID indexing ``` --- ## Task 4: Diff engine — compute operations **Files:** - Create: `src/reconcile/diff.ts` - Create: `src/reconcile/operations.ts` - Create: `src/reconcile/diff.test.ts` This is the core reconciliation logic. It compares desired state against actual state and produces an ordered list of operations. **Step 1: Define operation types in `src/reconcile/operations.ts`** ```typescript export type OperationType = | "create_group" | "update_group" | "delete_group" | "create_setup_key" | "delete_setup_key" | "rename_peer" | "update_peer_groups" | "delete_peer" | "create_policy" | "update_policy" | "delete_policy" | "create_route" | "update_route" | "delete_route" | "create_dns" | "update_dns" | "delete_dns"; export interface Operation { type: OperationType; name: string; details?: Record; } export interface OperationResult extends Operation { status: "success" | "failed" | "skipped"; error?: string; } /** Order in which operation types must be executed */ export const EXECUTION_ORDER: OperationType[] = [ // Create groups first (policies/routes depend on them) "create_group", "update_group", // Setup keys (peers depend on them) "create_setup_key", // Peer operations "rename_peer", "update_peer_groups", // Policies (depend on groups) "create_policy", "update_policy", // Routes (depend on groups) "create_route", "update_route", // DNS (depend on groups) "create_dns", "update_dns", // Deletions in reverse dependency order "delete_dns", "delete_route", "delete_policy", "delete_peer", "delete_setup_key", "delete_group", ]; ``` **Step 2: Write diff tests in `src/reconcile/diff.test.ts`** ```typescript import { assertEquals } from "jsr:@std/assert"; import { computeDiff } from "./diff.ts"; import type { DesiredState } from "../state/schema.ts"; import type { ActualState } from "../state/actual.ts"; function emptyActual(): ActualState { return { groups: [], groupsByName: new Map(), groupsById: new Map(), setupKeys: [], setupKeysByName: new Map(), peers: [], peersByName: new Map(), peersById: new Map(), policies: [], policiesByName: new Map(), routes: [], routesByNetworkId: new Map(), dns: [], dnsByName: new Map(), }; } const DESIRED: DesiredState = { groups: { pilots: { peers: ["Pilot-hawk-72"] } }, setup_keys: { "Pilot-hawk-72": { type: "one-off", expires_in: 604800, usage_limit: 1, auto_groups: ["pilots"], enrolled: false, }, }, policies: {}, routes: {}, dns: { nameserver_groups: {} }, }; Deno.test("computeDiff against empty actual produces create ops", () => { const ops = computeDiff(DESIRED, emptyActual()); const types = ops.map((o) => o.type); assertEquals(types.includes("create_group"), true); assertEquals(types.includes("create_setup_key"), true); }); Deno.test("computeDiff with matching state produces no ops", () => { const actual = emptyActual(); actual.groupsByName.set("pilots", { id: "g1", name: "pilots", peers_count: 1, peers: [{ id: "p1", name: "Pilot-hawk-72" }], issued: "api", }); actual.groups = [actual.groupsByName.get("pilots")!]; actual.setupKeysByName.set("Pilot-hawk-72", { id: 1, name: "Pilot-hawk-72", type: "one-off", key: "masked", expires: "2026-04-01T00:00:00Z", valid: true, revoked: false, used_times: 0, state: "valid", auto_groups: ["g1"], usage_limit: 1, }); actual.setupKeys = [actual.setupKeysByName.get("Pilot-hawk-72")!]; const ops = computeDiff(DESIRED, actual); assertEquals(ops.length, 0); }); ``` **Step 3: Run test — expect FAIL** Run: `deno test src/reconcile/diff.test.ts` **Step 4: Implement `src/reconcile/diff.ts`** This is a large module. The diff compares each resource type and produces operations. ```typescript import type { DesiredState } from "../state/schema.ts"; import type { ActualState } from "../state/actual.ts"; import type { Operation } from "./operations.ts"; import { EXECUTION_ORDER } from "./operations.ts"; export function computeDiff( desired: DesiredState, actual: ActualState, ): Operation[] { const ops: Operation[] = []; // --- Groups --- const desiredGroupNames = new Set(Object.keys(desired.groups)); for (const [name, group] of Object.entries(desired.groups)) { const existing = actual.groupsByName.get(name); if (!existing) { ops.push({ type: "create_group", name, details: { peers: group.peers } }); } else { // Check if peer membership changed const existingPeerNames = new Set(existing.peers.map((p) => p.name)); const desiredPeerNames = new Set(group.peers); const same = existingPeerNames.size === desiredPeerNames.size && [...desiredPeerNames].every((p) => existingPeerNames.has(p)); if (!same) { ops.push({ type: "update_group", name, details: { id: existing.id, peers: group.peers }, }); } } } // Groups in actual but not in desired — delete (only API-issued, not system groups) for (const group of actual.groups) { if (!desiredGroupNames.has(group.name) && group.issued === "api") { ops.push({ type: "delete_group", name: group.name, details: { id: group.id } }); } } // --- Setup Keys --- const desiredKeyNames = new Set(Object.keys(desired.setup_keys)); for (const [name, key] of Object.entries(desired.setup_keys)) { const existing = actual.setupKeysByName.get(name); if (!existing) { // Only create if not yet enrolled (enrolled keys already exist) if (!key.enrolled) { ops.push({ type: "create_setup_key", name, details: { type: key.type, expires_in: key.expires_in, auto_groups: key.auto_groups, usage_limit: key.usage_limit, }, }); } } // Setup keys are immutable in NetBird — no update operation. // If config changed, user must delete and recreate (manual process). } for (const key of actual.setupKeys) { if (!desiredKeyNames.has(key.name)) { ops.push({ type: "delete_setup_key", name: key.name, details: { id: key.id } }); } } // --- Peers --- // We don't create peers (they self-enroll). We rename and reassign groups. // Peer deletion is when a setup key is in desired but the peer should be removed. // For now, peers not in any desired group's peer list get flagged for deletion // only if they match a setup key that was removed from desired state. const allDesiredPeerNames = new Set(); for (const group of Object.values(desired.groups)) { for (const p of group.peers) allDesiredPeerNames.add(p); } // --- Policies --- const desiredPolicyNames = new Set(Object.keys(desired.policies)); for (const [name, policy] of Object.entries(desired.policies)) { const existing = actual.policiesByName.get(name); if (!existing) { ops.push({ type: "create_policy", name, details: { description: policy.description, enabled: policy.enabled, sources: policy.sources, destinations: policy.destinations, bidirectional: policy.bidirectional, protocol: policy.protocol, action: policy.action, ports: policy.ports, }, }); } else { // Check if policy needs update by comparing source/dest group names const existingSources = existing.rules.flatMap((r) => r.sources.map((s) => typeof s === "string" ? s : s.name) ); const existingDests = existing.rules.flatMap((r) => r.destinations.map((d) => typeof d === "string" ? d : d.name) ); const sourcesMatch = JSON.stringify(existingSources.sort()) === JSON.stringify([...policy.sources].sort()); const destsMatch = JSON.stringify(existingDests.sort()) === JSON.stringify([...policy.destinations].sort()); const enabledMatch = existing.enabled === policy.enabled; if (!sourcesMatch || !destsMatch || !enabledMatch) { ops.push({ type: "update_policy", name, details: { id: existing.id, description: policy.description, enabled: policy.enabled, sources: policy.sources, destinations: policy.destinations, bidirectional: policy.bidirectional, protocol: policy.protocol, action: policy.action, ports: policy.ports, }, }); } } } for (const policy of actual.policies) { if (!desiredPolicyNames.has(policy.name)) { ops.push({ type: "delete_policy", name: policy.name, details: { id: policy.id } }); } } // --- Routes --- const desiredRouteNames = new Set(Object.keys(desired.routes)); for (const [name, route] of Object.entries(desired.routes)) { const existing = actual.routesByNetworkId.get(name); if (!existing) { ops.push({ type: "create_route", name, details: { description: route.description, network: route.network, domains: route.domains, peer_groups: route.peer_groups, metric: route.metric, masquerade: route.masquerade, distribution_groups: route.distribution_groups, enabled: route.enabled, keep_route: route.keep_route, }, }); } else { // Simplified update check — compare key fields const needsUpdate = existing.enabled !== route.enabled || existing.description !== route.description || existing.network !== route.network; if (needsUpdate) { ops.push({ type: "update_route", name, details: { id: existing.id, description: route.description, network: route.network, domains: route.domains, peer_groups: route.peer_groups, metric: route.metric, masquerade: route.masquerade, distribution_groups: route.distribution_groups, enabled: route.enabled, keep_route: route.keep_route, }, }); } } } for (const route of actual.routes) { if (!desiredRouteNames.has(route.network_id)) { ops.push({ type: "delete_route", name: route.network_id, details: { id: route.id }, }); } } // --- DNS --- const desiredDnsNames = new Set( Object.keys(desired.dns.nameserver_groups), ); for (const [name, dns] of Object.entries(desired.dns.nameserver_groups)) { const existing = actual.dnsByName.get(name); if (!existing) { ops.push({ type: "create_dns", name, details: { ...dns } }); } else { const needsUpdate = existing.enabled !== dns.enabled || existing.primary !== dns.primary || JSON.stringify(existing.nameservers) !== JSON.stringify(dns.nameservers); if (needsUpdate) { ops.push({ type: "update_dns", name, details: { id: existing.id, ...dns }, }); } } } for (const dns of actual.dns) { if (!desiredDnsNames.has(dns.name)) { ops.push({ type: "delete_dns", name: dns.name, details: { id: dns.id } }); } } // Sort by execution order return ops.sort((a, b) => EXECUTION_ORDER.indexOf(a.type) - EXECUTION_ORDER.indexOf(b.type) ); } ``` **Step 5: Run tests** Run: `deno test src/reconcile/diff.test.ts` Expected: PASS **Step 6: Commit** ``` feat: add diff engine computing operations from desired vs actual state ``` --- ## Task 5: Operation executor **Files:** - Create: `src/reconcile/executor.ts` - Create: `src/reconcile/executor.test.ts` The executor takes a list of operations and applies them against the NetBird API. It resolves names to IDs (since the diff uses names but the API needs IDs), executes in order, and aborts on first failure. **Step 1: Write test in `src/reconcile/executor.test.ts`** Test that the executor calls the right client methods and aborts on failure. ```typescript import { assertEquals } from "jsr:@std/assert"; import { executeOperations } from "./executor.ts"; import type { Operation, OperationResult } from "./operations.ts"; import type { ActualState } from "../state/actual.ts"; Deno.test("executor calls createGroup for create_group op", async () => { const calls: string[] = []; const mockClient = { createGroup: (name: string) => { calls.push(`createGroup:${name}`); return Promise.resolve({ id: "new-g1", name, peers_count: 0, peers: [], issued: "api" as const }); }, }; const ops: Operation[] = [ { type: "create_group", name: "pilots" }, ]; const results = await executeOperations(ops, mockClient as never, emptyActual()); assertEquals(calls, ["createGroup:pilots"]); assertEquals(results[0].status, "success"); }); Deno.test("executor aborts on first failure", async () => { const mockClient = { createGroup: () => Promise.reject(new Error("API down")), createSetupKey: () => Promise.resolve({ id: 1, key: "k" }), }; const ops: Operation[] = [ { type: "create_group", name: "pilots" }, { type: "create_setup_key", name: "key1" }, ]; const results = await executeOperations(ops, mockClient as never, emptyActual()); assertEquals(results[0].status, "failed"); assertEquals(results.length, 1); // second op never executed }); function emptyActual(): ActualState { return { groups: [], groupsByName: new Map(), groupsById: new Map(), setupKeys: [], setupKeysByName: new Map(), peers: [], peersByName: new Map(), peersById: new Map(), policies: [], policiesByName: new Map(), routes: [], routesByNetworkId: new Map(), dns: [], dnsByName: new Map(), }; } ``` **Step 2: Run test — expect FAIL** **Step 3: Implement `src/reconcile/executor.ts`** The executor is a large switch/case that dispatches each operation type to the correct client method. It needs the actual state to resolve group name -> ID for policies/routes, and it tracks newly created group IDs to use in subsequent operations. ```typescript import type { NetbirdClient } from "../netbird/client.ts"; import type { ActualState } from "../state/actual.ts"; import type { Operation, OperationResult } from "./operations.ts"; /** * Execute operations sequentially. Aborts on first failure. * Returns results for all executed operations (including the failed one). * Also returns any newly created setup keys. */ export async function executeOperations( ops: Operation[], client: NetbirdClient, actual: ActualState, ): Promise { const results: OperationResult[] = []; // Track name->ID for resources created during this execution const createdGroupIds = new Map(); const createdKeys = new Map(); // name -> raw key /** Resolve group name to NetBird ID, checking both actual state and newly created */ const resolveGroupId = (name: string): string | undefined => { return createdGroupIds.get(name) ?? actual.groupsByName.get(name)?.id; }; /** Resolve multiple group names to IDs. Throws if any not found. */ const resolveGroupIds = (names: string[]): string[] => { return names.map((n) => { const id = resolveGroupId(n); if (!id) throw new Error(`Cannot resolve group "${n}" to ID`); return id; }); }; /** Resolve peer names to IDs using actual state */ const resolvePeerIds = (names: string[]): string[] => { return names.flatMap((n) => { const peer = actual.peersByName.get(n); return peer ? [peer.id] : []; }); }; for (const op of ops) { try { switch (op.type) { case "create_group": { const peerIds = resolvePeerIds( (op.details?.peers as string[]) ?? [], ); const created = await client.createGroup(op.name, peerIds); createdGroupIds.set(op.name, created.id); break; } case "update_group": { const peerIds = resolvePeerIds( (op.details?.peers as string[]) ?? [], ); await client.updateGroup( op.details!.id as string, op.name, peerIds, ); break; } case "delete_group": await client.deleteGroup(op.details!.id as string); break; case "create_setup_key": { const d = op.details!; const autoGroupIds = resolveGroupIds(d.auto_groups as string[]); const created = await client.createSetupKey({ name: op.name, type: d.type as "one-off" | "reusable", expires_in: d.expires_in as number, auto_groups: autoGroupIds, usage_limit: d.usage_limit as number, }); createdKeys.set(op.name, created.key); break; } case "delete_setup_key": await client.deleteSetupKey(op.details!.id as number); break; case "rename_peer": await client.updatePeer(op.details!.id as string, { name: op.name, ssh_enabled: false, login_expiration_enabled: false, inactivity_expiration_enabled: false, }); break; case "delete_peer": await client.deletePeer(op.details!.id as string); break; case "create_policy": { const d = op.details!; const sourceIds = resolveGroupIds(d.sources as string[]); const destIds = resolveGroupIds(d.destinations as string[]); await client.createPolicy({ name: op.name, description: (d.description as string) ?? "", enabled: d.enabled as boolean, rules: [{ name: op.name, description: (d.description as string) ?? "", enabled: d.enabled as boolean, action: (d.action as "accept" | "drop") ?? "accept", bidirectional: d.bidirectional as boolean, protocol: d.protocol as string, ports: d.ports as string[] | undefined, sources: sourceIds, destinations: destIds, }], }); break; } case "update_policy": { const d = op.details!; const sourceIds = resolveGroupIds(d.sources as string[]); const destIds = resolveGroupIds(d.destinations as string[]); await client.updatePolicy(d.id as string, { name: op.name, description: (d.description as string) ?? "", enabled: d.enabled as boolean, rules: [{ name: op.name, description: (d.description as string) ?? "", enabled: d.enabled as boolean, action: (d.action as "accept" | "drop") ?? "accept", bidirectional: d.bidirectional as boolean, protocol: d.protocol as string, ports: d.ports as string[] | undefined, sources: sourceIds, destinations: destIds, }], }); break; } case "delete_policy": await client.deletePolicy(op.details!.id as string); break; case "create_route": { const d = op.details!; const peerGroupIds = resolveGroupIds(d.peer_groups as string[]); const distGroupIds = resolveGroupIds(d.distribution_groups as string[]); await client.createRoute({ description: (d.description as string) ?? "", network_id: op.name, enabled: d.enabled as boolean, peer_groups: peerGroupIds, network: d.network as string | undefined, domains: d.domains as string[] | undefined, metric: (d.metric as number) ?? 9999, masquerade: (d.masquerade as boolean) ?? true, groups: distGroupIds, keep_route: (d.keep_route as boolean) ?? true, }); break; } case "update_route": { const d = op.details!; const peerGroupIds = resolveGroupIds(d.peer_groups as string[]); const distGroupIds = resolveGroupIds(d.distribution_groups as string[]); await client.updateRoute(d.id as string, { description: (d.description as string) ?? "", network_id: op.name, enabled: d.enabled as boolean, peer_groups: peerGroupIds, network: d.network as string | undefined, domains: d.domains as string[] | undefined, metric: (d.metric as number) ?? 9999, masquerade: (d.masquerade as boolean) ?? true, groups: distGroupIds, keep_route: (d.keep_route as boolean) ?? true, }); break; } case "delete_route": await client.deleteRoute(op.details!.id as string); break; case "create_dns": { const d = op.details!; const groupIds = resolveGroupIds(d.groups as string[]); await client.createDnsNameserverGroup({ name: op.name, description: (d.description as string) ?? "", nameservers: d.nameservers as Array<{ ip: string; ns_type: string; port: number }>, enabled: d.enabled as boolean, groups: groupIds, primary: d.primary as boolean, domains: d.domains as string[], search_domains_enabled: (d.search_domains_enabled as boolean) ?? false, }); break; } case "update_dns": { const d = op.details!; const groupIds = resolveGroupIds(d.groups as string[]); await client.updateDnsNameserverGroup(d.id as string, { name: op.name, description: (d.description as string) ?? "", nameservers: d.nameservers as Array<{ ip: string; ns_type: string; port: number }>, enabled: d.enabled as boolean, groups: groupIds, primary: d.primary as boolean, domains: d.domains as string[], search_domains_enabled: (d.search_domains_enabled as boolean) ?? false, }); break; } case "delete_dns": await client.deleteDnsNameserverGroup(op.details!.id as string); break; default: { const _exhaustive: never = op.type; throw new Error(`Unknown operation type: ${_exhaustive}`); } } results.push({ ...op, status: "success" }); } catch (err) { results.push({ ...op, status: "failed", error: err instanceof Error ? err.message : String(err), }); break; // Abort on first failure } } return results; } /** Extract created keys from executor results (call after execution) */ export function getCreatedKeys( _results: OperationResult[], ): Map { // The executor tracks this internally. For external access, we need // to return it from executeOperations. This is a placeholder — // refactor executeOperations to return { results, createdKeys }. return new Map(); } ``` Note: The `createdKeys` map is local to `executeOperations` right now. Refactor the return type to include it: ```typescript export interface ExecutionResult { results: OperationResult[]; createdKeys: Map; } ``` Update the function signature and return accordingly. The test should verify `createdKeys` is populated when a setup key is created. **Step 4: Run tests** Run: `deno test src/reconcile/executor.test.ts` Expected: PASS **Step 5: Commit** ``` feat: add operation executor with abort-on-failure semantics ``` --- ## Task 6: Event poller **Files:** - Create: `src/poller/poller.ts` - Create: `src/poller/poller.test.ts` **Step 1: Write test in `src/poller/poller.test.ts`** ```typescript import { assertEquals } from "jsr:@std/assert"; import { processEnrollmentEvents } from "./poller.ts"; import type { NbEvent } from "../netbird/types.ts"; Deno.test("processEnrollmentEvents detects peer.setupkey.add", () => { const events: NbEvent[] = [ { id: 1, timestamp: "2026-03-03T10:00:00Z", activity: "Peer added", activity_code: "peer.setupkey.add", initiator_id: "system", initiator_name: "System", target_id: "peer-abc", meta: { name: "random-hostname", setup_key: "GS-hawk-72" }, }, { id: 2, timestamp: "2026-03-03T10:01:00Z", activity: "Route created", activity_code: "route.add", initiator_id: "user-1", initiator_name: "John", target_id: "route-1", meta: { name: "my-route" }, }, ]; const knownKeys = new Set(["GS-hawk-72", "Pilot-hawk-72"]); const enrollments = processEnrollmentEvents(events, knownKeys, null); assertEquals(enrollments.length, 1); assertEquals(enrollments[0].setupKeyName, "GS-hawk-72"); assertEquals(enrollments[0].peerId, "peer-abc"); }); Deno.test("processEnrollmentEvents filters by lastTimestamp", () => { const events: NbEvent[] = [ { id: 1, timestamp: "2026-03-03T09:00:00Z", activity: "Peer added", activity_code: "peer.setupkey.add", initiator_id: "system", initiator_name: "System", target_id: "peer-old", meta: { name: "old-peer", setup_key: "GS-hawk-72" }, }, { id: 2, timestamp: "2026-03-03T11:00:00Z", activity: "Peer added", activity_code: "peer.setupkey.add", initiator_id: "system", initiator_name: "System", target_id: "peer-new", meta: { name: "new-peer", setup_key: "Pilot-hawk-72" }, }, ]; const knownKeys = new Set(["GS-hawk-72", "Pilot-hawk-72"]); const enrollments = processEnrollmentEvents( events, knownKeys, "2026-03-03T10:00:00Z", ); assertEquals(enrollments.length, 1); assertEquals(enrollments[0].setupKeyName, "Pilot-hawk-72"); }); Deno.test("processEnrollmentEvents ignores unknown keys", () => { const events: NbEvent[] = [ { id: 1, timestamp: "2026-03-03T10:00:00Z", activity: "Peer added", activity_code: "peer.setupkey.add", initiator_id: "system", initiator_name: "System", target_id: "peer-unknown", meta: { name: "mystery-peer", setup_key: "Unknown-key" }, }, ]; const knownKeys = new Set(["GS-hawk-72"]); const enrollments = processEnrollmentEvents(events, knownKeys, null); assertEquals(enrollments.length, 0); }); ``` **Step 2: Run test — expect FAIL** **Step 3: Implement `src/poller/poller.ts`** ```typescript import type { NbEvent } from "../netbird/types.ts"; export interface EnrollmentDetection { setupKeyName: string; peerId: string; peerHostname: string; timestamp: string; } /** * Filters enrollment events from the full event list. * Returns enrollments for peers that enrolled using a known setup key * and whose timestamp is after lastTimestamp (if provided). */ export function processEnrollmentEvents( events: NbEvent[], knownKeyNames: Set, lastTimestamp: string | null, ): EnrollmentDetection[] { return events .filter((e) => { if (e.activity_code !== "peer.setupkey.add") return false; if (lastTimestamp && e.timestamp <= lastTimestamp) return false; if (!knownKeyNames.has(e.meta.setup_key)) { console.log( JSON.stringify({ msg: "unknown_enrollment", setup_key: e.meta.setup_key, peer_id: e.target_id, }), ); return false; } return true; }) .map((e) => ({ setupKeyName: e.meta.setup_key, peerId: e.target_id, peerHostname: e.meta.name, timestamp: e.timestamp, })); } ``` **Step 4: Run tests** Run: `deno test src/poller/poller.test.ts` Expected: PASS **Step 5: Commit** ``` feat: add enrollment event detection from NetBird audit events ``` --- ## Task 7: Gitea API client (for state commits) **Files:** - Create: `src/gitea/client.ts` - Create: `src/gitea/client.test.ts` **Step 1: Write test in `src/gitea/client.test.ts`** ```typescript import { assertEquals } from "jsr:@std/assert"; import { GiteaClient } from "./client.ts"; function mockFetch( responses: Map, ): typeof fetch { return (input: string | URL | Request, init?: RequestInit) => { const url = typeof input === "string" ? input : input instanceof URL ? input.toString() : input.url; const method = init?.method ?? "GET"; const key = `${method} ${url}`; const resp = responses.get(key); if (!resp) throw new Error(`Unmocked: ${key}`); return Promise.resolve( new Response(JSON.stringify(resp.body), { status: resp.status, headers: { "Content-Type": "application/json" }, }), ); }; } Deno.test("GiteaClient.getFileContent fetches file with SHA", async () => { const client = new GiteaClient( "https://gitea.example.com", "test-token", "BlastPilot/netbird-gitops", mockFetch(new Map([ [ "GET https://gitea.example.com/api/v1/repos/BlastPilot/netbird-gitops/contents/netbird.json?ref=main", { status: 200, body: { content: btoa('{"groups":{}}'), sha: "abc123", }, }, ], ])), ); const result = await client.getFileContent("netbird.json", "main"); assertEquals(result.sha, "abc123"); assertEquals(result.content, '{"groups":{}}'); }); ``` **Step 2: Run test — expect FAIL** **Step 3: Implement `src/gitea/client.ts`** ```typescript type FetchFn = typeof fetch; export class GiteaClient { constructor( private readonly baseUrl: string, private readonly token: string, private readonly repo: string, // "owner/repo" private readonly fetchFn: FetchFn = fetch, ) {} private async request( method: string, path: string, body?: unknown, ): Promise { const url = `${this.baseUrl}/api/v1${path}`; const headers: Record = { Authorization: `token ${this.token}`, Accept: "application/json", }; if (body !== undefined) { headers["Content-Type"] = "application/json"; } const resp = await this.fetchFn(url, { method, headers, body: body !== undefined ? JSON.stringify(body) : undefined, }); if (!resp.ok) { const text = await resp.text().catch(() => ""); throw new Error(`Gitea API ${method} ${path} returned ${resp.status}: ${text}`); } return resp.json() as Promise; } /** Get file content and SHA for optimistic concurrency */ async getFileContent( path: string, ref: string, ): Promise<{ content: string; sha: string }> { const data = await this.request<{ content: string; sha: string }>( "GET", `/repos/${this.repo}/contents/${path}?ref=${ref}`, ); return { content: atob(data.content), sha: data.sha, }; } /** Update file with optimistic concurrency (SHA check) */ async updateFile( path: string, content: string, sha: string, message: string, branch: string, ): Promise { await this.request( "PUT", `/repos/${this.repo}/contents/${path}`, { content: btoa(content), sha, message, branch, }, ); } /** Post or update a PR comment */ async postIssueComment( issueNumber: number, body: string, ): Promise { await this.request( "POST", `/repos/${this.repo}/issues/${issueNumber}/comments`, { body }, ); } } ``` **Step 4: Run tests** Run: `deno test src/gitea/client.test.ts` Expected: PASS **Step 5: Commit** ``` feat: add Gitea API client for state commits and PR comments ``` --- ## Task 8: Poller background loop with Gitea commit **Files:** - Create: `src/poller/loop.ts` - Create: `src/poller/state.ts` This task wires the enrollment detection from Task 6 to the Gitea client from Task 7, creating the full background polling loop. **Step 1: Create `src/poller/state.ts`** — persists poll state to disk ```typescript import { join } from "jsr:@std/path"; export interface PollerState { lastEventTimestamp: string | null; } export async function loadPollerState(dataDir: string): Promise { const path = join(dataDir, "poller-state.json"); try { const text = await Deno.readTextFile(path); return JSON.parse(text) as PollerState; } catch { return { lastEventTimestamp: null }; } } export async function savePollerState( dataDir: string, state: PollerState, ): Promise { const path = join(dataDir, "poller-state.json"); await Deno.mkdir(dataDir, { recursive: true }); await Deno.writeTextFile(path, JSON.stringify(state, null, 2)); } ``` **Step 2: Create `src/poller/loop.ts`** — the background loop ```typescript import type { NetbirdClient } from "../netbird/client.ts"; import type { GiteaClient } from "../gitea/client.ts"; import type { Config } from "../config.ts"; import type { DesiredState } from "../state/schema.ts"; import { DesiredStateSchema } from "../state/schema.ts"; import { processEnrollmentEvents } from "./poller.ts"; import { loadPollerState, savePollerState } from "./state.ts"; export interface PollerContext { config: Config; netbird: NetbirdClient; gitea: GiteaClient; /** Set to true while a reconcile is in progress — poller defers */ reconcileInProgress: { value: boolean }; } export async function pollOnce(ctx: PollerContext): Promise { if (ctx.reconcileInProgress.value) { console.log(JSON.stringify({ msg: "poll_deferred", reason: "reconcile_in_progress" })); return; } const pollerState = await loadPollerState(ctx.config.dataDir); // Fetch current desired state from git let desired: DesiredState; try { const { content } = await ctx.gitea.getFileContent("netbird.json", "main"); desired = DesiredStateSchema.parse(JSON.parse(content)); } catch (err) { console.log(JSON.stringify({ msg: "poll_error", error: "failed to fetch desired state", detail: err instanceof Error ? err.message : String(err), })); return; } const knownKeys = new Set(Object.keys(desired.setup_keys)); const unenrolledKeys = new Set( Object.entries(desired.setup_keys) .filter(([_, v]) => !v.enrolled) .map(([k]) => k), ); if (unenrolledKeys.size === 0) { // Nothing to watch for return; } // Fetch events const events = await ctx.netbird.listEvents(); const enrollments = processEnrollmentEvents( events, unenrolledKeys, pollerState.lastEventTimestamp, ); if (enrollments.length === 0) return; // Process each enrollment for (const enrollment of enrollments) { console.log(JSON.stringify({ msg: "enrollment_detected", setup_key: enrollment.setupKeyName, peer_id: enrollment.peerId, })); // Rename peer try { await ctx.netbird.updatePeer(enrollment.peerId, { name: enrollment.setupKeyName, ssh_enabled: false, login_expiration_enabled: false, inactivity_expiration_enabled: false, }); console.log(JSON.stringify({ msg: "peer_renamed", peer_id: enrollment.peerId, new_name: enrollment.setupKeyName, })); } catch (err) { console.log(JSON.stringify({ msg: "peer_rename_failed", peer_id: enrollment.peerId, error: err instanceof Error ? err.message : String(err), })); continue; } // Update enrolled status in git try { const { content, sha } = await ctx.gitea.getFileContent( "netbird.json", "main", ); const state = JSON.parse(content); if (state.setup_keys?.[enrollment.setupKeyName]) { state.setup_keys[enrollment.setupKeyName].enrolled = true; } await ctx.gitea.updateFile( "netbird.json", JSON.stringify(state, null, 2), sha, `chore: mark ${enrollment.setupKeyName} as enrolled [automated]`, "main", ); console.log(JSON.stringify({ msg: "state_committed", setup_key: enrollment.setupKeyName, })); } catch (err) { console.log(JSON.stringify({ msg: "state_commit_failed", setup_key: enrollment.setupKeyName, error: err instanceof Error ? err.message : String(err), })); } } // Update last event timestamp const latestTimestamp = enrollments[enrollments.length - 1].timestamp; await savePollerState(ctx.config.dataDir, { lastEventTimestamp: latestTimestamp, }); } /** Starts the polling loop. Returns an AbortController to stop it. */ export function startPollerLoop(ctx: PollerContext): AbortController { const controller = new AbortController(); const intervalMs = ctx.config.pollIntervalSeconds * 1000; const run = async () => { while (!controller.signal.aborted) { try { await pollOnce(ctx); } catch (err) { console.log(JSON.stringify({ msg: "poll_error", error: err instanceof Error ? err.message : String(err), })); } await new Promise((resolve) => { const timer = setTimeout(resolve, intervalMs); controller.signal.addEventListener("abort", () => { clearTimeout(timer); resolve(); }, { once: true }); }); } }; run(); // Fire and forget — runs in background return controller; } ``` **Step 3: Commit** ``` feat: add poller background loop with Gitea state commit ``` --- ## Task 9: HTTP server **Files:** - Modify: `src/main.ts` - Create: `src/server.ts` **Step 1: Create `src/server.ts`** This is the HTTP server with three endpoints. It wires together all the components. ```typescript import type { Config } from "./config.ts"; import type { NetbirdClient } from "./netbird/client.ts"; import type { GiteaClient } from "./gitea/client.ts"; import { DesiredStateSchema, validateCrossReferences } from "./state/schema.ts"; import { fetchActualState } from "./state/actual.ts"; import { computeDiff } from "./reconcile/diff.ts"; import { executeOperations } from "./reconcile/executor.ts"; import { pollOnce, type PollerContext } from "./poller/loop.ts"; export interface ServerContext { config: Config; netbird: NetbirdClient; gitea: GiteaClient; reconcileInProgress: { value: boolean }; } export function createHandler(ctx: ServerContext): Deno.ServeHandler { return async (req: Request): Promise => { const url = new URL(req.url); // Health check — no auth if (url.pathname === "/health" && req.method === "GET") { return Response.json({ status: "ok" }); } // Auth check for all other endpoints const authHeader = req.headers.get("Authorization"); if (authHeader !== `Bearer ${ctx.config.reconcilerToken}`) { return Response.json({ error: "unauthorized" }, { status: 401 }); } if (url.pathname === "/reconcile" && req.method === "POST") { return handleReconcile(req, url, ctx); } if (url.pathname === "/sync-events" && req.method === "POST") { return handleSyncEvents(ctx); } return Response.json({ error: "not found" }, { status: 404 }); }; } async function handleReconcile( req: Request, url: URL, ctx: ServerContext, ): Promise { const dryRun = url.searchParams.get("dry_run") === "true"; // Parse and validate desired state let body: unknown; try { body = await req.json(); } catch { return Response.json({ error: "invalid JSON body" }, { status: 400 }); } const parseResult = DesiredStateSchema.safeParse(body); if (!parseResult.success) { return Response.json( { error: "invalid state schema", details: parseResult.error.issues }, { status: 400 }, ); } const desired = parseResult.data; const crossRefErrors = validateCrossReferences(desired); if (crossRefErrors.length > 0) { return Response.json( { error: "cross-reference validation failed", details: crossRefErrors }, { status: 400 }, ); } // Fetch actual state ctx.reconcileInProgress.value = true; try { const actual = await fetchActualState(ctx.netbird); const operations = computeDiff(desired, actual); if (dryRun) { return Response.json({ status: "planned", operations: operations.map((op) => ({ type: op.type, name: op.name, })), summary: summarize(operations.map((op) => ({ ...op, status: "pending" }))), }); } if (operations.length === 0) { return Response.json({ status: "applied", operations: [], created_keys: {}, summary: { created: 0, updated: 0, deleted: 0, failed: 0 }, }); } const { results, createdKeys } = await executeOperations( operations, ctx.netbird, actual, ); const failed = results.some((r) => r.status === "failed"); return Response.json({ status: failed ? "error" : "applied", operations: results.map((r) => ({ type: r.type, name: r.name, status: r.status, ...(r.error ? { error: r.error } : {}), })), created_keys: Object.fromEntries(createdKeys), summary: summarize(results), }); } finally { ctx.reconcileInProgress.value = false; } } async function handleSyncEvents(ctx: ServerContext): Promise { const pollerCtx: PollerContext = { config: ctx.config, netbird: ctx.netbird, gitea: ctx.gitea, reconcileInProgress: ctx.reconcileInProgress, }; // Force reconcileInProgress to false so pollOnce runs const saved = ctx.reconcileInProgress.value; ctx.reconcileInProgress.value = false; try { await pollOnce(pollerCtx); } finally { ctx.reconcileInProgress.value = saved; } return Response.json({ status: "synced" }); } function summarize( results: Array<{ type: string; status: string }>, ): { created: number; updated: number; deleted: number; failed: number } { let created = 0, updated = 0, deleted = 0, failed = 0; for (const r of results) { if (r.status === "failed") { failed++; continue; } if (r.type.startsWith("create_")) created++; else if (r.type.startsWith("update_") || r.type === "rename_peer") updated++; else if (r.type.startsWith("delete_")) deleted++; } return { created, updated, deleted, failed }; } ``` **Step 2: Update `src/main.ts` to start the server and poller** ```typescript import { loadConfig } from "./config.ts"; import { NetbirdClient } from "./netbird/client.ts"; import { GiteaClient } from "./gitea/client.ts"; import { createHandler } from "./server.ts"; import { startPollerLoop } from "./poller/loop.ts"; const config = loadConfig(); const netbird = new NetbirdClient(config.netbirdApiUrl, config.netbirdApiToken); const gitea = new GiteaClient( config.giteaUrl, config.giteaToken, config.giteaRepo, ); const reconcileInProgress = { value: false }; // Start background poller const pollerAbort = startPollerLoop({ config, netbird, gitea, reconcileInProgress, }); // Start HTTP server const handler = createHandler({ config, netbird, gitea, reconcileInProgress, }); console.log(JSON.stringify({ msg: "starting", port: config.port })); Deno.serve({ port: config.port, handler }); // Graceful shutdown Deno.addSignalListener("SIGTERM", () => { console.log(JSON.stringify({ msg: "shutting_down" })); pollerAbort.abort(); Deno.exit(0); }); ``` **Step 3: Verify compilation** Run: `deno check src/main.ts` Expected: no errors **Step 4: Commit** ``` feat: add HTTP server with /reconcile, /sync-events, /health endpoints ``` --- ## Task 10: CI workflow files **Files:** - Create: `.gitea/workflows/dry-run.yml` - Create: `.gitea/workflows/reconcile.yml` - Create: `.gitea/workflows/release.yml` **Step 1: Create `.gitea/workflows/dry-run.yml`** ```yaml name: Dry Run on: pull_request: paths: - 'netbird.json' jobs: dry-run: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Run dry-run reconcile id: plan run: | RESPONSE=$(curl -sf \ -X POST \ -H "Authorization: Bearer ${{ secrets.RECONCILER_TOKEN }}" \ -H "Content-Type: application/json" \ -d @netbird.json \ "${{ secrets.RECONCILER_URL }}/reconcile?dry_run=true") echo "response<> "$GITHUB_OUTPUT" echo "$RESPONSE" >> "$GITHUB_OUTPUT" echo "EOF" >> "$GITHUB_OUTPUT" - name: Format plan as markdown id: format run: | cat <<'SCRIPT' > format.py import json, sys data = json.loads(sys.stdin.read()) ops = data.get("operations", []) summary = data.get("summary", {}) lines = ["## NetBird Reconciliation Plan\n"] if not ops: lines.append("No changes detected.\n") else: lines.append("| Operation | Name |") lines.append("|-----------|------|") for op in ops: lines.append(f"| `{op['type']}` | {op['name']} |") lines.append("") s = summary lines.append(f"**Summary:** {s.get('created',0)} create, {s.get('updated',0)} update, {s.get('deleted',0)} delete") print("\n".join(lines)) SCRIPT COMMENT=$(echo '${{ steps.plan.outputs.response }}' | python3 format.py) echo "comment<> "$GITHUB_OUTPUT" echo "$COMMENT" >> "$GITHUB_OUTPUT" echo "EOF" >> "$GITHUB_OUTPUT" - name: Post PR comment env: GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} run: | curl -sf \ -X POST \ -H "Authorization: token ${GITEA_TOKEN}" \ -H "Content-Type: application/json" \ -d "{\"body\": $(echo '${{ steps.format.outputs.comment }}' | jq -Rs .)}" \ "${{ secrets.GITEA_URL }}/api/v1/repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments" ``` **Step 2: Create `.gitea/workflows/reconcile.yml`** ```yaml name: Reconcile on: push: branches: - main paths: - 'netbird.json' jobs: reconcile: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Sync events run: | curl -sf \ -X POST \ -H "Authorization: Bearer ${{ secrets.RECONCILER_TOKEN }}" \ "${{ secrets.RECONCILER_URL }}/sync-events" - name: Pull latest (poller may have committed) run: git pull --rebase - name: Apply reconcile id: reconcile run: | RESPONSE=$(curl -sf \ -X POST \ -H "Authorization: Bearer ${{ secrets.RECONCILER_TOKEN }}" \ -H "Content-Type: application/json" \ -d @netbird.json \ "${{ secrets.RECONCILER_URL }}/reconcile") echo "response<> "$GITHUB_OUTPUT" echo "$RESPONSE" >> "$GITHUB_OUTPUT" echo "EOF" >> "$GITHUB_OUTPUT" STATUS=$(echo "$RESPONSE" | jq -r '.status') if [ "$STATUS" = "error" ]; then echo "Reconcile failed" echo "$RESPONSE" | jq . exit 1 fi - name: Encrypt and upload setup keys if: success() run: | KEYS=$(echo '${{ steps.reconcile.outputs.response }}' | jq -r '.created_keys // empty') if [ -n "$KEYS" ] && [ "$KEYS" != "{}" ] && [ "$KEYS" != "null" ]; then echo "$KEYS" | age -r "${{ secrets.AGE_PUBLIC_KEY }}" -o setup-keys.age echo "Setup keys encrypted to setup-keys.age" else echo "No new keys created" exit 0 fi - name: Upload artifact if: success() uses: actions/upload-artifact@v4 with: name: setup-keys path: setup-keys.age if-no-files-found: ignore ``` **Step 3: Create `.gitea/workflows/release.yml`** ```yaml name: Release on: push: tags: - 'v*' jobs: build: runs-on: ubuntu-latest container: image: denoland/deno:debian steps: - uses: actions/checkout@v4 - name: Compile run: deno compile --allow-net --allow-read --allow-write --allow-env --output reconciler src/main.ts - name: Build Docker image run: | docker build -t ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:${{ github.ref_name }} . docker tag ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:${{ github.ref_name }} \ ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:latest - name: Push Docker image run: | echo "${{ secrets.PACKAGE_TOKEN }}" | docker login ${{ secrets.GITEA_URL }} -u achilles-ci-bot --password-stdin docker push ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:${{ github.ref_name }} docker push ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:latest ``` **Step 4: Commit** ``` feat: add Gitea Actions CI workflows for dry-run, reconcile, and release ``` --- ## Task 11: Seed `netbird.json` with initial state **Files:** - Create: `netbird.json` **Step 1: Create the initial state file** This should reflect the current BlastPilot NetBird configuration. Start minimal — populate with actual groups/policies after deploying the reconciler and importing existing state. ```json { "groups": {}, "setup_keys": {}, "policies": {}, "routes": {}, "dns": { "nameserver_groups": {} } } ``` **Step 2: Commit** ``` feat: add empty netbird.json state file ``` --- ## Task 12: Docker Compose deployment config **Files:** - Create: `deploy/docker-compose.yml` - Create: `deploy/.env.example` **Step 1: Create `deploy/docker-compose.yml`** ```yaml services: netbird-reconciler: image: gitea.internal/blastpilot/netbird-reconciler:latest restart: unless-stopped env_file: .env volumes: - reconciler-data:/data ports: - "127.0.0.1:8080:8080" healthcheck: test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"] interval: 30s timeout: 5s retries: 3 labels: - "traefik.enable=true" - "traefik.http.routers.reconciler.rule=Host(`reconciler.internal`)" - "traefik.http.services.reconciler.loadbalancer.server.port=8080" volumes: reconciler-data: ``` **Step 2: Create `deploy/.env.example`** ``` NETBIRD_API_URL=https://netbird.example.com/api NETBIRD_API_TOKEN= GITEA_URL=https://gitea.example.com GITEA_TOKEN= GITEA_REPO=BlastPilot/netbird-gitops RECONCILER_TOKEN= POLL_INTERVAL_SECONDS=30 PORT=8080 DATA_DIR=/data ``` **Step 3: Commit** ``` feat: add Docker Compose deployment config ``` --- ## Task 13: Integration test with mock NetBird server **Files:** - Create: `src/integration.test.ts` Write an end-to-end test that starts the HTTP server, posts a reconcile request with a known desired state against a mock NetBird API, and verifies the correct API calls were made. **Step 1: Write integration test** ```typescript import { assertEquals } from "jsr:@std/assert"; import { createHandler } from "./server.ts"; import { NetbirdClient } from "./netbird/client.ts"; import { GiteaClient } from "./gitea/client.ts"; /** Tracks all API calls made */ interface ApiCall { method: string; path: string; body?: unknown; } function createMockNetbird(): { client: NetbirdClient; calls: ApiCall[] } { const calls: ApiCall[] = []; const responses = new Map([ // List endpoints return empty by default ["GET /groups", { status: 200, body: [] }], ["GET /setup-keys", { status: 200, body: [] }], ["GET /peers", { status: 200, body: [] }], ["GET /policies", { status: 200, body: [] }], ["GET /routes", { status: 200, body: [] }], ["GET /dns/nameservers", { status: 200, body: [] }], ["GET /events/audit", { status: 200, body: [] }], ]); const mockFetch: typeof fetch = async (input, init) => { const url = typeof input === "string" ? input : input instanceof URL ? input.toString() : input.url; const path = url.replace("https://nb.test/api", ""); const method = init?.method ?? "GET"; calls.push({ method, path, body: init?.body ? JSON.parse(init.body as string) : undefined, }); // Handle create operations if (method === "POST" && path === "/groups") { const body = JSON.parse(init?.body as string); return new Response( JSON.stringify({ id: `g-${body.name}`, name: body.name, peers_count: 0, peers: [], issued: "api" }), { status: 201, headers: { "Content-Type": "application/json" } }, ); } if (method === "POST" && path === "/setup-keys") { const body = JSON.parse(init?.body as string); return new Response( JSON.stringify({ id: 1, name: body.name, key: "TEST-KEY-12345", state: "valid" }), { status: 201, headers: { "Content-Type": "application/json" } }, ); } if (method === "POST" && path === "/policies") { const body = JSON.parse(init?.body as string); return new Response( JSON.stringify({ id: `p-${body.name}`, ...body }), { status: 201, headers: { "Content-Type": "application/json" } }, ); } const key = `${method} ${path}`; const resp = responses.get(key); if (!resp) { return new Response(JSON.stringify({ message: "not found" }), { status: 404 }); } return new Response(JSON.stringify(resp.body), { status: resp.status, headers: { "Content-Type": "application/json" }, }); }; const client = new NetbirdClient("https://nb.test/api", "test", mockFetch); return { client, calls }; } Deno.test("POST /reconcile dry_run returns planned operations", async () => { const { client } = createMockNetbird(); const handler = createHandler({ config: { reconcilerToken: "secret" } as never, netbird: client, gitea: {} as never, reconcileInProgress: { value: false }, }); const body = JSON.stringify({ groups: { pilots: { peers: [] } }, setup_keys: { "Pilot-hawk-72": { type: "one-off", expires_in: 604800, usage_limit: 1, auto_groups: ["pilots"], enrolled: false, }, }, policies: {}, routes: {}, dns: { nameserver_groups: {} }, }); const resp = await handler( new Request("http://localhost/reconcile?dry_run=true", { method: "POST", headers: { "Authorization": "Bearer secret", "Content-Type": "application/json", }, body, }), ); assertEquals(resp.status, 200); const data = await resp.json(); assertEquals(data.status, "planned"); const types = data.operations.map((o: { type: string }) => o.type); assertEquals(types.includes("create_group"), true); assertEquals(types.includes("create_setup_key"), true); }); Deno.test("POST /reconcile apply creates resources and returns keys", async () => { const { client, calls } = createMockNetbird(); const handler = createHandler({ config: { reconcilerToken: "secret" } as never, netbird: client, gitea: {} as never, reconcileInProgress: { value: false }, }); const body = JSON.stringify({ groups: { pilots: { peers: [] } }, setup_keys: { "Pilot-hawk-72": { type: "one-off", expires_in: 604800, usage_limit: 1, auto_groups: ["pilots"], enrolled: false, }, }, policies: {}, routes: {}, dns: { nameserver_groups: {} }, }); const resp = await handler( new Request("http://localhost/reconcile", { method: "POST", headers: { "Authorization": "Bearer secret", "Content-Type": "application/json", }, body, }), ); assertEquals(resp.status, 200); const data = await resp.json(); assertEquals(data.status, "applied"); assertEquals(data.created_keys["Pilot-hawk-72"], "TEST-KEY-12345"); // Verify API calls were made const postCalls = calls.filter((c) => c.method === "POST"); assertEquals(postCalls.some((c) => c.path === "/groups"), true); assertEquals(postCalls.some((c) => c.path === "/setup-keys"), true); }); Deno.test("POST /reconcile rejects unauthorized requests", async () => { const handler = createHandler({ config: { reconcilerToken: "secret" } as never, netbird: {} as never, gitea: {} as never, reconcileInProgress: { value: false }, }); const resp = await handler( new Request("http://localhost/reconcile", { method: "POST", headers: { "Authorization": "Bearer wrong" }, }), ); assertEquals(resp.status, 401); }); ``` **Step 2: Run tests** Run: `deno test src/integration.test.ts` Expected: PASS **Step 3: Commit** ``` test: add integration tests for reconcile HTTP endpoint ``` --- ## Task 14: Update blastpilot-public enrollment pipeline **Files:** - Modify: `../blastpilot-public/api/src/services/enrollment-pipeline.ts` - Modify: `../blastpilot-public/api/src/services/netbird.ts` This task modifies the enrollment pipeline to write to `netbird.json` in the `netbird-gitops` repo instead of creating `peers/enrollment-{N}.json` files. **Step 1: Update `handleApproval()` in enrollment-pipeline.ts** Change from creating a standalone peer JSON file to modifying `netbird.json`: - Fetch current `netbird.json` from `netbird-gitops` repo via Gitea API - Add setup key entries for GS and Pilot - Add peer references to appropriate groups - Create PR with the modified `netbird.json` **Step 2: Remove direct NetBird API calls from `handlePRMerge()`** The reconciler now handles key creation. `handlePRMerge()` should be simplified or removed (key delivery is manual for now). **Step 3: Update tests** **Step 4: Commit** ``` refactor: update enrollment pipeline to write to netbird.json instead of peer files ``` --- ## Task 15: Deploy and test **Step 1: Create Gitea repo `BlastPilot/netbird-gitops`** Via Gitea UI or API. Push all code. **Step 2: Generate tokens** - Generate `RECONCILER_TOKEN` (random 32-byte hex) - Ensure `NETBIRD_API_TOKEN` has management API access - Ensure `GITEA_TOKEN` has repo write access - Generate `AGE_PUBLIC_KEY` / private key pair **Step 3: Deploy Docker Compose on VPS** ```bash cd deploy cp .env.example .env # Fill in .env values docker compose up -d docker compose logs -f ``` **Step 4: Test health endpoint** ```bash curl http://localhost:8080/health # Expected: {"status":"ok"} ``` **Step 5: Test dry-run** ```bash curl -X POST \ -H "Authorization: Bearer " \ -H "Content-Type: application/json" \ -d @netbird.json \ http://localhost:8080/reconcile?dry_run=true ``` **Step 6: Test with a real enrollment** - Add a test setup key + peer to `netbird.json` - Push, verify CI creates key - Use the key to enroll a device - Verify poller detects enrollment, renames peer, commits state **Step 7: Commit any fixes** ``` fix: address deployment issues found during testing ```