BlastPilot/netbird-gitops
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
netbird-gitops/docs/plans/2026-03-03-netbird-reconciler-implementation.md
Prox b4f3ce7870 reconciler PoC
2026-03-06 13:21:08 +02:00

3171 lines
85 KiB
Markdown
Raw Blame History

# NetBird Reconciler Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to
> implement this plan task-by-task.
**Goal:** Build a Deno-based HTTP service that reconciles NetBird VPN
configuration from a declarative `netbird.json` state file, with event-driven
peer enrollment detection and Gitea Actions CI integration.
**Architecture:** Three-layer design. A NetBird API client wraps all management
API calls. A reconciliation engine diffs desired vs actual state and produces an
ordered operation plan. An HTTP server exposes `/reconcile`, `/sync-events`, and
`/health` endpoints. A background event poller detects peer enrollments and
commits state updates via Gitea API.
**Tech Stack:** Deno 2.x, Zod (schema validation), Deno standard library (HTTP
server), Docker
---
## Task 0: Scaffold project structure
**Files:**
- Create: `deno.json`
- Create: `src/main.ts`
- Create: `src/config.ts`
- Create: `.gitignore`
- Create: `Dockerfile`
**Step 1: Create `deno.json`**
```json
{
"name": "@blastpilot/netbird-reconciler",
"version": "0.1.0",
"tasks": {
"dev": "deno run --allow-net --allow-read --allow-write --allow-env --watch src/main.ts",
"start": "deno run --allow-net --allow-read --allow-write --allow-env src/main.ts",
"test": "deno test --allow-net --allow-read --allow-write --allow-env",
"check": "deno check src/main.ts",
"lint": "deno lint",
"fmt": "deno fmt"
},
"imports": {
"zod": "npm:zod@^3.23.0"
},
"compilerOptions": {
"strict": true
}
}
```
**Step 2: Create `src/config.ts`**
```typescript
import { z } from "zod";
const ConfigSchema = z.object({
netbirdApiUrl: z.string().url(),
netbirdApiToken: z.string().min(1),
giteaUrl: z.string().url(),
giteaToken: z.string().min(1),
giteaRepo: z.string().regex(/^[^/]+\/[^/]+$/), // owner/repo
reconcilerToken: z.string().min(1),
pollIntervalSeconds: z.coerce.number().int().positive().default(30),
port: z.coerce.number().int().positive().default(8080),
dataDir: z.string().default("/data"),
});
export type Config = z.infer<typeof ConfigSchema>;
export function loadConfig(): Config {
return ConfigSchema.parse({
netbirdApiUrl: Deno.env.get("NETBIRD_API_URL"),
netbirdApiToken: Deno.env.get("NETBIRD_API_TOKEN"),
giteaUrl: Deno.env.get("GITEA_URL"),
giteaToken: Deno.env.get("GITEA_TOKEN"),
giteaRepo: Deno.env.get("GITEA_REPO"),
reconcilerToken: Deno.env.get("RECONCILER_TOKEN"),
pollIntervalSeconds: Deno.env.get("POLL_INTERVAL_SECONDS"),
port: Deno.env.get("PORT"),
dataDir: Deno.env.get("DATA_DIR"),
});
}
```
**Step 3: Create `src/main.ts`** (minimal placeholder)
```typescript
import { loadConfig } from "./config.ts";
const config = loadConfig();
console.log(JSON.stringify({ msg: "starting", port: config.port }));
```
**Step 4: Create `.gitignore`**
```
/data/
*.log
```
**Step 5: Create `Dockerfile`**
```dockerfile
FROM denoland/deno:2.2.2 AS builder
WORKDIR /app
COPY deno.json .
COPY src/ src/
RUN deno compile --allow-net --allow-read --allow-write --allow-env --output reconciler src/main.ts
FROM gcr.io/distroless/cc-debian12
COPY --from=builder /app/reconciler /usr/local/bin/reconciler
ENTRYPOINT ["reconciler"]
```
**Step 6: Verify project compiles**
Run: `deno check src/main.ts` Expected: no errors
**Step 7: Commit**
```
feat: scaffold netbird-reconciler project
```
---
## Task 1: NetBird API client — types and base client
**Files:**
- Create: `src/netbird/types.ts`
- Create: `src/netbird/client.ts`
- Create: `src/netbird/client.test.ts`
**Step 1: Define NetBird API response types in `src/netbird/types.ts`**
These types model the NetBird Management API responses. Only the fields we need
for reconciliation are included.
```typescript
/** Group as returned by GET /api/groups */
export interface NbGroup {
id: string;
name: string;
peers_count: number;
peers: Array<{ id: string; name: string }>;
issued: "api" | "jwt" | "integration";
}
/** Setup key as returned by GET /api/setup-keys */
export interface NbSetupKey {
id: number;
name: string;
type: "one-off" | "reusable";
key: string;
expires: string;
valid: boolean;
revoked: boolean;
used_times: number;
state: "valid" | "expired" | "revoked" | "overused";
auto_groups: string[];
usage_limit: number;
}
/** Peer as returned by GET /api/peers */
export interface NbPeer {
id: string;
name: string;
ip: string;
connected: boolean;
hostname: string;
os: string;
version: string;
groups: Array<{ id: string; name: string }>;
last_seen: string;
dns_label: string;
login_expiration_enabled: boolean;
ssh_enabled: boolean;
inactivity_expiration_enabled: boolean;
}
/** Policy as returned by GET /api/policies */
export interface NbPolicy {
id: string;
name: string;
description: string;
enabled: boolean;
rules: NbPolicyRule[];
}
export interface NbPolicyRule {
id?: string;
name: string;
description: string;
enabled: boolean;
action: "accept" | "drop";
bidirectional: boolean;
protocol: "tcp" | "udp" | "icmp" | "all";
ports?: string[];
sources: Array<string | { id: string; name: string }>;
destinations: Array<string | { id: string; name: string }>;
}
/** Route as returned by GET /api/routes */
export interface NbRoute {
id: string;
description: string;
network_id: string;
enabled: boolean;
peer?: string;
peer_groups?: string[];
network?: string;
domains?: string[];
metric: number;
masquerade: boolean;
groups: string[];
keep_route: boolean;
}
/** DNS nameserver group as returned by GET /api/dns/nameservers */
export interface NbDnsNameserverGroup {
id: string;
name: string;
description: string;
nameservers: Array<{
ip: string;
ns_type: string;
port: number;
}>;
enabled: boolean;
groups: string[];
primary: boolean;
domains: string[];
search_domains_enabled: boolean;
}
/** Audit event as returned by GET /api/events/audit */
export interface NbEvent {
id: number;
timestamp: string;
activity: string;
activity_code: string;
initiator_id: string;
initiator_name: string;
target_id: string;
meta: Record<string, string>;
}
```
**Step 2: Write test for base HTTP client in `src/netbird/client.test.ts`**
Test the client can be constructed and makes authenticated requests. Use a mock
fetch pattern: inject a fetch function so tests don't hit a real API.
```typescript
import { assertEquals } from "jsr:@std/assert";
import { NetbirdClient } from "./client.ts";
function mockFetch(
responses: Map<string, { status: number; body: unknown }>,
): typeof fetch {
return (input: string | URL | Request, init?: RequestInit) => {
const url = typeof input === "string"
? input
: input instanceof URL
? input.toString()
: input.url;
const method = init?.method ?? "GET";
const key = `${method} ${url}`;
const resp = responses.get(key);
if (!resp) throw new Error(`Unmocked request: ${key}`);
return Promise.resolve(
new Response(JSON.stringify(resp.body), {
status: resp.status,
headers: { "Content-Type": "application/json" },
}),
);
};
}
Deno.test("NetbirdClient.listGroups sends auth header and parses response", async () => {
const groups = [{
id: "g1",
name: "pilots",
peers_count: 1,
peers: [],
issued: "api",
}];
const client = new NetbirdClient(
"https://nb.example.com/api",
"test-token",
mockFetch(
new Map([
["GET https://nb.example.com/api/groups", {
status: 200,
body: groups,
}],
]),
),
);
const result = await client.listGroups();
assertEquals(result.length, 1);
assertEquals(result[0].name, "pilots");
});
Deno.test("NetbirdClient throws on non-2xx response", async () => {
const client = new NetbirdClient(
"https://nb.example.com/api",
"test-token",
mockFetch(
new Map([
["GET https://nb.example.com/api/groups", {
status: 401,
body: { message: "unauthorized" },
}],
]),
),
);
try {
await client.listGroups();
throw new Error("Should have thrown");
} catch (e) {
assertEquals((e as Error).message.includes("401"), true);
}
});
```
**Step 3: Run tests to verify they fail**
Run: `deno test src/netbird/client.test.ts` Expected: FAIL — `NetbirdClient` not
found
**Step 4: Implement base client in `src/netbird/client.ts`**
```typescript
import type {
NbDnsNameserverGroup,
NbEvent,
NbGroup,
NbPeer,
NbPolicy,
NbRoute,
NbSetupKey,
} from "./types.ts";
export class NetbirdApiError extends Error {
constructor(
public readonly status: number,
public readonly method: string,
public readonly path: string,
public readonly body: unknown,
) {
super(`NetBird API ${method} ${path} returned ${status}`);
this.name = "NetbirdApiError";
}
}
type FetchFn = typeof fetch;
export class NetbirdClient {
constructor(
private readonly baseUrl: string,
private readonly token: string,
private readonly fetchFn: FetchFn = fetch,
) {}
private async request<T>(
method: string,
path: string,
body?: unknown,
): Promise<T> {
const url = `${this.baseUrl}${path}`;
const headers: Record<string, string> = {
"Authorization": `Token ${this.token}`,
"Accept": "application/json",
};
if (body !== undefined) {
headers["Content-Type"] = "application/json";
}
const resp = await this.fetchFn(url, {
method,
headers,
body: body !== undefined ? JSON.stringify(body) : undefined,
});
if (!resp.ok) {
const text = await resp.text().catch(() => "");
throw new NetbirdApiError(resp.status, method, path, text);
}
if (resp.status === 204) return undefined as T;
return resp.json() as Promise<T>;
}
// --- Groups ---
listGroups(): Promise<NbGroup[]> {
return this.request("GET", "/groups");
}
createGroup(name: string, peers: string[] = []): Promise<NbGroup> {
return this.request("POST", "/groups", { name, peers });
}
updateGroup(
id: string,
name: string,
peers: string[] = [],
): Promise<NbGroup> {
return this.request("PUT", `/groups/${id}`, { name, peers });
}
deleteGroup(id: string): Promise<void> {
return this.request("DELETE", `/groups/${id}`);
}
// --- Setup Keys ---
listSetupKeys(): Promise<NbSetupKey[]> {
return this.request("GET", "/setup-keys");
}
createSetupKey(params: {
name: string;
type: "one-off" | "reusable";
expires_in: number;
auto_groups: string[];
usage_limit: number;
}): Promise<NbSetupKey> {
return this.request("POST", "/setup-keys", params);
}
deleteSetupKey(id: number): Promise<void> {
return this.request("DELETE", `/setup-keys/${id}`);
}
// --- Peers ---
listPeers(): Promise<NbPeer[]> {
return this.request("GET", "/peers");
}
updatePeer(id: string, params: {
name: string;
ssh_enabled: boolean;
login_expiration_enabled: boolean;
inactivity_expiration_enabled: boolean;
}): Promise<NbPeer> {
return this.request("PUT", `/peers/${id}`, params);
}
deletePeer(id: string): Promise<void> {
return this.request("DELETE", `/peers/${id}`);
}
// --- Policies ---
listPolicies(): Promise<NbPolicy[]> {
return this.request("GET", "/policies");
}
createPolicy(params: {
name: string;
description: string;
enabled: boolean;
rules: Array<{
name: string;
description: string;
enabled: boolean;
action: "accept" | "drop";
bidirectional: boolean;
protocol: string;
ports?: string[];
sources: string[];
destinations: string[];
}>;
}): Promise<NbPolicy> {
return this.request("POST", "/policies", params);
}
updatePolicy(id: string, params: {
name: string;
description: string;
enabled: boolean;
rules: Array<{
name: string;
description: string;
enabled: boolean;
action: "accept" | "drop";
bidirectional: boolean;
protocol: string;
ports?: string[];
sources: string[];
destinations: string[];
}>;
}): Promise<NbPolicy> {
return this.request("PUT", `/policies/${id}`, params);
}
deletePolicy(id: string): Promise<void> {
return this.request("DELETE", `/policies/${id}`);
}
// --- Routes ---
listRoutes(): Promise<NbRoute[]> {
return this.request("GET", "/routes");
}
createRoute(params: {
description: string;
network_id: string;
enabled: boolean;
peer_groups?: string[];
network?: string;
domains?: string[];
metric: number;
masquerade: boolean;
groups: string[];
keep_route: boolean;
}): Promise<NbRoute> {
return this.request("POST", "/routes", params);
}
updateRoute(id: string, params: {
description: string;
network_id: string;
enabled: boolean;
peer_groups?: string[];
network?: string;
domains?: string[];
metric: number;
masquerade: boolean;
groups: string[];
keep_route: boolean;
}): Promise<NbRoute> {
return this.request("PUT", `/routes/${id}`, params);
}
deleteRoute(id: string): Promise<void> {
return this.request("DELETE", `/routes/${id}`);
}
// --- DNS ---
listDnsNameserverGroups(): Promise<NbDnsNameserverGroup[]> {
return this.request("GET", "/dns/nameservers");
}
createDnsNameserverGroup(params: {
name: string;
description: string;
nameservers: Array<{ ip: string; ns_type: string; port: number }>;
enabled: boolean;
groups: string[];
primary: boolean;
domains: string[];
search_domains_enabled: boolean;
}): Promise<NbDnsNameserverGroup> {
return this.request("POST", "/dns/nameservers", params);
}
updateDnsNameserverGroup(id: string, params: {
name: string;
description: string;
nameservers: Array<{ ip: string; ns_type: string; port: number }>;
enabled: boolean;
groups: string[];
primary: boolean;
domains: string[];
search_domains_enabled: boolean;
}): Promise<NbDnsNameserverGroup> {
return this.request("PUT", `/dns/nameservers/${id}`, params);
}
deleteDnsNameserverGroup(id: string): Promise<void> {
return this.request("DELETE", `/dns/nameservers/${id}`);
}
// --- Events ---
listEvents(): Promise<NbEvent[]> {
return this.request("GET", "/events/audit");
}
}
```
**Step 5: Run tests**
Run: `deno test src/netbird/client.test.ts` Expected: PASS
**Step 6: Commit**
```
feat: add NetBird API client with types and tests
```
---
## Task 2: State file schema and validation
**Files:**
- Create: `src/state/schema.ts`
- Create: `src/state/schema.test.ts`
**Step 1: Define the `netbird.json` schema using Zod in `src/state/schema.ts`**
```typescript
import { z } from "zod";
const SetupKeySchema = z.object({
type: z.enum(["one-off", "reusable"]),
expires_in: z.number().int().positive(),
usage_limit: z.number().int().nonnegative(),
auto_groups: z.array(z.string()),
enrolled: z.boolean(),
});
const GroupSchema = z.object({
peers: z.array(z.string()),
});
const PolicySchema = z.object({
description: z.string().default(""),
enabled: z.boolean(),
sources: z.array(z.string()),
destinations: z.array(z.string()),
bidirectional: z.boolean(),
protocol: z.enum(["tcp", "udp", "icmp", "all"]).default("all"),
action: z.enum(["accept", "drop"]).default("accept"),
ports: z.array(z.string()).optional(),
});
const RouteSchema = z.object({
description: z.string().default(""),
network: z.string().optional(),
domains: z.array(z.string()).optional(),
peer_groups: z.array(z.string()),
metric: z.number().int().min(1).max(9999).default(9999),
masquerade: z.boolean().default(true),
distribution_groups: z.array(z.string()),
enabled: z.boolean(),
keep_route: z.boolean().default(true),
});
const NameserverSchema = z.object({
ip: z.string(),
ns_type: z.string().default("udp"),
port: z.number().int().default(53),
});
const DnsNameserverGroupSchema = z.object({
description: z.string().default(""),
nameservers: z.array(NameserverSchema).min(1).max(3),
enabled: z.boolean(),
groups: z.array(z.string()),
primary: z.boolean(),
domains: z.array(z.string()),
search_domains_enabled: z.boolean().default(false),
});
export const DesiredStateSchema = z.object({
groups: z.record(z.string(), GroupSchema),
setup_keys: z.record(z.string(), SetupKeySchema),
policies: z.record(z.string(), PolicySchema).default({}),
routes: z.record(z.string(), RouteSchema).default({}),
dns: z
.object({
nameserver_groups: z.record(z.string(), DnsNameserverGroupSchema).default(
{},
),
})
.default({}),
});
export type DesiredState = z.infer<typeof DesiredStateSchema>;
export type SetupKeyConfig = z.infer<typeof SetupKeySchema>;
export type GroupConfig = z.infer<typeof GroupSchema>;
export type PolicyConfig = z.infer<typeof PolicySchema>;
export type RouteConfig = z.infer<typeof RouteSchema>;
export type DnsNameserverGroupConfig = z.infer<typeof DnsNameserverGroupSchema>;
/**
* Validates cross-references within the desired state:
* - Groups referenced in policies must exist
* - Groups referenced in setup key auto_groups must exist
* - Peers referenced in groups must have a corresponding setup key
* - Groups referenced in routes must exist
* - Groups referenced in DNS nameserver groups must exist
*
* Returns an array of validation error strings. Empty means valid.
*/
export function validateCrossReferences(state: DesiredState): string[] {
const errors: string[] = [];
const groupNames = new Set(Object.keys(state.groups));
const keyNames = new Set(Object.keys(state.setup_keys));
// Check peers in groups reference existing setup keys
for (const [groupName, group] of Object.entries(state.groups)) {
for (const peer of group.peers) {
if (!keyNames.has(peer)) {
errors.push(
`Group "${groupName}" references peer "${peer}" which has no setup key`,
);
}
}
}
// Check auto_groups in setup keys reference existing groups
for (const [keyName, key] of Object.entries(state.setup_keys)) {
for (const groupRef of key.auto_groups) {
if (!groupNames.has(groupRef)) {
errors.push(
`Setup key "${keyName}" references auto_group "${groupRef}" which does not exist`,
);
}
}
}
// Check groups in policies reference existing groups
for (const [policyName, policy] of Object.entries(state.policies)) {
for (const src of policy.sources) {
if (!groupNames.has(src)) {
errors.push(
`Policy "${policyName}" source "${src}" does not exist as a group`,
);
}
}
for (const dst of policy.destinations) {
if (!groupNames.has(dst)) {
errors.push(
`Policy "${policyName}" destination "${dst}" does not exist as a group`,
);
}
}
}
// Check groups in routes
for (const [routeName, route] of Object.entries(state.routes)) {
for (const g of route.peer_groups) {
if (!groupNames.has(g)) {
errors.push(
`Route "${routeName}" peer_group "${g}" does not exist`,
);
}
}
for (const g of route.distribution_groups) {
if (!groupNames.has(g)) {
errors.push(
`Route "${routeName}" distribution_group "${g}" does not exist`,
);
}
}
}
// Check groups in DNS
for (const [dnsName, dns] of Object.entries(state.dns.nameserver_groups)) {
for (const g of dns.groups) {
if (!groupNames.has(g)) {
errors.push(
`DNS nameserver group "${dnsName}" group "${g}" does not exist`,
);
}
}
}
return errors;
}
```
**Step 2: Write tests in `src/state/schema.test.ts`**
```typescript
import { assertEquals, assertThrows } from "jsr:@std/assert";
import { DesiredStateSchema, validateCrossReferences } from "./schema.ts";
const VALID_STATE = {
groups: {
pilots: { peers: ["Pilot-hawk-72"] },
"ground-stations": { peers: ["GS-hawk-72"] },
},
setup_keys: {
"GS-hawk-72": {
type: "one-off" as const,
expires_in: 604800,
usage_limit: 1,
auto_groups: ["ground-stations"],
enrolled: true,
},
"Pilot-hawk-72": {
type: "one-off" as const,
expires_in: 604800,
usage_limit: 1,
auto_groups: ["pilots"],
enrolled: false,
},
},
policies: {
"pilots-to-gs": {
description: "Allow pilots to reach ground stations",
enabled: true,
sources: ["pilots"],
destinations: ["ground-stations"],
bidirectional: true,
protocol: "all" as const,
},
},
routes: {},
dns: { nameserver_groups: {} },
};
Deno.test("DesiredStateSchema parses valid state", () => {
const result = DesiredStateSchema.parse(VALID_STATE);
assertEquals(Object.keys(result.groups).length, 2);
assertEquals(Object.keys(result.setup_keys).length, 2);
});
Deno.test("DesiredStateSchema rejects invalid setup key type", () => {
const invalid = structuredClone(VALID_STATE);
(invalid.setup_keys["GS-hawk-72"] as Record<string, unknown>).type =
"invalid";
assertThrows(() => DesiredStateSchema.parse(invalid));
});
Deno.test("validateCrossReferences returns empty for valid state", () => {
const state = DesiredStateSchema.parse(VALID_STATE);
const errors = validateCrossReferences(state);
assertEquals(errors.length, 0);
});
Deno.test("validateCrossReferences catches missing group in policy", () => {
const bad = structuredClone(VALID_STATE);
bad.policies["pilots-to-gs"].sources = ["nonexistent"];
const state = DesiredStateSchema.parse(bad);
const errors = validateCrossReferences(state);
assertEquals(errors.length, 1);
assertEquals(errors[0].includes("nonexistent"), true);
});
Deno.test("validateCrossReferences catches peer without setup key", () => {
const bad = structuredClone(VALID_STATE);
bad.groups.pilots.peers.push("Unknown-peer");
const state = DesiredStateSchema.parse(bad);
const errors = validateCrossReferences(state);
assertEquals(errors.length, 1);
assertEquals(errors[0].includes("Unknown-peer"), true);
});
```
**Step 3: Run tests**
Run: `deno test src/state/schema.test.ts` Expected: PASS
**Step 4: Commit**
```
feat: add desired state schema with Zod validation and cross-reference checks
```
---
## Task 3: Actual state fetcher
**Files:**
- Create: `src/state/actual.ts`
- Create: `src/state/actual.test.ts`
The actual state fetcher calls all NetBird list endpoints and normalizes the
results into a structure that can be compared with the desired state. The key
job is building bidirectional name<->ID mappings.
**Step 1: Write test in `src/state/actual.test.ts`**
```typescript
import { assertEquals } from "jsr:@std/assert";
import { fetchActualState } from "./actual.ts";
import type {
NbDnsNameserverGroup,
NbEvent,
NbGroup,
NbPeer,
NbPolicy,
NbRoute,
NbSetupKey,
} from "../netbird/types.ts";
/** Minimal mock NetBird client that returns predetermined data */
function mockClient(data: {
groups?: NbGroup[];
setupKeys?: NbSetupKey[];
peers?: NbPeer[];
policies?: NbPolicy[];
routes?: NbRoute[];
dns?: NbDnsNameserverGroup[];
}) {
return {
listGroups: () => Promise.resolve(data.groups ?? []),
listSetupKeys: () => Promise.resolve(data.setupKeys ?? []),
listPeers: () => Promise.resolve(data.peers ?? []),
listPolicies: () => Promise.resolve(data.policies ?? []),
listRoutes: () => Promise.resolve(data.routes ?? []),
listDnsNameserverGroups: () => Promise.resolve(data.dns ?? []),
listEvents: () => Promise.resolve([] as NbEvent[]),
};
}
Deno.test("fetchActualState builds name-to-id maps", async () => {
const actual = await fetchActualState(mockClient({
groups: [
{ id: "g1", name: "pilots", peers_count: 0, peers: [], issued: "api" },
],
setupKeys: [
{
id: 1,
name: "Pilot-hawk-72",
type: "one-off",
key: "masked",
expires: "2026-04-01T00:00:00Z",
valid: true,
revoked: false,
used_times: 0,
state: "valid",
auto_groups: ["g1"],
usage_limit: 1,
},
],
}));
assertEquals(actual.groupsByName.get("pilots")?.id, "g1");
assertEquals(actual.setupKeysByName.get("Pilot-hawk-72")?.id, 1);
});
```
**Step 2: Run test — expect FAIL**
Run: `deno test src/state/actual.test.ts`
**Step 3: Implement `src/state/actual.ts`**
```typescript
import type { NetbirdClient } from "../netbird/client.ts";
import type {
NbDnsNameserverGroup,
NbGroup,
NbPeer,
NbPolicy,
NbRoute,
NbSetupKey,
} from "../netbird/types.ts";
/** Indexed view of all current NetBird state */
export interface ActualState {
groups: NbGroup[];
groupsByName: Map<string, NbGroup>;
groupsById: Map<string, NbGroup>;
setupKeys: NbSetupKey[];
setupKeysByName: Map<string, NbSetupKey>;
peers: NbPeer[];
peersByName: Map<string, NbPeer>;
peersById: Map<string, NbPeer>;
policies: NbPolicy[];
policiesByName: Map<string, NbPolicy>;
routes: NbRoute[];
routesByNetworkId: Map<string, NbRoute>;
dns: NbDnsNameserverGroup[];
dnsByName: Map<string, NbDnsNameserverGroup>;
}
type ClientLike = Pick<
NetbirdClient,
| "listGroups"
| "listSetupKeys"
| "listPeers"
| "listPolicies"
| "listRoutes"
| "listDnsNameserverGroups"
>;
export async function fetchActualState(
client: ClientLike,
): Promise<ActualState> {
const [groups, setupKeys, peers, policies, routes, dns] = await Promise.all([
client.listGroups(),
client.listSetupKeys(),
client.listPeers(),
client.listPolicies(),
client.listRoutes(),
client.listDnsNameserverGroups(),
]);
return {
groups,
groupsByName: new Map(groups.map((g) => [g.name, g])),
groupsById: new Map(groups.map((g) => [g.id, g])),
setupKeys,
setupKeysByName: new Map(setupKeys.map((k) => [k.name, k])),
peers,
peersByName: new Map(peers.map((p) => [p.name, p])),
peersById: new Map(peers.map((p) => [p.id, p])),
policies,
policiesByName: new Map(policies.map((p) => [p.name, p])),
routes,
routesByNetworkId: new Map(routes.map((r) => [r.network_id, r])),
dns,
dnsByName: new Map(dns.map((d) => [d.name, d])),
};
}
```
**Step 4: Run tests**
Run: `deno test src/state/actual.test.ts` Expected: PASS
**Step 5: Commit**
```
feat: add actual state fetcher with name/ID indexing
```
---
## Task 4: Diff engine — compute operations
**Files:**
- Create: `src/reconcile/diff.ts`
- Create: `src/reconcile/operations.ts`
- Create: `src/reconcile/diff.test.ts`
This is the core reconciliation logic. It compares desired state against actual
state and produces an ordered list of operations.
**Step 1: Define operation types in `src/reconcile/operations.ts`**
```typescript
export type OperationType =
| "create_group"
| "update_group"
| "delete_group"
| "create_setup_key"
| "delete_setup_key"
| "rename_peer"
| "update_peer_groups"
| "delete_peer"
| "create_policy"
| "update_policy"
| "delete_policy"
| "create_route"
| "update_route"
| "delete_route"
| "create_dns"
| "update_dns"
| "delete_dns";
export interface Operation {
type: OperationType;
name: string;
details?: Record<string, unknown>;
}
export interface OperationResult extends Operation {
status: "success" | "failed" | "skipped";
error?: string;
}
/** Order in which operation types must be executed */
export const EXECUTION_ORDER: OperationType[] = [
// Create groups first (policies/routes depend on them)
"create_group",
"update_group",
// Setup keys (peers depend on them)
"create_setup_key",
// Peer operations
"rename_peer",
"update_peer_groups",
// Policies (depend on groups)
"create_policy",
"update_policy",
// Routes (depend on groups)
"create_route",
"update_route",
// DNS (depend on groups)
"create_dns",
"update_dns",
// Deletions in reverse dependency order
"delete_dns",
"delete_route",
"delete_policy",
"delete_peer",
"delete_setup_key",
"delete_group",
];
```
**Step 2: Write diff tests in `src/reconcile/diff.test.ts`**
```typescript
import { assertEquals } from "jsr:@std/assert";
import { computeDiff } from "./diff.ts";
import type { DesiredState } from "../state/schema.ts";
import type { ActualState } from "../state/actual.ts";
function emptyActual(): ActualState {
return {
groups: [],
groupsByName: new Map(),
groupsById: new Map(),
setupKeys: [],
setupKeysByName: new Map(),
peers: [],
peersByName: new Map(),
peersById: new Map(),
policies: [],
policiesByName: new Map(),
routes: [],
routesByNetworkId: new Map(),
dns: [],
dnsByName: new Map(),
};
}
const DESIRED: DesiredState = {
groups: { pilots: { peers: ["Pilot-hawk-72"] } },
setup_keys: {
"Pilot-hawk-72": {
type: "one-off",
expires_in: 604800,
usage_limit: 1,
auto_groups: ["pilots"],
enrolled: false,
},
},
policies: {},
routes: {},
dns: { nameserver_groups: {} },
};
Deno.test("computeDiff against empty actual produces create ops", () => {
const ops = computeDiff(DESIRED, emptyActual());
const types = ops.map((o) => o.type);
assertEquals(types.includes("create_group"), true);
assertEquals(types.includes("create_setup_key"), true);
});
Deno.test("computeDiff with matching state produces no ops", () => {
const actual = emptyActual();
actual.groupsByName.set("pilots", {
id: "g1",
name: "pilots",
peers_count: 1,
peers: [{ id: "p1", name: "Pilot-hawk-72" }],
issued: "api",
});
actual.groups = [actual.groupsByName.get("pilots")!];
actual.setupKeysByName.set("Pilot-hawk-72", {
id: 1,
name: "Pilot-hawk-72",
type: "one-off",
key: "masked",
expires: "2026-04-01T00:00:00Z",
valid: true,
revoked: false,
used_times: 0,
state: "valid",
auto_groups: ["g1"],
usage_limit: 1,
});
actual.setupKeys = [actual.setupKeysByName.get("Pilot-hawk-72")!];
const ops = computeDiff(DESIRED, actual);
assertEquals(ops.length, 0);
});
```
**Step 3: Run test — expect FAIL**
Run: `deno test src/reconcile/diff.test.ts`
**Step 4: Implement `src/reconcile/diff.ts`**
This is a large module. The diff compares each resource type and produces
operations.
```typescript
import type { DesiredState } from "../state/schema.ts";
import type { ActualState } from "../state/actual.ts";
import type { Operation } from "./operations.ts";
import { EXECUTION_ORDER } from "./operations.ts";
export function computeDiff(
desired: DesiredState,
actual: ActualState,
): Operation[] {
const ops: Operation[] = [];
// --- Groups ---
const desiredGroupNames = new Set(Object.keys(desired.groups));
for (const [name, group] of Object.entries(desired.groups)) {
const existing = actual.groupsByName.get(name);
if (!existing) {
ops.push({ type: "create_group", name, details: { peers: group.peers } });
} else {
// Check if peer membership changed
const existingPeerNames = new Set(existing.peers.map((p) => p.name));
const desiredPeerNames = new Set(group.peers);
const same = existingPeerNames.size === desiredPeerNames.size &&
[...desiredPeerNames].every((p) => existingPeerNames.has(p));
if (!same) {
ops.push({
type: "update_group",
name,
details: { id: existing.id, peers: group.peers },
});
}
}
}
// Groups in actual but not in desired — delete (only API-issued, not system groups)
for (const group of actual.groups) {
if (!desiredGroupNames.has(group.name) && group.issued === "api") {
ops.push({
type: "delete_group",
name: group.name,
details: { id: group.id },
});
}
}
// --- Setup Keys ---
const desiredKeyNames = new Set(Object.keys(desired.setup_keys));
for (const [name, key] of Object.entries(desired.setup_keys)) {
const existing = actual.setupKeysByName.get(name);
if (!existing) {
// Only create if not yet enrolled (enrolled keys already exist)
if (!key.enrolled) {
ops.push({
type: "create_setup_key",
name,
details: {
type: key.type,
expires_in: key.expires_in,
auto_groups: key.auto_groups,
usage_limit: key.usage_limit,
},
});
}
}
// Setup keys are immutable in NetBird — no update operation.
// If config changed, user must delete and recreate (manual process).
}
for (const key of actual.setupKeys) {
if (!desiredKeyNames.has(key.name)) {
ops.push({
type: "delete_setup_key",
name: key.name,
details: { id: key.id },
});
}
}
// --- Peers ---
// We don't create peers (they self-enroll). We rename and reassign groups.
// Peer deletion is when a setup key is in desired but the peer should be removed.
// For now, peers not in any desired group's peer list get flagged for deletion
// only if they match a setup key that was removed from desired state.
const allDesiredPeerNames = new Set<string>();
for (const group of Object.values(desired.groups)) {
for (const p of group.peers) allDesiredPeerNames.add(p);
}
// --- Policies ---
const desiredPolicyNames = new Set(Object.keys(desired.policies));
for (const [name, policy] of Object.entries(desired.policies)) {
const existing = actual.policiesByName.get(name);
if (!existing) {
ops.push({
type: "create_policy",
name,
details: {
description: policy.description,
enabled: policy.enabled,
sources: policy.sources,
destinations: policy.destinations,
bidirectional: policy.bidirectional,
protocol: policy.protocol,
action: policy.action,
ports: policy.ports,
},
});
} else {
// Check if policy needs update by comparing source/dest group names
const existingSources = existing.rules.flatMap((r) =>
r.sources.map((s) => typeof s === "string" ? s : s.name)
);
const existingDests = existing.rules.flatMap((r) =>
r.destinations.map((d) => typeof d === "string" ? d : d.name)
);
const sourcesMatch = JSON.stringify(existingSources.sort()) ===
JSON.stringify([...policy.sources].sort());
const destsMatch = JSON.stringify(existingDests.sort()) ===
JSON.stringify([...policy.destinations].sort());
const enabledMatch = existing.enabled === policy.enabled;
if (!sourcesMatch || !destsMatch || !enabledMatch) {
ops.push({
type: "update_policy",
name,
details: {
id: existing.id,
description: policy.description,
enabled: policy.enabled,
sources: policy.sources,
destinations: policy.destinations,
bidirectional: policy.bidirectional,
protocol: policy.protocol,
action: policy.action,
ports: policy.ports,
},
});
}
}
}
for (const policy of actual.policies) {
if (!desiredPolicyNames.has(policy.name)) {
ops.push({
type: "delete_policy",
name: policy.name,
details: { id: policy.id },
});
}
}
// --- Routes ---
const desiredRouteNames = new Set(Object.keys(desired.routes));
for (const [name, route] of Object.entries(desired.routes)) {
const existing = actual.routesByNetworkId.get(name);
if (!existing) {
ops.push({
type: "create_route",
name,
details: {
description: route.description,
network: route.network,
domains: route.domains,
peer_groups: route.peer_groups,
metric: route.metric,
masquerade: route.masquerade,
distribution_groups: route.distribution_groups,
enabled: route.enabled,
keep_route: route.keep_route,
},
});
} else {
// Simplified update check — compare key fields
const needsUpdate = existing.enabled !== route.enabled ||
existing.description !== route.description ||
existing.network !== route.network;
if (needsUpdate) {
ops.push({
type: "update_route",
name,
details: {
id: existing.id,
description: route.description,
network: route.network,
domains: route.domains,
peer_groups: route.peer_groups,
metric: route.metric,
masquerade: route.masquerade,
distribution_groups: route.distribution_groups,
enabled: route.enabled,
keep_route: route.keep_route,
},
});
}
}
}
for (const route of actual.routes) {
if (!desiredRouteNames.has(route.network_id)) {
ops.push({
type: "delete_route",
name: route.network_id,
details: { id: route.id },
});
}
}
// --- DNS ---
const desiredDnsNames = new Set(
Object.keys(desired.dns.nameserver_groups),
);
for (const [name, dns] of Object.entries(desired.dns.nameserver_groups)) {
const existing = actual.dnsByName.get(name);
if (!existing) {
ops.push({ type: "create_dns", name, details: { ...dns } });
} else {
const needsUpdate = existing.enabled !== dns.enabled ||
existing.primary !== dns.primary ||
JSON.stringify(existing.nameservers) !==
JSON.stringify(dns.nameservers);
if (needsUpdate) {
ops.push({
type: "update_dns",
name,
details: { id: existing.id, ...dns },
});
}
}
}
for (const dns of actual.dns) {
if (!desiredDnsNames.has(dns.name)) {
ops.push({ type: "delete_dns", name: dns.name, details: { id: dns.id } });
}
}
// Sort by execution order
return ops.sort((a, b) =>
EXECUTION_ORDER.indexOf(a.type) - EXECUTION_ORDER.indexOf(b.type)
);
}
```
**Step 5: Run tests**
Run: `deno test src/reconcile/diff.test.ts` Expected: PASS
**Step 6: Commit**
```
feat: add diff engine computing operations from desired vs actual state
```
---
## Task 5: Operation executor
**Files:**
- Create: `src/reconcile/executor.ts`
- Create: `src/reconcile/executor.test.ts`
The executor takes a list of operations and applies them against the NetBird
API. It resolves names to IDs (since the diff uses names but the API needs IDs),
executes in order, and aborts on first failure.
**Step 1: Write test in `src/reconcile/executor.test.ts`**
Test that the executor calls the right client methods and aborts on failure.
```typescript
import { assertEquals } from "jsr:@std/assert";
import { executeOperations } from "./executor.ts";
import type { Operation, OperationResult } from "./operations.ts";
import type { ActualState } from "../state/actual.ts";
Deno.test("executor calls createGroup for create_group op", async () => {
const calls: string[] = [];
const mockClient = {
createGroup: (name: string) => {
calls.push(`createGroup:${name}`);
return Promise.resolve({
id: "new-g1",
name,
peers_count: 0,
peers: [],
issued: "api" as const,
});
},
};
const ops: Operation[] = [
{ type: "create_group", name: "pilots" },
];
const results = await executeOperations(
ops,
mockClient as never,
emptyActual(),
);
assertEquals(calls, ["createGroup:pilots"]);
assertEquals(results[0].status, "success");
});
Deno.test("executor aborts on first failure", async () => {
const mockClient = {
createGroup: () => Promise.reject(new Error("API down")),
createSetupKey: () => Promise.resolve({ id: 1, key: "k" }),
};
const ops: Operation[] = [
{ type: "create_group", name: "pilots" },
{ type: "create_setup_key", name: "key1" },
];
const results = await executeOperations(
ops,
mockClient as never,
emptyActual(),
);
assertEquals(results[0].status, "failed");
assertEquals(results.length, 1); // second op never executed
});
function emptyActual(): ActualState {
return {
groups: [],
groupsByName: new Map(),
groupsById: new Map(),
setupKeys: [],
setupKeysByName: new Map(),
peers: [],
peersByName: new Map(),
peersById: new Map(),
policies: [],
policiesByName: new Map(),
routes: [],
routesByNetworkId: new Map(),
dns: [],
dnsByName: new Map(),
};
}
```
**Step 2: Run test — expect FAIL**
**Step 3: Implement `src/reconcile/executor.ts`**
The executor is a large switch/case that dispatches each operation type to the
correct client method. It needs the actual state to resolve group name -> ID for
policies/routes, and it tracks newly created group IDs to use in subsequent
operations.
```typescript
import type { NetbirdClient } from "../netbird/client.ts";
import type { ActualState } from "../state/actual.ts";
import type { Operation, OperationResult } from "./operations.ts";
/**
* Execute operations sequentially. Aborts on first failure.
* Returns results for all executed operations (including the failed one).
* Also returns any newly created setup keys.
*/
export async function executeOperations(
ops: Operation[],
client: NetbirdClient,
actual: ActualState,
): Promise<OperationResult[]> {
const results: OperationResult[] = [];
// Track name->ID for resources created during this execution
const createdGroupIds = new Map<string, string>();
const createdKeys = new Map<string, string>(); // name -> raw key
/** Resolve group name to NetBird ID, checking both actual state and newly created */
const resolveGroupId = (name: string): string | undefined => {
return createdGroupIds.get(name) ?? actual.groupsByName.get(name)?.id;
};
/** Resolve multiple group names to IDs. Throws if any not found. */
const resolveGroupIds = (names: string[]): string[] => {
return names.map((n) => {
const id = resolveGroupId(n);
if (!id) throw new Error(`Cannot resolve group "${n}" to ID`);
return id;
});
};
/** Resolve peer names to IDs using actual state */
const resolvePeerIds = (names: string[]): string[] => {
return names.flatMap((n) => {
const peer = actual.peersByName.get(n);
return peer ? [peer.id] : [];
});
};
for (const op of ops) {
try {
switch (op.type) {
case "create_group": {
const peerIds = resolvePeerIds(
(op.details?.peers as string[]) ?? [],
);
const created = await client.createGroup(op.name, peerIds);
createdGroupIds.set(op.name, created.id);
break;
}
case "update_group": {
const peerIds = resolvePeerIds(
(op.details?.peers as string[]) ?? [],
);
await client.updateGroup(
op.details!.id as string,
op.name,
peerIds,
);
break;
}
case "delete_group":
await client.deleteGroup(op.details!.id as string);
break;
case "create_setup_key": {
const d = op.details!;
const autoGroupIds = resolveGroupIds(d.auto_groups as string[]);
const created = await client.createSetupKey({
name: op.name,
type: d.type as "one-off" | "reusable",
expires_in: d.expires_in as number,
auto_groups: autoGroupIds,
usage_limit: d.usage_limit as number,
});
createdKeys.set(op.name, created.key);
break;
}
case "delete_setup_key":
await client.deleteSetupKey(op.details!.id as number);
break;
case "rename_peer":
await client.updatePeer(op.details!.id as string, {
name: op.name,
ssh_enabled: false,
login_expiration_enabled: false,
inactivity_expiration_enabled: false,
});
break;
case "delete_peer":
await client.deletePeer(op.details!.id as string);
break;
case "create_policy": {
const d = op.details!;
const sourceIds = resolveGroupIds(d.sources as string[]);
const destIds = resolveGroupIds(d.destinations as string[]);
await client.createPolicy({
name: op.name,
description: (d.description as string) ?? "",
enabled: d.enabled as boolean,
rules: [{
name: op.name,
description: (d.description as string) ?? "",
enabled: d.enabled as boolean,
action: (d.action as "accept" | "drop") ?? "accept",
bidirectional: d.bidirectional as boolean,
protocol: d.protocol as string,
ports: d.ports as string[] | undefined,
sources: sourceIds,
destinations: destIds,
}],
});
break;
}
case "update_policy": {
const d = op.details!;
const sourceIds = resolveGroupIds(d.sources as string[]);
const destIds = resolveGroupIds(d.destinations as string[]);
await client.updatePolicy(d.id as string, {
name: op.name,
description: (d.description as string) ?? "",
enabled: d.enabled as boolean,
rules: [{
name: op.name,
description: (d.description as string) ?? "",
enabled: d.enabled as boolean,
action: (d.action as "accept" | "drop") ?? "accept",
bidirectional: d.bidirectional as boolean,
protocol: d.protocol as string,
ports: d.ports as string[] | undefined,
sources: sourceIds,
destinations: destIds,
}],
});
break;
}
case "delete_policy":
await client.deletePolicy(op.details!.id as string);
break;
case "create_route": {
const d = op.details!;
const peerGroupIds = resolveGroupIds(d.peer_groups as string[]);
const distGroupIds = resolveGroupIds(
d.distribution_groups as string[],
);
await client.createRoute({
description: (d.description as string) ?? "",
network_id: op.name,
enabled: d.enabled as boolean,
peer_groups: peerGroupIds,
network: d.network as string | undefined,
domains: d.domains as string[] | undefined,
metric: (d.metric as number) ?? 9999,
masquerade: (d.masquerade as boolean) ?? true,
groups: distGroupIds,
keep_route: (d.keep_route as boolean) ?? true,
});
break;
}
case "update_route": {
const d = op.details!;
const peerGroupIds = resolveGroupIds(d.peer_groups as string[]);
const distGroupIds = resolveGroupIds(
d.distribution_groups as string[],
);
await client.updateRoute(d.id as string, {
description: (d.description as string) ?? "",
network_id: op.name,
enabled: d.enabled as boolean,
peer_groups: peerGroupIds,
network: d.network as string | undefined,
domains: d.domains as string[] | undefined,
metric: (d.metric as number) ?? 9999,
masquerade: (d.masquerade as boolean) ?? true,
groups: distGroupIds,
keep_route: (d.keep_route as boolean) ?? true,
});
break;
}
case "delete_route":
await client.deleteRoute(op.details!.id as string);
break;
case "create_dns": {
const d = op.details!;
const groupIds = resolveGroupIds(d.groups as string[]);
await client.createDnsNameserverGroup({
name: op.name,
description: (d.description as string) ?? "",
nameservers: d.nameservers as Array<
{ ip: string; ns_type: string; port: number }
>,
enabled: d.enabled as boolean,
groups: groupIds,
primary: d.primary as boolean,
domains: d.domains as string[],
search_domains_enabled: (d.search_domains_enabled as boolean) ??
false,
});
break;
}
case "update_dns": {
const d = op.details!;
const groupIds = resolveGroupIds(d.groups as string[]);
await client.updateDnsNameserverGroup(d.id as string, {
name: op.name,
description: (d.description as string) ?? "",
nameservers: d.nameservers as Array<
{ ip: string; ns_type: string; port: number }
>,
enabled: d.enabled as boolean,
groups: groupIds,
primary: d.primary as boolean,
domains: d.domains as string[],
search_domains_enabled: (d.search_domains_enabled as boolean) ??
false,
});
break;
}
case "delete_dns":
await client.deleteDnsNameserverGroup(op.details!.id as string);
break;
default: {
const _exhaustive: never = op.type;
throw new Error(`Unknown operation type: ${_exhaustive}`);
}
}
results.push({ ...op, status: "success" });
} catch (err) {
results.push({
...op,
status: "failed",
error: err instanceof Error ? err.message : String(err),
});
break; // Abort on first failure
}
}
return results;
}
/** Extract created keys from executor results (call after execution) */
export function getCreatedKeys(
_results: OperationResult[],
): Map<string, string> {
// The executor tracks this internally. For external access, we need
// to return it from executeOperations. This is a placeholder —
// refactor executeOperations to return { results, createdKeys }.
return new Map();
}
```
Note: The `createdKeys` map is local to `executeOperations` right now. Refactor
the return type to include it:
```typescript
export interface ExecutionResult {
results: OperationResult[];
createdKeys: Map<string, string>;
}
```
Update the function signature and return accordingly. The test should verify
`createdKeys` is populated when a setup key is created.
**Step 4: Run tests**
Run: `deno test src/reconcile/executor.test.ts` Expected: PASS
**Step 5: Commit**
```
feat: add operation executor with abort-on-failure semantics
```
---
## Task 6: Event poller
**Files:**
- Create: `src/poller/poller.ts`
- Create: `src/poller/poller.test.ts`
**Step 1: Write test in `src/poller/poller.test.ts`**
```typescript
import { assertEquals } from "jsr:@std/assert";
import { processEnrollmentEvents } from "./poller.ts";
import type { NbEvent } from "../netbird/types.ts";
Deno.test("processEnrollmentEvents detects peer.setupkey.add", () => {
const events: NbEvent[] = [
{
id: 1,
timestamp: "2026-03-03T10:00:00Z",
activity: "Peer added",
activity_code: "peer.setupkey.add",
initiator_id: "system",
initiator_name: "System",
target_id: "peer-abc",
meta: { name: "random-hostname", setup_key: "GS-hawk-72" },
},
{
id: 2,
timestamp: "2026-03-03T10:01:00Z",
activity: "Route created",
activity_code: "route.add",
initiator_id: "user-1",
initiator_name: "John",
target_id: "route-1",
meta: { name: "my-route" },
},
];
const knownKeys = new Set(["GS-hawk-72", "Pilot-hawk-72"]);
const enrollments = processEnrollmentEvents(events, knownKeys, null);
assertEquals(enrollments.length, 1);
assertEquals(enrollments[0].setupKeyName, "GS-hawk-72");
assertEquals(enrollments[0].peerId, "peer-abc");
});
Deno.test("processEnrollmentEvents filters by lastTimestamp", () => {
const events: NbEvent[] = [
{
id: 1,
timestamp: "2026-03-03T09:00:00Z",
activity: "Peer added",
activity_code: "peer.setupkey.add",
initiator_id: "system",
initiator_name: "System",
target_id: "peer-old",
meta: { name: "old-peer", setup_key: "GS-hawk-72" },
},
{
id: 2,
timestamp: "2026-03-03T11:00:00Z",
activity: "Peer added",
activity_code: "peer.setupkey.add",
initiator_id: "system",
initiator_name: "System",
target_id: "peer-new",
meta: { name: "new-peer", setup_key: "Pilot-hawk-72" },
},
];
const knownKeys = new Set(["GS-hawk-72", "Pilot-hawk-72"]);
const enrollments = processEnrollmentEvents(
events,
knownKeys,
"2026-03-03T10:00:00Z",
);
assertEquals(enrollments.length, 1);
assertEquals(enrollments[0].setupKeyName, "Pilot-hawk-72");
});
Deno.test("processEnrollmentEvents ignores unknown keys", () => {
const events: NbEvent[] = [
{
id: 1,
timestamp: "2026-03-03T10:00:00Z",
activity: "Peer added",
activity_code: "peer.setupkey.add",
initiator_id: "system",
initiator_name: "System",
target_id: "peer-unknown",
meta: { name: "mystery-peer", setup_key: "Unknown-key" },
},
];
const knownKeys = new Set(["GS-hawk-72"]);
const enrollments = processEnrollmentEvents(events, knownKeys, null);
assertEquals(enrollments.length, 0);
});
```
**Step 2: Run test — expect FAIL**
**Step 3: Implement `src/poller/poller.ts`**
```typescript
import type { NbEvent } from "../netbird/types.ts";
export interface EnrollmentDetection {
setupKeyName: string;
peerId: string;
peerHostname: string;
timestamp: string;
}
/**
* Filters enrollment events from the full event list.
* Returns enrollments for peers that enrolled using a known setup key
* and whose timestamp is after lastTimestamp (if provided).
*/
export function processEnrollmentEvents(
events: NbEvent[],
knownKeyNames: Set<string>,
lastTimestamp: string | null,
): EnrollmentDetection[] {
return events
.filter((e) => {
if (e.activity_code !== "peer.setupkey.add") return false;
if (lastTimestamp && e.timestamp <= lastTimestamp) return false;
if (!knownKeyNames.has(e.meta.setup_key)) {
console.log(
JSON.stringify({
msg: "unknown_enrollment",
setup_key: e.meta.setup_key,
peer_id: e.target_id,
}),
);
return false;
}
return true;
})
.map((e) => ({
setupKeyName: e.meta.setup_key,
peerId: e.target_id,
peerHostname: e.meta.name,
timestamp: e.timestamp,
}));
}
```
**Step 4: Run tests**
Run: `deno test src/poller/poller.test.ts` Expected: PASS
**Step 5: Commit**
```
feat: add enrollment event detection from NetBird audit events
```
---
## Task 7: Gitea API client (for state commits)
**Files:**
- Create: `src/gitea/client.ts`
- Create: `src/gitea/client.test.ts`
**Step 1: Write test in `src/gitea/client.test.ts`**
```typescript
import { assertEquals } from "jsr:@std/assert";
import { GiteaClient } from "./client.ts";
function mockFetch(
responses: Map<string, { status: number; body: unknown }>,
): typeof fetch {
return (input: string | URL | Request, init?: RequestInit) => {
const url = typeof input === "string"
? input
: input instanceof URL
? input.toString()
: input.url;
const method = init?.method ?? "GET";
const key = `${method} ${url}`;
const resp = responses.get(key);
if (!resp) throw new Error(`Unmocked: ${key}`);
return Promise.resolve(
new Response(JSON.stringify(resp.body), {
status: resp.status,
headers: { "Content-Type": "application/json" },
}),
);
};
}
Deno.test("GiteaClient.getFileContent fetches file with SHA", async () => {
const client = new GiteaClient(
"https://gitea.example.com",
"test-token",
"BlastPilot/netbird-gitops",
mockFetch(
new Map([
[
"GET https://gitea.example.com/api/v1/repos/BlastPilot/netbird-gitops/contents/netbird.json?ref=main",
{
status: 200,
body: {
content: btoa('{"groups":{}}'),
sha: "abc123",
},
},
],
]),
),
);
const result = await client.getFileContent("netbird.json", "main");
assertEquals(result.sha, "abc123");
assertEquals(result.content, '{"groups":{}}');
});
```
**Step 2: Run test — expect FAIL**
**Step 3: Implement `src/gitea/client.ts`**
```typescript
type FetchFn = typeof fetch;
export class GiteaClient {
constructor(
private readonly baseUrl: string,
private readonly token: string,
private readonly repo: string, // "owner/repo"
private readonly fetchFn: FetchFn = fetch,
) {}
private async request<T>(
method: string,
path: string,
body?: unknown,
): Promise<T> {
const url = `${this.baseUrl}/api/v1${path}`;
const headers: Record<string, string> = {
Authorization: `token ${this.token}`,
Accept: "application/json",
};
if (body !== undefined) {
headers["Content-Type"] = "application/json";
}
const resp = await this.fetchFn(url, {
method,
headers,
body: body !== undefined ? JSON.stringify(body) : undefined,
});
if (!resp.ok) {
const text = await resp.text().catch(() => "");
throw new Error(
`Gitea API ${method} ${path} returned ${resp.status}: ${text}`,
);
}
return resp.json() as Promise<T>;
}
/** Get file content and SHA for optimistic concurrency */
async getFileContent(
path: string,
ref: string,
): Promise<{ content: string; sha: string }> {
const data = await this.request<{ content: string; sha: string }>(
"GET",
`/repos/${this.repo}/contents/${path}?ref=${ref}`,
);
return {
content: atob(data.content),
sha: data.sha,
};
}
/** Update file with optimistic concurrency (SHA check) */
async updateFile(
path: string,
content: string,
sha: string,
message: string,
branch: string,
): Promise<void> {
await this.request(
"PUT",
`/repos/${this.repo}/contents/${path}`,
{
content: btoa(content),
sha,
message,
branch,
},
);
}
/** Post or update a PR comment */
async postIssueComment(
issueNumber: number,
body: string,
): Promise<void> {
await this.request(
"POST",
`/repos/${this.repo}/issues/${issueNumber}/comments`,
{ body },
);
}
}
```
**Step 4: Run tests**
Run: `deno test src/gitea/client.test.ts` Expected: PASS
**Step 5: Commit**
```
feat: add Gitea API client for state commits and PR comments
```
---
## Task 8: Poller background loop with Gitea commit
**Files:**
- Create: `src/poller/loop.ts`
- Create: `src/poller/state.ts`
This task wires the enrollment detection from Task 6 to the Gitea client from
Task 7, creating the full background polling loop.
**Step 1: Create `src/poller/state.ts`** — persists poll state to disk
```typescript
import { join } from "jsr:@std/path";
export interface PollerState {
lastEventTimestamp: string | null;
}
export async function loadPollerState(dataDir: string): Promise<PollerState> {
const path = join(dataDir, "poller-state.json");
try {
const text = await Deno.readTextFile(path);
return JSON.parse(text) as PollerState;
} catch {
return { lastEventTimestamp: null };
}
}
export async function savePollerState(
dataDir: string,
state: PollerState,
): Promise<void> {
const path = join(dataDir, "poller-state.json");
await Deno.mkdir(dataDir, { recursive: true });
await Deno.writeTextFile(path, JSON.stringify(state, null, 2));
}
```
**Step 2: Create `src/poller/loop.ts`** — the background loop
```typescript
import type { NetbirdClient } from "../netbird/client.ts";
import type { GiteaClient } from "../gitea/client.ts";
import type { Config } from "../config.ts";
import type { DesiredState } from "../state/schema.ts";
import { DesiredStateSchema } from "../state/schema.ts";
import { processEnrollmentEvents } from "./poller.ts";
import { loadPollerState, savePollerState } from "./state.ts";
export interface PollerContext {
config: Config;
netbird: NetbirdClient;
gitea: GiteaClient;
/** Set to true while a reconcile is in progress — poller defers */
reconcileInProgress: { value: boolean };
}
export async function pollOnce(ctx: PollerContext): Promise<void> {
if (ctx.reconcileInProgress.value) {
console.log(
JSON.stringify({ msg: "poll_deferred", reason: "reconcile_in_progress" }),
);
return;
}
const pollerState = await loadPollerState(ctx.config.dataDir);
// Fetch current desired state from git
let desired: DesiredState;
try {
const { content } = await ctx.gitea.getFileContent("netbird.json", "main");
desired = DesiredStateSchema.parse(JSON.parse(content));
} catch (err) {
console.log(JSON.stringify({
msg: "poll_error",
error: "failed to fetch desired state",
detail: err instanceof Error ? err.message : String(err),
}));
return;
}
const knownKeys = new Set(Object.keys(desired.setup_keys));
const unenrolledKeys = new Set(
Object.entries(desired.setup_keys)
.filter(([_, v]) => !v.enrolled)
.map(([k]) => k),
);
if (unenrolledKeys.size === 0) {
// Nothing to watch for
return;
}
// Fetch events
const events = await ctx.netbird.listEvents();
const enrollments = processEnrollmentEvents(
events,
unenrolledKeys,
pollerState.lastEventTimestamp,
);
if (enrollments.length === 0) return;
// Process each enrollment
for (const enrollment of enrollments) {
console.log(JSON.stringify({
msg: "enrollment_detected",
setup_key: enrollment.setupKeyName,
peer_id: enrollment.peerId,
}));
// Rename peer
try {
await ctx.netbird.updatePeer(enrollment.peerId, {
name: enrollment.setupKeyName,
ssh_enabled: false,
login_expiration_enabled: false,
inactivity_expiration_enabled: false,
});
console.log(JSON.stringify({
msg: "peer_renamed",
peer_id: enrollment.peerId,
new_name: enrollment.setupKeyName,
}));
} catch (err) {
console.log(JSON.stringify({
msg: "peer_rename_failed",
peer_id: enrollment.peerId,
error: err instanceof Error ? err.message : String(err),
}));
continue;
}
// Update enrolled status in git
try {
const { content, sha } = await ctx.gitea.getFileContent(
"netbird.json",
"main",
);
const state = JSON.parse(content);
if (state.setup_keys?.[enrollment.setupKeyName]) {
state.setup_keys[enrollment.setupKeyName].enrolled = true;
}
await ctx.gitea.updateFile(
"netbird.json",
JSON.stringify(state, null, 2),
sha,
`chore: mark ${enrollment.setupKeyName} as enrolled [automated]`,
"main",
);
console.log(JSON.stringify({
msg: "state_committed",
setup_key: enrollment.setupKeyName,
}));
} catch (err) {
console.log(JSON.stringify({
msg: "state_commit_failed",
setup_key: enrollment.setupKeyName,
error: err instanceof Error ? err.message : String(err),
}));
}
}
// Update last event timestamp
const latestTimestamp = enrollments[enrollments.length - 1].timestamp;
await savePollerState(ctx.config.dataDir, {
lastEventTimestamp: latestTimestamp,
});
}
/** Starts the polling loop. Returns an AbortController to stop it. */
export function startPollerLoop(ctx: PollerContext): AbortController {
const controller = new AbortController();
const intervalMs = ctx.config.pollIntervalSeconds * 1000;
const run = async () => {
while (!controller.signal.aborted) {
try {
await pollOnce(ctx);
} catch (err) {
console.log(JSON.stringify({
msg: "poll_error",
error: err instanceof Error ? err.message : String(err),
}));
}
await new Promise<void>((resolve) => {
const timer = setTimeout(resolve, intervalMs);
controller.signal.addEventListener("abort", () => {
clearTimeout(timer);
resolve();
}, { once: true });
});
}
};
run(); // Fire and forget — runs in background
return controller;
}
```
**Step 3: Commit**
```
feat: add poller background loop with Gitea state commit
```
---
## Task 9: HTTP server
**Files:**
- Modify: `src/main.ts`
- Create: `src/server.ts`
**Step 1: Create `src/server.ts`**
This is the HTTP server with three endpoints. It wires together all the
components.
```typescript
import type { Config } from "./config.ts";
import type { NetbirdClient } from "./netbird/client.ts";
import type { GiteaClient } from "./gitea/client.ts";
import { DesiredStateSchema, validateCrossReferences } from "./state/schema.ts";
import { fetchActualState } from "./state/actual.ts";
import { computeDiff } from "./reconcile/diff.ts";
import { executeOperations } from "./reconcile/executor.ts";
import { type PollerContext, pollOnce } from "./poller/loop.ts";
export interface ServerContext {
config: Config;
netbird: NetbirdClient;
gitea: GiteaClient;
reconcileInProgress: { value: boolean };
}
export function createHandler(ctx: ServerContext): Deno.ServeHandler {
return async (req: Request): Promise<Response> => {
const url = new URL(req.url);
// Health check — no auth
if (url.pathname === "/health" && req.method === "GET") {
return Response.json({ status: "ok" });
}
// Auth check for all other endpoints
const authHeader = req.headers.get("Authorization");
if (authHeader !== `Bearer ${ctx.config.reconcilerToken}`) {
return Response.json({ error: "unauthorized" }, { status: 401 });
}
if (url.pathname === "/reconcile" && req.method === "POST") {
return handleReconcile(req, url, ctx);
}
if (url.pathname === "/sync-events" && req.method === "POST") {
return handleSyncEvents(ctx);
}
return Response.json({ error: "not found" }, { status: 404 });
};
}
async function handleReconcile(
req: Request,
url: URL,
ctx: ServerContext,
): Promise<Response> {
const dryRun = url.searchParams.get("dry_run") === "true";
// Parse and validate desired state
let body: unknown;
try {
body = await req.json();
} catch {
return Response.json({ error: "invalid JSON body" }, { status: 400 });
}
const parseResult = DesiredStateSchema.safeParse(body);
if (!parseResult.success) {
return Response.json(
{ error: "invalid state schema", details: parseResult.error.issues },
{ status: 400 },
);
}
const desired = parseResult.data;
const crossRefErrors = validateCrossReferences(desired);
if (crossRefErrors.length > 0) {
return Response.json(
{ error: "cross-reference validation failed", details: crossRefErrors },
{ status: 400 },
);
}
// Fetch actual state
ctx.reconcileInProgress.value = true;
try {
const actual = await fetchActualState(ctx.netbird);
const operations = computeDiff(desired, actual);
if (dryRun) {
return Response.json({
status: "planned",
operations: operations.map((op) => ({
type: op.type,
name: op.name,
})),
summary: summarize(
operations.map((op) => ({ ...op, status: "pending" })),
),
});
}
if (operations.length === 0) {
return Response.json({
status: "applied",
operations: [],
created_keys: {},
summary: { created: 0, updated: 0, deleted: 0, failed: 0 },
});
}
const { results, createdKeys } = await executeOperations(
operations,
ctx.netbird,
actual,
);
const failed = results.some((r) => r.status === "failed");
return Response.json({
status: failed ? "error" : "applied",
operations: results.map((r) => ({
type: r.type,
name: r.name,
status: r.status,
...(r.error ? { error: r.error } : {}),
})),
created_keys: Object.fromEntries(createdKeys),
summary: summarize(results),
});
} finally {
ctx.reconcileInProgress.value = false;
}
}
async function handleSyncEvents(ctx: ServerContext): Promise<Response> {
const pollerCtx: PollerContext = {
config: ctx.config,
netbird: ctx.netbird,
gitea: ctx.gitea,
reconcileInProgress: ctx.reconcileInProgress,
};
// Force reconcileInProgress to false so pollOnce runs
const saved = ctx.reconcileInProgress.value;
ctx.reconcileInProgress.value = false;
try {
await pollOnce(pollerCtx);
} finally {
ctx.reconcileInProgress.value = saved;
}
return Response.json({ status: "synced" });
}
function summarize(
results: Array<{ type: string; status: string }>,
): { created: number; updated: number; deleted: number; failed: number } {
let created = 0, updated = 0, deleted = 0, failed = 0;
for (const r of results) {
if (r.status === "failed") {
failed++;
continue;
}
if (r.type.startsWith("create_")) created++;
else if (r.type.startsWith("update_") || r.type === "rename_peer") {
updated++;
} else if (r.type.startsWith("delete_")) deleted++;
}
return { created, updated, deleted, failed };
}
```
**Step 2: Update `src/main.ts` to start the server and poller**
```typescript
import { loadConfig } from "./config.ts";
import { NetbirdClient } from "./netbird/client.ts";
import { GiteaClient } from "./gitea/client.ts";
import { createHandler } from "./server.ts";
import { startPollerLoop } from "./poller/loop.ts";
const config = loadConfig();
const netbird = new NetbirdClient(config.netbirdApiUrl, config.netbirdApiToken);
const gitea = new GiteaClient(
config.giteaUrl,
config.giteaToken,
config.giteaRepo,
);
const reconcileInProgress = { value: false };
// Start background poller
const pollerAbort = startPollerLoop({
config,
netbird,
gitea,
reconcileInProgress,
});
// Start HTTP server
const handler = createHandler({
config,
netbird,
gitea,
reconcileInProgress,
});
console.log(JSON.stringify({ msg: "starting", port: config.port }));
Deno.serve({ port: config.port, handler });
// Graceful shutdown
Deno.addSignalListener("SIGTERM", () => {
console.log(JSON.stringify({ msg: "shutting_down" }));
pollerAbort.abort();
Deno.exit(0);
});
```
**Step 3: Verify compilation**
Run: `deno check src/main.ts` Expected: no errors
**Step 4: Commit**
```
feat: add HTTP server with /reconcile, /sync-events, /health endpoints
```
---
## Task 10: CI workflow files
**Files:**
- Create: `.gitea/workflows/dry-run.yml`
- Create: `.gitea/workflows/reconcile.yml`
- Create: `.gitea/workflows/release.yml`
**Step 1: Create `.gitea/workflows/dry-run.yml`**
```yaml
name: Dry Run
on:
pull_request:
paths:
- "netbird.json"
jobs:
dry-run:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run dry-run reconcile
id: plan
run: |
RESPONSE=$(curl -sf \
-X POST \
-H "Authorization: Bearer ${{ secrets.RECONCILER_TOKEN }}" \
-H "Content-Type: application/json" \
-d @netbird.json \
"${{ secrets.RECONCILER_URL }}/reconcile?dry_run=true")
echo "response<<EOF" >> "$GITHUB_OUTPUT"
echo "$RESPONSE" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
- name: Format plan as markdown
id: format
run: |
cat <<'SCRIPT' > format.py
import json, sys
data = json.loads(sys.stdin.read())
ops = data.get("operations", [])
summary = data.get("summary", {})
lines = ["## NetBird Reconciliation Plan\n"]
if not ops:
lines.append("No changes detected.\n")
else:
lines.append("| Operation | Name |")
lines.append("|-----------|------|")
for op in ops:
lines.append(f"| `{op['type']}` | {op['name']} |")
lines.append("")
s = summary
lines.append(f"**Summary:** {s.get('created',0)} create, {s.get('updated',0)} update, {s.get('deleted',0)} delete")
print("\n".join(lines))
SCRIPT
COMMENT=$(echo '${{ steps.plan.outputs.response }}' | python3 format.py)
echo "comment<<EOF" >> "$GITHUB_OUTPUT"
echo "$COMMENT" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
- name: Post PR comment
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
run: |
curl -sf \
-X POST \
-H "Authorization: token ${GITEA_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"body\": $(echo '${{ steps.format.outputs.comment }}' | jq -Rs .)}" \
"${{ secrets.GITEA_URL }}/api/v1/repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments"
```
**Step 2: Create `.gitea/workflows/reconcile.yml`**
```yaml
name: Reconcile
on:
push:
branches:
- main
paths:
- "netbird.json"
jobs:
reconcile:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Sync events
run: |
curl -sf \
-X POST \
-H "Authorization: Bearer ${{ secrets.RECONCILER_TOKEN }}" \
"${{ secrets.RECONCILER_URL }}/sync-events"
- name: Pull latest (poller may have committed)
run: git pull --rebase
- name: Apply reconcile
id: reconcile
run: |
RESPONSE=$(curl -sf \
-X POST \
-H "Authorization: Bearer ${{ secrets.RECONCILER_TOKEN }}" \
-H "Content-Type: application/json" \
-d @netbird.json \
"${{ secrets.RECONCILER_URL }}/reconcile")
echo "response<<EOF" >> "$GITHUB_OUTPUT"
echo "$RESPONSE" >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
STATUS=$(echo "$RESPONSE" | jq -r '.status')
if [ "$STATUS" = "error" ]; then
echo "Reconcile failed"
echo "$RESPONSE" | jq .
exit 1
fi
- name: Encrypt and upload setup keys
if: success()
run: |
KEYS=$(echo '${{ steps.reconcile.outputs.response }}' | jq -r '.created_keys // empty')
if [ -n "$KEYS" ] && [ "$KEYS" != "{}" ] && [ "$KEYS" != "null" ]; then
echo "$KEYS" | age -r "${{ secrets.AGE_PUBLIC_KEY }}" -o setup-keys.age
echo "Setup keys encrypted to setup-keys.age"
else
echo "No new keys created"
exit 0
fi
- name: Upload artifact
if: success()
uses: actions/upload-artifact@v4
with:
name: setup-keys
path: setup-keys.age
if-no-files-found: ignore
```
**Step 3: Create `.gitea/workflows/release.yml`**
```yaml
name: Release
on:
push:
tags:
- "v*"
jobs:
build:
runs-on: ubuntu-latest
container:
image: denoland/deno:debian
steps:
- uses: actions/checkout@v4
- name: Compile
run: deno compile --allow-net --allow-read --allow-write --allow-env --output reconciler src/main.ts
- name: Build Docker image
run: |
docker build -t ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:${{ github.ref_name }} .
docker tag ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:${{ github.ref_name }} \
${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:latest
- name: Push Docker image
run: |
echo "${{ secrets.PACKAGE_TOKEN }}" | docker login ${{ secrets.GITEA_URL }} -u achilles-ci-bot --password-stdin
docker push ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:${{ github.ref_name }}
docker push ${{ secrets.GITEA_URL }}/blastpilot/netbird-reconciler:latest
```
**Step 4: Commit**
```
feat: add Gitea Actions CI workflows for dry-run, reconcile, and release
```
---
## Task 11: Seed `netbird.json` with initial state
**Files:**
- Create: `netbird.json`
**Step 1: Create the initial state file**
This should reflect the current BlastPilot NetBird configuration. Start minimal
— populate with actual groups/policies after deploying the reconciler and
importing existing state.
```json
{
"groups": {},
"setup_keys": {},
"policies": {},
"routes": {},
"dns": {
"nameserver_groups": {}
}
}
```
**Step 2: Commit**
```
feat: add empty netbird.json state file
```
---
## Task 12: Docker Compose deployment config
**Files:**
- Create: `deploy/docker-compose.yml`
- Create: `deploy/.env.example`
**Step 1: Create `deploy/docker-compose.yml`**
```yaml
services:
netbird-reconciler:
image: gitea.internal/blastpilot/netbird-reconciler:latest
restart: unless-stopped
env_file: .env
volumes:
- reconciler-data:/data
ports:
- "127.0.0.1:8080:8080"
healthcheck:
test: ["CMD", "wget", "-q", "--spider", "http://localhost:8080/health"]
interval: 30s
timeout: 5s
retries: 3
labels:
- "traefik.enable=true"
- "traefik.http.routers.reconciler.rule=Host(`reconciler.internal`)"
- "traefik.http.services.reconciler.loadbalancer.server.port=8080"
volumes:
reconciler-data:
```
**Step 2: Create `deploy/.env.example`**
```
NETBIRD_API_URL=https://netbird.example.com/api
NETBIRD_API_TOKEN=
GITEA_URL=https://gitea.example.com
GITEA_TOKEN=
GITEA_REPO=BlastPilot/netbird-gitops
RECONCILER_TOKEN=
POLL_INTERVAL_SECONDS=30
PORT=8080
DATA_DIR=/data
```
**Step 3: Commit**
```
feat: add Docker Compose deployment config
```
---
## Task 13: Integration test with mock NetBird server
**Files:**
- Create: `src/integration.test.ts`
Write an end-to-end test that starts the HTTP server, posts a reconcile request
with a known desired state against a mock NetBird API, and verifies the correct
API calls were made.
**Step 1: Write integration test**
```typescript
import { assertEquals } from "jsr:@std/assert";
import { createHandler } from "./server.ts";
import { NetbirdClient } from "./netbird/client.ts";
import { GiteaClient } from "./gitea/client.ts";
/** Tracks all API calls made */
interface ApiCall {
method: string;
path: string;
body?: unknown;
}
function createMockNetbird(): { client: NetbirdClient; calls: ApiCall[] } {
const calls: ApiCall[] = [];
const responses = new Map<string, { status: number; body: unknown }>([
// List endpoints return empty by default
["GET /groups", { status: 200, body: [] }],
["GET /setup-keys", { status: 200, body: [] }],
["GET /peers", { status: 200, body: [] }],
["GET /policies", { status: 200, body: [] }],
["GET /routes", { status: 200, body: [] }],
["GET /dns/nameservers", { status: 200, body: [] }],
["GET /events/audit", { status: 200, body: [] }],
]);
const mockFetch: typeof fetch = async (input, init) => {
const url = typeof input === "string"
? input
: input instanceof URL
? input.toString()
: input.url;
const path = url.replace("https://nb.test/api", "");
const method = init?.method ?? "GET";
calls.push({
method,
path,
body: init?.body ? JSON.parse(init.body as string) : undefined,
});
// Handle create operations
if (method === "POST" && path === "/groups") {
const body = JSON.parse(init?.body as string);
return new Response(
JSON.stringify({
id: `g-${body.name}`,
name: body.name,
peers_count: 0,
peers: [],
issued: "api",
}),
{ status: 201, headers: { "Content-Type": "application/json" } },
);
}
if (method === "POST" && path === "/setup-keys") {
const body = JSON.parse(init?.body as string);
return new Response(
JSON.stringify({
id: 1,
name: body.name,
key: "TEST-KEY-12345",
state: "valid",
}),
{ status: 201, headers: { "Content-Type": "application/json" } },
);
}
if (method === "POST" && path === "/policies") {
const body = JSON.parse(init?.body as string);
return new Response(
JSON.stringify({ id: `p-${body.name}`, ...body }),
{ status: 201, headers: { "Content-Type": "application/json" } },
);
}
const key = `${method} ${path}`;
const resp = responses.get(key);
if (!resp) {
return new Response(JSON.stringify({ message: "not found" }), {
status: 404,
});
}
return new Response(JSON.stringify(resp.body), {
status: resp.status,
headers: { "Content-Type": "application/json" },
});
};
const client = new NetbirdClient("https://nb.test/api", "test", mockFetch);
return { client, calls };
}
Deno.test("POST /reconcile dry_run returns planned operations", async () => {
const { client } = createMockNetbird();
const handler = createHandler({
config: { reconcilerToken: "secret" } as never,
netbird: client,
gitea: {} as never,
reconcileInProgress: { value: false },
});
const body = JSON.stringify({
groups: { pilots: { peers: [] } },
setup_keys: {
"Pilot-hawk-72": {
type: "one-off",
expires_in: 604800,
usage_limit: 1,
auto_groups: ["pilots"],
enrolled: false,
},
},
policies: {},
routes: {},
dns: { nameserver_groups: {} },
});
const resp = await handler(
new Request("http://localhost/reconcile?dry_run=true", {
method: "POST",
headers: {
"Authorization": "Bearer secret",
"Content-Type": "application/json",
},
body,
}),
);
assertEquals(resp.status, 200);
const data = await resp.json();
assertEquals(data.status, "planned");
const types = data.operations.map((o: { type: string }) => o.type);
assertEquals(types.includes("create_group"), true);
assertEquals(types.includes("create_setup_key"), true);
});
Deno.test("POST /reconcile apply creates resources and returns keys", async () => {
const { client, calls } = createMockNetbird();
const handler = createHandler({
config: { reconcilerToken: "secret" } as never,
netbird: client,
gitea: {} as never,
reconcileInProgress: { value: false },
});
const body = JSON.stringify({
groups: { pilots: { peers: [] } },
setup_keys: {
"Pilot-hawk-72": {
type: "one-off",
expires_in: 604800,
usage_limit: 1,
auto_groups: ["pilots"],
enrolled: false,
},
},
policies: {},
routes: {},
dns: { nameserver_groups: {} },
});
const resp = await handler(
new Request("http://localhost/reconcile", {
method: "POST",
headers: {
"Authorization": "Bearer secret",
"Content-Type": "application/json",
},
body,
}),
);
assertEquals(resp.status, 200);
const data = await resp.json();
assertEquals(data.status, "applied");
assertEquals(data.created_keys["Pilot-hawk-72"], "TEST-KEY-12345");
// Verify API calls were made
const postCalls = calls.filter((c) => c.method === "POST");
assertEquals(postCalls.some((c) => c.path === "/groups"), true);
assertEquals(postCalls.some((c) => c.path === "/setup-keys"), true);
});
Deno.test("POST /reconcile rejects unauthorized requests", async () => {
const handler = createHandler({
config: { reconcilerToken: "secret" } as never,
netbird: {} as never,
gitea: {} as never,
reconcileInProgress: { value: false },
});
const resp = await handler(
new Request("http://localhost/reconcile", {
method: "POST",
headers: { "Authorization": "Bearer wrong" },
}),
);
assertEquals(resp.status, 401);
});
```
**Step 2: Run tests**
Run: `deno test src/integration.test.ts` Expected: PASS
**Step 3: Commit**
```
test: add integration tests for reconcile HTTP endpoint
```
---
## Task 14: Update blastpilot-public enrollment pipeline
**Files:**
- Modify: `../blastpilot-public/api/src/services/enrollment-pipeline.ts`
- Modify: `../blastpilot-public/api/src/services/netbird.ts`
This task modifies the enrollment pipeline to write to `netbird.json` in the
`netbird-gitops` repo instead of creating `peers/enrollment-{N}.json` files.
**Step 1: Update `handleApproval()` in enrollment-pipeline.ts**
Change from creating a standalone peer JSON file to modifying `netbird.json`:
- Fetch current `netbird.json` from `netbird-gitops` repo via Gitea API
- Add setup key entries for GS and Pilot
- Add peer references to appropriate groups
- Create PR with the modified `netbird.json`
**Step 2: Remove direct NetBird API calls from `handlePRMerge()`**
The reconciler now handles key creation. `handlePRMerge()` should be simplified
or removed (key delivery is manual for now).
**Step 3: Update tests**
**Step 4: Commit**
```
refactor: update enrollment pipeline to write to netbird.json instead of peer files
```
---
## Task 15: Deploy and test
**Step 1: Create Gitea repo `BlastPilot/netbird-gitops`**
Via Gitea UI or API. Push all code.
**Step 2: Generate tokens**
- Generate `RECONCILER_TOKEN` (random 32-byte hex)
- Ensure `NETBIRD_API_TOKEN` has management API access
- Ensure `GITEA_TOKEN` has repo write access
- Generate `AGE_PUBLIC_KEY` / private key pair
**Step 3: Deploy Docker Compose on VPS**
```bash
cd deploy
cp .env.example .env
# Fill in .env values
docker compose up -d
docker compose logs -f
```
**Step 4: Test health endpoint**
```bash
curl http://localhost:8080/health
# Expected: {"status":"ok"}
```
**Step 5: Test dry-run**
```bash
curl -X POST \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d @netbird.json \
http://localhost:8080/reconcile?dry_run=true
```
**Step 6: Test with a real enrollment**
- Add a test setup key + peer to `netbird.json`
- Push, verify CI creates key
- Use the key to enroll a device
- Verify poller detects enrollment, renames peer, commits state
**Step 7: Commit any fixes**
```
fix: address deployment issues found during testing
```
Reference in New Issue View Git Blame Copy Permalink