Codebase Organization Plan
One plan for the backend: seven library layers, twelve domains, two HTTP servers. Six phases, each with an exit measured by a command.
Outcome
| Measure | Today | Target | Reached in |
|---|---|---|---|
| Layering violations | 146 | 0 | Phase 4 |
| Longest dependency chain | 36 libraries | 15 | Phase 4 |
| Libraries affected by a typical change | 13 | 3 | Phase 4 |
| Libraries affected by the worst change above core | 100 | 42 | Phase 4 |
| HTTP server processes | 8 | 2 | Phase 2 |
| Heap for those servers at boot | 1,810 MB | 685 MB | Phase 2 |
| Server bundle size | 85.5 MB | about 70 MB | Phase 5 |
| Libraries in a domain folder | 41 of 192 | all | Phase 6 |
The heap figures are one copy of each process. Every replica of a merged server carries its full heap.
Problems
- No layering. Domain libraries import each other sideways.
media → requests → chats → user → phone → commsis one chain of six. A change toentityaffects 96 libraries. - Eight HTTP servers that load the same code. api-server alone carries 142 libraries. The other seven add 27.
- Dead weight loaded at boot. Eval fixtures, the whole Google API surface and eagerly built schemas are in every server's heap.
- ES5 output. Our code ships as ES5 because no compile target is set. Two file-level import cycles prevent changing it.
- Flat folders. 151 libraries sit directly under
libs/. The folder tree says nothing about what may depend on what.
Target
Layers
A library may import from any lower layer. Within its own layer it may import only libraries of its own domain. Imports the compiler erases are free.
| Layer | Libraries | Holds |
|---|---|---|
| feature | 26 | Orchestration across domains: Command Center, front desk, website backend |
| domain | 64 | One business area each |
| platform | 17 | AI runtime and messaging |
| core | 14 | Identity: entity, auth, user, location |
| integration | 37 | One third-party client each |
| contracts | 6, growing | Types, constants, pure functions, port and queue definitions |
| foundation | 28 | config, monitoring, drizzle-orm, bull, llm |
Domains
| Layer | Domain | Libraries |
|---|---|---|
| core | identity | 14 |
| platform | ai | 6 |
| platform | comms | 11 |
| domain | scheduling | 13 |
| domain | gbp | 13 |
| domain | local-seo | 9 |
| domain | website | 8 |
| domain | content | 7 |
| domain | brain | 6 |
| domain | clients | 3 |
| domain | scraping | 3 |
| domain | billing | 2 |
| feature | command-center | 10 |
| feature | frontdesk | 6 |
| feature | backoffice | 5 |
| feature | platforms, website-backend, business-profile, evals | 5 |
The library-by-library assignment is in Library Layers and in tools/lib-layers/layers.json.
Three ways across a boundary
A library that needs something from another domain uses one of these. It never imports the other domain's library.
<domain>-contracts. Anyone may import it.Folders
libs/
foundation/<lib>
contracts/<domain>
integration/<vendor>
core/identity/<lib>
platform/<domain>/<lib>
domain/<domain>/<lib>
feature/<group>/<lib>
Import aliases keep their current names during the move. @mononest/scheduling still resolves. Only tsconfig.base.json paths change.
Servers
| Process | Contains | URLs |
|---|---|---|
| unified-server | api, tasks, scp, auth, internal | unchanged. api at the root, the others under their existing prefixes |
| unified-public-server | public, leads | unchanged. Both at the root |
| workers | sync, website, content, platform | unchanged |
| relay-server | itself | unchanged. Not evaluated |
links-server is retired. public-server already serves all five of its paths.
Phases
Phases 1, 2 and 3 do not depend on each other and can run in parallel after Phase 0. Phase 4 follows 3. Phase 5 can start any time. Phase 6 is last.
Phase 0 — Guardrails and decisions
Nothing else is safe to start until new violations are blocked.
Review the layer and domain assignment
Each owner confirms their rows. A reassigned library changes which edges count as violations. Re-run pnpm layers:plan after any edit.
Settle four decisions
Port module convention. Whether workers join a merged server. Whether the cached redirect moves into public-server. Whether public-facing hosts keep separate load balancer targets.
Merge the tooling and the two worked fixes
Baseline at 146. From this merge a pull request cannot add a violation.
Make the layer check required
Fix the pre-existing type errors that block library builds
nx build for any library that depends on bull fails today. Every later phase needs these builds green.
Exit: pnpm check:layers is a required check and passes at 146.
Phase 1 — Remove dead weight
Each item is one pull request and changes no behaviour.
| Change | Where | Saves per server |
|---|---|---|
Stop importing agent-eval in tasks-server and scp-server | two AppModules | about 14 MB heap |
Build the agents zod schemas on first use | libs/agents | about 13 MB heap |
Replace googleapis with per-API packages | 8 files in 6 libraries | about 6 MB heap, 10 MB bundle |
| Move the three contracts-only symbols | Wave 1 of the layering work list | 3 violations |
agent-eval holds eval scenario fixtures. Confirm why the two servers import it before removing it. libs/auth is among the most imported libraries, so the googleapis pull request lists its dependents.
Exit: a booted server's heap is at least 30 MB lower. Baseline at 143.
Phase 2 — Two servers
The merge is built and boots. Routes, status codes and response shapes match the standalone servers for unauthenticated GET requests. These steps take it from evaluation to production.
Move each server's root module into a library
The unified apps import other apps by relative path, which @nx/enforce-module-boundaries rejects. Each AppModule and its controllers move to a library that the standalone app and the unified app both import.
Unit-test the mount helper
It scopes each server's global guards, interceptors and filters to that server's routes, and re-mounts controller modules that several servers share. It has no tests.
Extend the equivalence check to authenticated requests and writes
The current check sends no credentials and only GET. Add a token per server and a set of safe writes against a seeded entity. Compare standalone against merged.
Exercise what the check cannot reach
Three servers install a WebSocket adapter. Confirm each gateway connects and each cron and processor is registered once.
Move the cached redirect into public-server
links-server caches the link in Redis and sends Cache-Control: no-store. public-server's handler for the same path does neither.
Infrastructure
Two new services. Each hostname that pointed at a standalone server points at the merged one. Environment variables are the union of the merged servers'.
Roll out one server at a time
Move one hostname or prefix to the merged process, watch error rate and latency, then the next. Keep the standalone service running until its traffic is zero.
One process means one event loop and one heap limit. Size the task for the sum of the merged servers' load, not for one of them.
Exit: two services carry all HTTP traffic and the eight standalone services are deleted.
Phase 3 — Layering, mechanical fixes
51 edges. Each is a small, repeated change.
| Wave | Edges | Fix | Baseline after |
|---|---|---|---|
| 1 | 3 | Move symbols to a contracts library | 143 |
| 2 | 6 | Replace a side-effect call with a Bull job | 137 |
| 3 | 42 | Add a one-method port | 95 |
The procedure for each fix and every edge by name are in Library Layers. Two fixes are already done and serve as the template: scheduling → booking-agent for contracts and gateway → scheduling for a port.
A queue fix changes delivery from inline to eventual. Confirm with the domain owner that the caller does not rely on the effect having happened. The processor must be idempotent.
Exit: baseline at 95.
Phase 4 — Layering, owning libraries
95 edges. Work one owning library at a time.
Multi-method ports
List what other layers call on the library. Group the methods into ports of two to six by purpose. Move shared types to contracts. Convert side-effect calls to jobs. Repoint each importer in its own pull request. Start with media and gbp-location, ten importers each.
Manual edges
mcp → auth and mcp → agents are a foundation library importing upward. Either the symbols move down or mcp moves up a layer. frontdesk-agent-tools → win reads 17 methods and is a split, not a port.
Split the widest libraries
| Library | Size | Methods read by other layers | Split into |
|---|---|---|---|
win | 85k lines | 28 | call history and analytics, follow-ups, agent context, quiet hours |
gbp-location | 30k lines | 21 | location core, insights, posts, activity |
entity | 14k lines | 20 | entity reads, preferences, location data |
media | 19k lines | 9 | media store, stock media bank, tagging |
The split lines come from the service names other layers call. Owners have not reviewed them.
scheduling is the largest library at 137k lines, but other domains read it through two methods. Splitting it along its existing sub-packs is worthwhile and is not on this plan's critical path.
Exit: baseline at 0. Longest chain 15. A typical change affects 3 libraries.
Phase 5 — Modern build target
Find every file-level import cycle
Build with an ES2022 target and load each bundle. Each "Cannot access before initialization" error names one cycle. ChatbotService and PostService are the two found so far. ES5 output hides them because it hoists classes.
Break each cycle
Set the compile target
Exit: every server bundle loads under ES2022 and is 16–19% smaller.
Phase 6 — Folders and lock
Move libraries into layer and domain folders
git mv the library, update its project.json paths and its tsconfig.base.json entry. The import alias does not change, so no source file changes.
Delete the baseline
The check then allows no violation.
Mirror the manifest into Nx tags
The editor then reports a violation as it is typed.
Decide on type-only imports across domains
They cost nothing at runtime and still couple the code.
Exit: every library sits under libs/<layer>/… and the check runs with no baseline.
Done so far
| Item | State |
|---|---|
| Layer and domain manifest for all 192 libraries | Built. Not reviewed by owners |
Layer check with ratchet, CI workflow, pnpm check:layers | Built. Fails on a new violation and on a stale baseline entry |
| Planner that classifies every violation and simulates the result | Built |
Contracts fix: scheduling → booking-agent | Done. 17 unit tests pass |
Port fix: gateway → scheduling | Done. api-server boots. 372 of 372 probed routes unchanged |
unified-server and unified-public-server | Built. Boot. 3,367 and 158 routes match the standalone servers |
| Bundle, memory, heap-profile and boot-probe tools | Built |
Nothing is committed. All of it is in one worktree.
Rules for every pull request
- One edge, one owning library, or one domain folder per pull request.
- Run
pnpm check:layers. When a violation is fixed, runnode tools/lib-layers/check.js --updateand commit the smaller baseline. - A library with a domain pack gets its pack updated in the same pull request.
- Moving a symbol and re-exporting it does not change a contract. Changing delivery from inline to queued can. That needs an ADR when it touches a "must never break" entry.
- New pure functions and new processors get mirrored unit tests.
- Boot the affected server before and after and compare with
boot.js --probe=true.
Measuring progress
$ pnpm check:layers $ pnpm layers:plan $ node tools/lib-footprint/graph.js $ node tools/lib-footprint/boot.js --main=dist/apps/unified-server/main.js --name=unified-server --port=8100 --env=.env --probe=true
check:layers gives the violation count. layers:plan gives chain length and blast radius. graph.js gives libraries carried per server. boot.js gives booted heap and the route comparison.
Risks
| Risk | Where | Mitigation |
|---|---|---|
| A merged server applies one server's guard to another's routes | Phase 2 | The mount helper scopes them. Step 3 tests it with real credentials |
| A background job starves API requests in a shared process | Phase 2 | Workers stay separate. Size the task for combined load |
| A queued side effect was relied on as immediate | Phases 3 and 4 | Owner sign-off per edge. Idempotent processors |
| A server mounts a consumer without the owner's ports module | Phases 3 and 4 | Fails at boot, not at request time |
| The layer assignment is wrong for a library | Phase 0 | Owner review before the check becomes required |
Local database is three migrations behind develop | Any local boot | Migration 0617 fails because its table already exists. Unrelated to this plan. Needs a manual fix |
Open decisions
- Is the layer and domain assignment right?
- Are port modules global, or imported by each consumer module?
- Do workers stay separate processes? The plan assumes yes.
- Does the cached short-link redirect move into public-server, or does links-server stay?
- Is relay-server merged? It was not evaluated because its native dependency does not load on macOS.
- Who owns each phase?
Evidence
- Library Layers: the layer model, every violating edge, the fix procedures and the worked examples.
- Server Footprint: bundle and memory measurements, heap composition, and the merged-server equivalence check.