Nacorid
c96447e3e0
Extract ActivityPub outbox walk into internal/activitypub/outbox.go with webfinger/actor resolution, Create/Note filtering and IRI dereferencing. Fix searchStatus returning wrong post, nil-account panic in SearchActors, @local handle resolution, empty Author via noteToInternalPost, protocol- relative URL handling, pagination loops, limit clamping (200) and cacheErr masking. Add per-host token-bucket throttle (1/s, cross-host concurrency) in throttle.go and background GTS warming via search?resolve for outbox- fetched posts without following remotes. Align AGENTS.md. Co-Authored-By: internal-model
97 lines
6.2 KiB
Markdown
97 lines
6.2 KiB
Markdown
# AGENTS.md — naco-api
|
|
|
|
## Quick commands
|
|
|
|
```bash
|
|
# Verify (no CI — match these locally)
|
|
gofmt -l . # fail if any output
|
|
go build -v ./...
|
|
go test -v -cover ./...
|
|
|
|
# Single package / single test
|
|
go test -v -cover ./internal/server/... -run TestGetPost
|
|
go test -v -cover ./internal/bluesky/...
|
|
|
|
# Integration tests (requires Postgres + pgvector; never use prod DB)
|
|
docker compose -f docker-compose.test.yml up -d # postgres:5432 + ollama:11434
|
|
TEST_DATABASE_URL="postgres://postgres:postgres@localhost:5432/mcpserver?sslmode=disable" \
|
|
go test -v -cover ./internal/store/...
|
|
```
|
|
|
|
## What this is
|
|
|
|
Go MCP server (13 tools) for **Bluesky / Nostr / ActivityPub / Twitter**. Every tool is gated by **x402** (HTTP 402 crypto payments, EVM `base`); settlement → Postgres, optional **Matrix** webhook.
|
|
|
|
Tools: `getPost`, `getLatestPosts`, `getPostsInTime`, `getFeed` (free), `getActor`, `searchActors`, `getStyleProfile`, `findSimilarUsers`, `getEngagementMetrics`, `getPostingPatterns`, `getTopicModeling`, `getSentimentTrends`, `getEngagementDrivers`.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
cmd/server/main.go entrypoint — wires config, DB, all clients, readyz sidecar
|
|
internal/server/ MCP handlers + x402 + analytics (~1800 lines in server.go — start there)
|
|
server.go tool dispatch, settlement, prewarm, cache
|
|
analytics.go analytics compute + coverage/cache logic
|
|
backends.go platform/backend interfaces (fakes for tests)
|
|
storefake_test.go in-memory Store fake (DB-free analytics tests)
|
|
types.go request/response types, platform enum
|
|
auth.go / webhook.go / cache*.go / *Middleware.go / networkInfo.go
|
|
*_test.go split suites: mcp_test.go, analytics_test.go, validation_test.go, etc.
|
|
internal/activitypub/ AP client; throttle.go per-host token bucket (1/s) for background GTS warming
|
|
internal/{bluesky,nostr,twitter,ollama,config,utils,actor,x402}
|
|
sql/schema/ 001..007 goose migrations
|
|
sql/queries/ sqlc query files → internal/store/
|
|
config/models.env single source of truth for EMBEDDING_MODEL / STYLE_MODEL
|
|
testClient/rest/ manual live REST endpoint client (not part of go test)
|
|
testClient/mcp/ manual live MCP endpoint client (not part of go test)
|
|
```
|
|
|
|
## Code generation
|
|
|
|
`sqlc` generates `internal/store/*.go` from `sql/queries/*.sql` + `sql/schema/`:
|
|
```bash
|
|
sqlc generate # after editing any file in sql/
|
|
```
|
|
Never hand-edit `// Code generated by sqlc. DO NOT EDIT.` files.
|
|
|
|
## Migrations
|
|
|
|
Goose-style (`-- +goose up` / `-- +goose down`) in `sql/schema/` — 7 files `001_transactions.sql` … `007_add_analytics_coverage.sql`. Applied at boot when `MIGRATE_ON_START=true` (default `true`). New files must be numbered sequentially (`008_*.sql`).
|
|
|
|
## Testing patterns
|
|
|
|
- Interface fakes in `internal/server/backends.go` + `storefake_test.go` — no live clients/DB needed.
|
|
- Call tools via `callTool(s, name, args)` through the MCP handler (see `mcp_test.go:302`).
|
|
- Integration `internal/store/integration_test.go` skipped unless `TEST_DATABASE_URL` set — use `docker-compose.test.yml` (pgvector:pg16). Point it at a throwaway DB.
|
|
- `internal/server` tests split by concern (`analytics_test.go`, `searchactors_test.go`, `styleprofile_test.go`, `validation_test.go`, `coverage_test.go`, `twitter_analytics_test.go`); `mcp_test.go` holds cross-cutting dispatch/cache.
|
|
|
|
## Config
|
|
|
|
Loaded in order: `config/models.env` → `.env` (gitignored, via `godotenv`) → env. `getEnv(k, fallback)` fatals if fallback `""` and var unset.
|
|
|
|
| Variable | Default | Notes |
|
|
|---|---|---|
|
|
| `GOOSE_DBSTRING` | `postgres:///mcpserver` | Server DB (not tests) |
|
|
| `TEST_DATABASE_URL` | — | **Tests only**, separate DB |
|
|
| `SERVER_ADDRESS` / `SERVER_PORT` | `127.0.0.1` / `8080` | Unix socket via x402 `X402Server`; TCP sidecar on `READYZ_PORT` |
|
|
| `READYZ_PORT` | `9090` | Sidecar serves `/readyz` (DB+Ollama health) + `/metrics` |
|
|
| `MIGRATE_ON_START` | `true` | Apply `sql/schema` on boot |
|
|
| `OLLAMA_HOST` | `http://localhost:11434` | Models from `config/models.env` |
|
|
| `MATRIX_WEBHOOK_URL` | — | If set, payment notifications |
|
|
| `X402_FACILITATOR_URL` | `https://facilitator.x402.rs` | |
|
|
| `ACTIVE_NETWORKS` | `base` | Comma eip155 IDs |
|
|
| `ENRICH_CROSS_PLATFORM` | `true` | |
|
|
|
|
`findProjectRoot()` walks up to `go.mod` to locate `config/models.env`; tests with `os.Chdir` can break it.
|
|
|
|
## Gotchas
|
|
|
|
- `bin/` gitignored — `go build -o bin/mcpserver ./cmd/server/`.
|
|
- Server DB `GOOSE_DBSTRING` vs test DB `TEST_DATABASE_URL` intentionally separate.
|
|
- `testClient/` hits live `api.naco.li` — not run by `go test`.
|
|
- `server.go` is large; new tool: register in `registerTools()` and add `func (s *Server) myTool(*mcp.CallToolRequest)`.
|
|
- Unix socket MCP + TCP readyz sidecar; `mcpServer.service` expects `bin/main` at repo root.
|
|
- Platform inferred from author handle in `determinePlatform`/`fetch*` — not caller-supplied.
|
|
- Bluesky auth refresh via `isAuthError`/`withRefresh` — 401/expired token retried once.
|
|
- Twitter client clamps `limit` (`>100` → 100), retries 429/5xx/transport errors with `x-rate-limit-*`/`Retry-After` backoff (bounded by `maxRateLimitWait`), and has a 60s per-request timeout.
|
|
- ActivityPub client: GTS Mastodon API only serves locally-federated statuses. Remote actors are resolved via `/accounts/search?resolve=true` (not `/accounts/lookup`, which skips WebFinger). When the local cache can't satisfy full `limit` (clamped to 200), `GetLatestNPosts`/`GetPostsInTimeframe` fall back to walking the remote outbox directly (webfinger → actor → `OrderedCollection`/`OrderedCollectionPage`), filtering `Create`/`Note`-like activities and dereferencing IRI-only objects; `GetPost` derefs status URLs over AP when GTS `/search` can't resolve them. The `read:search` scope is needed for status-URL resolution. Outbox-fetched posts are warmed into GTS in background via `GET /api/v2/search?resolve=true&type=statuses&limit=1` per post, throttled at 1 req/s per remote host via per-host token bucket (`internal/activitypub/throttle.go`) with cross-host concurrency, no follows.
|
|
- Coverage ranges for analytics are interval-merged; disjoint gaps must remain uncovered (see `coverage_test.go`).
|