CepatEdge

Appearance

Internal Logger Improvement Plan ​

This document describes the improvements made to the internal logger and the plan for consistent use across services.

Implemented (current) ​

1. Per-user index (userId index) ​

  • Write path: When a log entry has userId set, the internal cache service also updates a per-shard user index: key internal:userindex:{userId} on the same shard, value array of { id, timestamp } sorted newest first.
  • Read path: When getLogs({ userId, ... }) is called, the multi-shard reader uses the user index instead of the global index: it reads internal:userindex:{userId} from each shard. Only that user’s log IDs are scanned, so listing is much faster and bounded.

2. Cursor-based pagination ​

  • No offset/page: The logs API uses cursor-based pagination only.
  • Route: GET /me/security/logs?limit=100&cursor=...&level=...&category=... (cursor optional).
  • Response: { data: InternalLogEntry[], pagination: { limit, nextCursor?, hasMore } }.
  • Internal: InternalLogFilters has cursor?: string; getLogs() returns InternalLogListResult { entries, nextCursor?, hasMore }. The multi-shard reader supports cursor (format timestamp:id) and returns nextCursor when there are more results.

3. Early exit when limit reached ​

  • When reading from shards, the multi-shard reader stops requesting more shards once allEntries.length >= limit, so we don’t over-fetch.

4. Services using internalLogger with userId ​

  • Auth login: Already logs with userId (suspicious, warning, info).
  • Auth register: logAuthEvent('register_success', userId, ...).
  • Auth logout: logAuthEvent('logout_success', session.userId, ...), logSecurityEvent for failures with userId.
  • Auth profile: logUserEvent, logSecurityEvent with userId.
  • Auth verify: logSecurityEvent('auth_verification_failed', ...) and logSystemEvent('auth_verification_error', ...) on verification failure.
  • Me/avatar: logUserEvent with userId on upload/delete failures.
  • Maintenance core: logMaintenance and logSuspicious (logSecurityEvent-style) with userId for maintenance and security events.
  • InternalLoggerService exposes: logAuthEvent, logSecurityEvent, logSystemEvent, logUserEvent, logMaintenance, plus warning/error/suspicious/info with optional userId/ipAddress.

Types and locations ​

  • Cache types: InternalLogEntry, InternalLogFilters, InternalLogListResult in src/types/cache/internal.ts.
  • User logs types: UserLogsListOptions, UserLogsListResult in src/types/me/logs.ts (cursor-based: cursor, nextCursor, hasMore; no page/total).
  • Internal cache: apps/worker/src/lib/services/cache/internal/index.ts (write: global index + user index when userId present; getLogs returns list result with cursor).
  • Multi-shard reader: apps/worker/src/lib/services/cache/multi-shard-reader/index.ts (supports indexKeySuffix for user index, cursor, limit, early exit).
  • User logs service: apps/worker/src/lib/services/me/logs/index.ts (cursor-only, uses internal logger’s getLogs with userId).
  • Route: GET /me/security/logs in apps/worker/src/routes/me/security.ts (query: limit, cursor, level, category).

Optional future work ​

  • Cursor in response for admin/system logs: When implementing GET /system/internal/logs, use the same cursor-based API and InternalLogListResult.
  • More services: Ensure 2FA, secure-action, password reset, session revoke, user management (admin actions), and API/rate-limit paths log failures and important events with userId where applicable (see user-security-logs.md “Where else to use the internal logger”).

References ​