wangdl/api-server
2
0
Fork 0
Code Issues 28 Pull Requests Actions Packages Projects Releases Wiki Activity
499 Commits 1 Branch 0 Tags
Commit Graph

27 Commits

Author SHA1 Message Date
wangdl
a621bd169d feat: 文档清理 + RAG Worker 迁移 + 批量操作 + HTTP 日志 + 学习会话修复
Some checks failed
Deploy API Server / build-and-unit (push) Successful in 41s
Details
Deploy API Server / current-integration (push) Successful in 33s
Details
Deploy API Server / backward-compat (push) Successful in 0s
Details
Deploy API Server / deploy (push) Failing after 57s
Details 
- 删除根 docs/ 中的 audit/issue 文档
- 删除 api-server/docs/ 中的 16 个审核/里程碑文档
- rag-worker/ 从 api-server 迁出为独立项目
- 新增 admin 导入批量重新解析、按知识库筛选、后端排序
- 新增 source 软删除标记，防止已删 source 重新解析
- 修复 upload-url 500（重复 UploadSession 创建 + 缺 sizeBytes）
- 修复 loadContextByScope 读 chunks 而非依赖 textLength
- 修复 MaterialReadingProgress.create() increment 语法错误
- 新增 LoggingInterceptor 记录所有 HTTP 请求
- 新增学习会话批量删除接口
- enrichWithNames 增加 email 回退
- 全局异常过滤器和 name-resolver 增强

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-07-01 22:44:36 +08:00
wangdl
add6a1ab4c feat: LocalStorageProvider 与文件上传生命周期 Integration
All checks were successful
Deploy API Server / build-and-unit (push) Successful in 41s
Details
Deploy API Server / current-integration (push) Successful in 32s
Details
Deploy API Server / backward-compat (push) Successful in 0s
Details
Deploy API Server / deploy (push) Successful in 1m3s
Details 
- 新增 LocalStorageProvider (本地磁盘存储)
- 新增 LocalStorageController (PUT upload / GET download)
- StorageService 支持 STORAGE_DRIVER=local 切换
- 完整文件生命周期: upload-url → PUT → complete → head → delete
- 重复 complete 幂等、删除释放
- 8 项 Integration 测试全部 PASS

无需生产 COS 密钥即可本地测试文件上传
 2026-06-27 13:46:18 +08:00
wangdl
05cf369bee feat: 本地开发环境配置 + Python 3.9 兼容修复
Some checks failed
Deploy API Server / build-and-unit (push) Successful in 42s
Details
Deploy API Server / current-integration (push) Failing after 3m6s
Details
Deploy API Server / backward-compat (push) Successful in 17s
Details
Deploy API Server / deploy (push) Has been skipped
Details 
- 添加 docker-compose.local.yml（MySQL/Redis/Qdrant 本地基础设施）
- 添加 mysql.conf.d 本地 MySQL 低资源配置
- RAG Worker Python 3.9 兼容（from __future__ import annotations）
- 同步最新业务代码变更
 2026-06-27 12:42:44 +08:00
wangdl
49151117c8 fix: M-AI-03-GATE-FIX — Admin Retry 统一链路 + 队列测试修正 + E2E 激活 + CI 门禁
Some checks failed
Deploy API Server / build-and-unit (push) Successful in 32s
Details
Deploy API Server / current-integration (push) Failing after 1m34s
Details
Deploy API Server / backward-compat (push) Successful in 16s
Details
Deploy API Server / m-ai-03-synthetic-e2e (push) Failing after 12s
Details
Deploy API Server / deploy (push) Has been skipped
Details 
### 一、生命周期契约核对
- ADR-003 §1.1: 5 状态（queued/running/succeeded/failed/cancelled）
- cancel_requested = cancelRequestedAt 信号（非状态）
- created/retrying 不在状态机中
- 代码完全一致 ✅

### 二、Admin Retry 修复
- retryJob 不再绕过 Registry/CreationService/Outbox
- 改为调用 AiJobCreationService.createJob()（统一事务链路）
- parentJobId + triggerType=admin_manual 正确传入
- 幂等键: admin-retry:<jobId>:<timestamp>（允许不同重试产生不同 Job）
- 原 Job 不修改，新 Job 使用新 ID
- 新增 AiJobCreationService.retrySnapshotContent 支持复用 Snapshot

### 三、队列定义测试修正
- 旧队列断言保持 6 个 legacy queues 的默认值检查
- 新增 M-AI-03 per-queue 断言（ai-interactive: concurrency=2/attempts=2/backoff=2000; ai-background: concurrency=1/attempts=3/backoff=5000/lockDuration=60s）
- lockDuration 测试改为 >= 30s（允许 per-queue 差异）

### 四、Synthetic E2E 激活
- test/m-ai-03-synthetic.e2e-spec.ts: 12 真实场景（创建/幂等/原子/cancel/JWT/隔离/脱敏/未知type/状态机）
- test/jest-m-ai-03.json: 真实 infra config（无 mock Prisma/BullMQ/Redis）
- CI Workflow 新增 m-ai-03-synthetic-e2e job
- deploy needs 新增 m-ai-03-synthetic-e2e 依赖

### 五、测试结果
- 单元测试: 471/471 passed, 0 failed
- E2E: 12 场景待 CI 真实 infra 执行
- queue-definitions: 19/19 passed

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-21 10:47:09 +08:00
wangdl
5108a9a250 fix: queue-definitions.spec.ts 显式导入 QUEUE_AI_INTERACTIVE/QUEUE_AI_BACKGROUND
Some checks failed
Deploy API Server / build-and-unit (push) Failing after 28s
Details
Deploy API Server / current-integration (push) Has been skipped
Details
Deploy API Server / backward-compat (push) Has been skipped
Details
Deploy API Server / deploy (push) Has been skipped
Details
2026-06-21 10:26:52 +08:00
wangdl
fcce35742f feat: M-AI-03 注册新队列 ai-interactive/ai-background 与薄 Worker Processor
Some checks failed
Deploy API Server / build-and-unit (push) Failing after 29s
Details
Deploy API Server / current-integration (push) Has been skipped
Details
Deploy API Server / backward-compat (push) Has been skipped
Details
Deploy API Server / deploy (push) Has been skipped
Details 
- queue.constants.ts: 新增 QUEUE_AI_INTERACTIVE / QUEUE_AI_BACKGROUND
- queue-definitions.ts: 队列配置对齐 ADR-003 §3.1
  ai-interactive: concurrency=2, attempts=2, backoff=2000ms
  ai-background: concurrency=1, attempts=3, backoff=5000ms
- queue.service.ts: QueueService 注入并路由新队列
- queue.module.ts: BullModule.registerQueue 注册新队列
- AiJobExecutionEngine 接口 + AI_JOB_EXECUTION_ENGINE 注入 Token
- AiInteractiveJobWorker + AiBackgroundJobWorker: 薄 Processor，仅接收 {jobId} 调用 Engine
- worker.module.ts: 注册两个新 Processor（#291 提供 Engine 真实实现前为 no-op）

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-20 18:02:26 +08:00
wangdl
0dab626cd8 feat(M-AI-02-07): transactional OutboxEvent model 
- 16 columns, dedupeKey UNIQUE, (status,availableAt) + (lockedAt) indexes
- CAS markProcessing (updateMany where status='pending')
- createInTransaction(tx) for atomic Job+Outbox writes
- 12 unit tests (CAS, transaction, lock release, constraints)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-20 12:24:36 +08:00
wangdl
cdc228ccf0 fix: break circular dependency between queue-definitions and queue-service
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 45s
Details 
Extract queue name constants to queue.constants.ts to resolve
the require cycle that caused 'Cannot read properties of undefined'
at startup. Both queue-definitions.ts and queue.service.ts now
import from the shared constants file.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-19 21:24:33 +08:00
wangdl
4fb652d273 feat: M-AI-01 Worker 进程边界与部署收口（全 8 个 Issue）
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 39s
Details 
M-AI-01-01: ADR-001 统一 AI 架构决策记录 + 附录 A 依赖闭包审计
M-AI-01-02: Worker 执行依赖闭包审计（附录 A 文档）
M-AI-01-03: 分离 AppModule/WorkerModule Processor 注册
M-AI-01-04: 统一 Queue Definition Registry + 环境变量覆盖
M-AI-01-05: 独立 Worker 启动与生命周期（instanceId/校验/优雅关闭）
M-AI-01-06: Docker + systemd 部署拓扑（unit 文件 + 回滚文档）
M-AI-01-07: Worker Heartbeat（Redis TTL 90s）+ Admin 在线状态 API
M-AI-01-08: 集成验收脚本

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-06-19 21:22:48 +08:00
wangdl
859156a59b fix: COS 自定义域名空值时回退到原生域名
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 45s
Details 
- CosStorageProvider 初始化时 Domain 为空不再传入 SDK
- 原生 COS 域名 (cos.ap-beijing.myqcloud.com) 有有效 TLS 证书
- 生产环境移除 STORAGE_COS_DOMAIN 配置

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-27 21:48:33 +08:00
WangDL
f0ddd7cf38 fix: add try-catch to all OnModuleInit hooks to prevent startup crashes
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 34s
Details 
- PrismaService.onModuleInit: catch DB connection failures
- CostAggregationService.onModuleInit: catch aggregation errors
- MetricsCleanupService.onModuleInit: catch cleanup errors

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-24 12:48:30 +08:00
WangDL
af800b2eb5 fix: TypeScript build errors — RedisService.keys(), eventBus void, Qdrant API
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 22s
Details 
- Add keys() method to RedisService for WorkerHeartbeat
- Wrap eventBus.publish() calls in try/catch (returns void)
- Fix Qdrant createPayloadIndex API signature (2 args, not 3)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-24 10:22:56 +08:00
WangDL
5fd737967f feat: M1-01~03 — AI Gateway deepening, Vector module, Task Queue deepening
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 23s
Details 
M1-01 AI Gateway:
- DB-driven ModelRoute/ProviderConfig/FallbackEvent tables
- ModelRouter rewrite with loadFromDb() hot-reload
- Fallback event recording + AIUsageRecorded event publishing
- Admin AAPI: routes CRUD, provider enable/disable, fallback events log

M1-02 Vector & Retrieval:
- VectorService with Qdrant client (upsert/delete/search/rerank)
- Admin AAPI: collection status, vector count, reindex trigger

M1-03 Task Queue:
- 16 task types with default retry/timeout configs
- Task stats dashboard, worker status panel, batch retry endpoint

M0 audit fixes:
- ApiMetric retention policy (30-day cleanup)
- Content Safety integration in Files module
- Queue registration centralized (domain-events)
- SECRET_MASTER_KEY production validation

E2E tests:
- M0: 28 smoke tests covering all 14 M0 issues
- M1: 16 tests covering M1-01/02/03
- Mock infrastructure: prisma, ioredis, jose, bullmq, qdrant

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-24 10:18:07 +08:00
WangDL
903474b7c2 fix: eventBus at end of params
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 35s
Details
2026-05-23 19:48:18 +08:00
WangDL
7c1e439c3b fix: reorder optional params
Some checks failed
Deploy API Server / build-and-deploy (push) Has been cancelled
Details
2026-05-23 19:48:06 +08:00
WangDL
ad0e4f397a fix: break circular dependency between EventBusService and QueueService
Some checks failed
Deploy API Server / build-and-deploy (push) Has been cancelled
Details
2026-05-23 19:47:40 +08:00
WangDL
99d03bb26c feat: M0-10 — TaskLog table + task events + retry audit
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 32s
Details
2026-05-23 19:29:08 +08:00
WangDL
2c6d56bcfc feat: M0-10 — Task types enum + worker heartbeat + Domain Events for tasks
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 40s
Details
2026-05-23 19:27:46 +08:00
WangDL
9244db05b4 feat: M0-09 — BullMQ cleanup queue + Domain Events + async COS delete
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 35s
Details
2026-05-23 09:47:30 +08:00
WangDL
b5a983dc6b feat: M0-04 Audit — async BullMQ writes + riskLevel + reason + SecurityEvent
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 39s
Details
2026-05-22 23:03:32 +08:00
WangDL
3fd5f94db5 feat: complete M0-01 — TraceId + BaseService + DomainEvent + SensitiveLogger
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 35s
Details
2026-05-22 22:19:06 +08:00
WangDL
bced62c8f6 fix: add missing redis client connect call in onModuleInit
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 58s
Details 
ioredis with lazyConnect: true requires explicit .connect() — without it
the client never connects and isHealthy() always returns false.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-17 23:00:11 +08:00
WangDL
ca90d34b22 fix: 将 RateLimitService 注入从 StorageService 移到 FilesService
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 56s
Details 
StorageService 在 @Global StorageModule 中，无法注入 AppModule 的 RateLimitService。
将限流调用上移到 FilesService.requestUploadUrl 中。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-17 22:50:31 +08:00
WangDL
6d7cbffc3b feat: COS 对象存储接入 — CosStorageProvider + FilesModule
Some checks failed
Deploy API Server / build-and-deploy (push) Failing after 3m0s
Details 
- 安装 cos-nodejs-sdk-v5，封装 CosStorageProvider（upload/download/delete/healthCheck）
- 重写 StorageService，新增 createUploadUrl/verifyUpload/getDownloadUrl/deleteObject
- 创建 FilesModule：POST /files/upload-url, POST /files/complete, GET /files/:id, DELETE /files/:id
- UploadedFile 新增 objectKey/bucket 字段
- 对象键格式 {userId}/{YYYYMM}/{sanitizedName}.{ext}
- 接入文件类型校验（ALLOWED_FILE_TYPES）+ 上传限流（10次/小时/用户）
- 配置文件 cos.longde.cloud → zhixi-1259685406 / ap-guangzhou

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-17 22:30:14 +08:00
WangDL
08f31dd5b6 feat: P0 后端补全 — BullMQ Workers 注册 + 用户 Profile API + 角色权限 
- AppModule 注册 3 个 BullMQ Workers (AiAnalysis/DocumentImport/Notification)
- Users 模块新增 GET/PATCH /users/me/profile 端点:
  - GET 读取 UserProfile (learningIdentity, learningDirection, bio, currentGoal)
  - PATCH upsert UserProfile
  - GET /users/me 返回 profile + preferences (include join)
- 新增 RolesGuard + @Roles() 装饰器 (UserRole enum)
- QueueModule/QueueService 改进
- 各模块 controller/repository/service 完善

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-17 19:08:07 +08:00
WangDL
007b56dad5 feat: AI三层架构 + 全局JwtAuthGuard + 12个Repository迁Prisma
All checks were successful
Deploy API Server / build-and-deploy (push) Successful in 1m0s
Details 
- AI: 新三层架构 Provider→Gateway→Workflow（15文件，DeepSeek+MiniMax）
- Auth: 全局JwtAuthGuard + @Public()装饰器白名单路由
- DB: 12个Repository从Map/Array迁到Prisma
- Schema: 新增AiUsageLog、WaitlistEntry模型
- API: /api-docs-json加Basic Auth保护
- 清理: 删除infrastructure/ai、docs/旧文档

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-05-17 00:39:46 +08:00
WangDL
35de65e99b feat: 重构 api-server 为模块化单体架构，接入 MySQL + Redis 
- 按 BACKEND-PLAN.md 将项目重构为 4 层架构:
  config/ -> common/ -> infrastructure/ -> modules/
- 15 个业务模块，遵循 Controller → Service → Repository 分层
- infrastructure: PrismaService / RedisService / QueueService / AiService / StorageService
- common: guards / interceptors / filters / pipes / decorators / dto / types / utils
- Prisma schema 含 27 张表，MySQL 8.0 服务器 db push 成功
- Redis 7 接入: 限流/任务状态/分布式锁/队列预留
- ai-analysis 模块: 每日 50 次限流 + 重复提交锁 + 异步任务状态追踪
- document-import 模块: 异步导入流程 + 进度追踪
- notifications 模块: BullMQ notification 队列预留
- /health 端点实时返回 database + redis 连接状态
- Swagger 注册 15 个 tag，67 个路由全部映射
 2026-05-09 18:25:04 +08:00