feat(quota): implement monthly quota, role-based limits, reset mechanism and cron scheduler (!17) · Merge Requests · ThinhNC / upgrade-data-crawler-be

feat(quota): implement monthly quota, role-based limits, reset mechanism and cron scheduler

Features & Enhancements: Monthly Quota, Role-Based Resource Limits, Zero-Loss Reset & Cron Scheduler

Summary

This PR introduces enterprise-grade resource quota management, role-based quota templating with automatic user synchronization, zero-data-loss quota reset capabilities, and an automated background job scheduler (Cron) with defense-in-depth permission resolution.

Key Changes

1. Database & Schema Updates (Prisma)

Added maxPagesPerMonthLimit (default 10,000 / 1,000), maxJobsPerMonthLimit (default 500 / 100), and quotaResetAt (DateTime?) to both User and Role models.
Applied PostgreSQL migration 20260909030359_add_role_and_monthly_quota.

2. Resource Quota Calculation & Zero-Loss Reset (`auth` & `crawl-jobs`)

UTC+7 Timezone Precision: Calculated startOfDay and startOfMonth using Asia/Ho_Chi_Minh.
Effective Horizon: Quota consumption is evaluated against effectiveSince = Math.max(startOfDay/startOfMonth, quotaResetAt).
Zero Data Loss: Resetting a quota updates quotaResetAt rather than mutating or deleting historical crawl records.
Implemented POST /api/v1/users/:id/reset-quota and POST /api/v1/roles/:id/reset-quota with audit logging (AUDIT_ACTIONS.USER_QUOTA_RESET, AUDIT_ACTIONS.ROLE_QUOTA_RESET).

3. Role-Based Quota Templates & Auto-Sync (`roles`)

Extended CreateRoleDto and UpdateRoleDto to define default quota limits for roles.
Supported syncUsersQuota flag in UpdateRoleDto to bulk update limits for all users assigned to the role.

4. Background Job Scheduler Module (`cron`)

Built complete 5-layer scheduler system with BullMQ queue (cron-scheduler-queue) and worker.
Supports periodic maintenance tasks: log cleanup, expired token purge, orphan asset cleanup, and summary email digests.
Integrated automatic system permission bootstrapping (permissionService.ensureSystemPermissions()) and ADMIN default permission inheritance in permission.middleware.ts.

5. Environment Configuration

Updated .env and .env.example with USER_MAX_PAGES_PER_MONTH=1000 and USER_MAX_JOBS_PER_MONTH=100.

## Features & Enhancements: Monthly Quota, Role-Based Resource Limits, Zero-Loss Reset & Cron Scheduler

### Summary
This PR introduces enterprise-grade resource quota management, role-based quota templating with automatic user synchronization, zero-data-loss quota reset capabilities, and an automated background job scheduler (Cron) with defense-in-depth permission resolution.

---

### Key Changes

#### 1. Database & Schema Updates (Prisma)
- Added `maxPagesPerMonthLimit` (default `10,000` / `1,000`), `maxJobsPerMonthLimit` (default `500` / `100`), and `quotaResetAt` (`DateTime?`) to both `User` and `Role` models.
- Applied PostgreSQL migration `20260909030359_add_role_and_monthly_quota`.

#### 2. Resource Quota Calculation & Zero-Loss Reset (`auth` & `crawl-jobs`)
- **UTC+7 Timezone Precision**: Calculated `startOfDay` and `startOfMonth` using `Asia/Ho_Chi_Minh`.
- **Effective Horizon**: Quota consumption is evaluated against `effectiveSince = Math.max(startOfDay/startOfMonth, quotaResetAt)`.
- **Zero Data Loss**: Resetting a quota updates `quotaResetAt` rather than mutating or deleting historical crawl records.
- Implemented `POST /api/v1/users/:id/reset-quota` and `POST /api/v1/roles/:id/reset-quota` with audit logging (`AUDIT_ACTIONS.USER_QUOTA_RESET`, `AUDIT_ACTIONS.ROLE_QUOTA_RESET`).

#### 3. Role-Based Quota Templates & Auto-Sync (`roles`)
- Extended `CreateRoleDto` and `UpdateRoleDto` to define default quota limits for roles.
- Supported `syncUsersQuota` flag in `UpdateRoleDto` to bulk update limits for all users assigned to the role.

#### 4. Background Job Scheduler Module (`cron`)
- Built complete 5-layer scheduler system with BullMQ queue (`cron-scheduler-queue`) and worker.
- Supports periodic maintenance tasks: log cleanup, expired token purge, orphan asset cleanup, and summary email digests.
- Integrated automatic system permission bootstrapping (`permissionService.ensureSystemPermissions()`) and `ADMIN` default permission inheritance in `permission.middleware.ts`.

#### 5. Environment Configuration
- Updated `.env` and `.env.example` with `USER_MAX_PAGES_PER_MONTH=1000` and `USER_MAX_JOBS_PER_MONTH=100`.