feat: Add database migration service with Alembic for PostgreSQL, integrating job management and counting functionalities. Update Docker Compose to include db-migrations service, enhancing the microservices architecture. Revise documentation for clarity on database schema and migration processes.

This commit is contained in:
erik
2026-07-06 00:50:50 -05:00
parent d95f8f6ad8
commit 081da999d4
43 changed files with 2911 additions and 2271 deletions
+56 -19
View File
@@ -1,29 +1,66 @@
# IPSExtractor API Gateway
# api_gateway
Gateway HTTP opcional. Esta carpeta puede vivir como repo propio: solo depende
de HTTP para hablar con `extraction` y `cleaning`; no importa codigo interno de
esos servicios.
Puerta de enlace HTTP (FastAPI) que actúa como punto de entrada único para el sistema IPSExtractor.
Enruta las peticiones del cliente hacia los microservicios internos sin lógica de negocio propia.
## Estructura
---
```text
api/ Endpoints FastAPI y proxy HTTP
requirements.txt
## Puerto
**8000**
---
## Enrutamiento
| Ruta cliente | Servicio destino | Descripción |
|--------------|------------------|-------------|
| `GET /health` | Agrega `/health` de los tres servicios | Estado del sistema |
| `GET /models` | `extraction:8001/models` | Modelos LLM disponibles |
| `GET /config` | `extraction:8001/config` + `cleaning:8002/rules` | Configuración |
| `/extraction/{path}` | `extraction:8001/{path}` | Extracción (quita prefijo) |
| `/tasks/{path}` | `extraction:8001/tasks/{path}` | Tareas de extracción |
| `/jobs/{path}` | `extraction:8001/jobs/{path}` | Jobs |
| `/count-jobs/{path}` | `extraction:8001/count-jobs/{path}` | Jobs de conteo |
| `/cleaning/{path}` | `cleaning:8002/{path}` | Limpieza |
| `/count/{path}` | `count-api:8080/{path}` | Conteo OCR+ML (quita prefijo) |
El proxy reenvía método, query string, headers y body sin modificaciones.
Timeout: ilimitado (para uploads grandes).
---
## Variables de entorno
| Variable | Default | Descripción |
|----------|---------|-------------|
| `EXTRACTION_URL` | `http://extraction:8001` | URL interna del servicio extraction |
| `CLEANING_URL` | `http://cleaning:8002` | URL interna del servicio cleaning |
| `COUNT_URL` | `http://count-api:8080` | URL interna del servicio count-api |
---
## CORS
Solo permite orígenes `localhost` y `127.0.0.1`. Sin autenticación.
---
## Estructura del código
```
api/
main.py — FastAPI app con proxy HTTP (httpx)
Dockerfile
requirements.txt — fastapi, httpx, uvicorn
```
## Desarrollo local
---
## Arrancar localmente
```bash
pip install -r requirements.txt
uvicorn api.main:app --host 0.0.0.0 --port 8000
docker compose up --build api_gateway
```
## Endpoints principales
- `GET /health`
- `GET /models`
- `GET /config`
- `ANY /extraction/{path}`
- `ANY /cleaning/{path}`
API disponible en `http://localhost:8000`.
+28
View File
@@ -11,6 +11,7 @@ from fastapi.middleware.cors import CORSMiddleware
EXTRACTION_URL = os.getenv("EXTRACTION_URL", "http://extraction:8001").rstrip("/")
CLEANING_URL = os.getenv("CLEANING_URL", "http://cleaning:8002").rstrip("/")
COUNT_URL = os.getenv("COUNT_URL", "http://count-api:8080").rstrip("/")
app = FastAPI(title="IPSExtractor API Gateway", version="1.0.0")
@@ -28,11 +29,13 @@ async def health() -> dict[str, Any]:
async with httpx.AsyncClient(timeout=10.0) as client:
extraction = await _safe_health(client, EXTRACTION_URL)
cleaning = await _safe_health(client, CLEANING_URL)
counting = await _safe_health(client, COUNT_URL)
return {
"status": "ok" if extraction["ok"] and cleaning["ok"] else "degraded",
"services": {
"extraction": extraction,
"cleaning": cleaning,
"counting": counting,
},
}
@@ -73,6 +76,31 @@ async def cleaning_proxy(path: str, request: Request) -> Response:
return await _proxy_request(request, f"{CLEANING_URL}/{path}")
@app.api_route("/count/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
async def count_proxy(path: str, request: Request) -> Response:
return await _proxy_request(request, f"{COUNT_URL}/{path}")
@app.api_route("/count-jobs", methods=["GET", "POST"])
async def count_jobs_root_proxy(request: Request) -> Response:
return await _proxy_request(request, f"{EXTRACTION_URL}/count-jobs")
@app.api_route("/count-jobs/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
async def count_jobs_proxy(path: str, request: Request) -> Response:
return await _proxy_request(request, f"{EXTRACTION_URL}/count-jobs/{path}")
@app.api_route("/jobs", methods=["GET"])
async def jobs_root_proxy(request: Request) -> Response:
return await _proxy_request(request, f"{EXTRACTION_URL}/jobs")
@app.api_route("/jobs/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
async def jobs_proxy(path: str, request: Request) -> Response:
return await _proxy_request(request, f"{EXTRACTION_URL}/jobs/{path}")
async def _safe_health(client: httpx.AsyncClient, base_url: str) -> dict[str, Any]:
try:
response = await client.get(f"{base_url}/health")