fix: improved server side cleanup after ingest

[cyclotruc]

Refactor repository cleanup from background task to immediate cleanup

Overview

This PR refactors the repository cleanup mechanism from a background cleanup task to immediate cleanup after each processing operation, improving resource management and simplifying the codebase.

Changes Made

🧹 Cleanup Strategy Change

• Before: Background task periodically scanned and deleted repositories older than 1 hour
• After: Immediate cleanup after each repository processing (success or failure)

🔧 Key Modifications

src/server/main.py

• Removed lifespan import and usage from FastAPI app initialization
• Simplified app setup by removing the lifespan context manager

src/server/query_processor.py

• Added new _cleanup_repository() function for immediate cleanup
• Integrated cleanup calls in process_query():
  • Cleanup on processing failure (in exception handler)
  • Cleanup on successful processing completion
• Uses shutil.rmtree() with proper error handling and logging

src/server/server_utils.py

• Removed entire background cleanup system:
  • lifespan() context manager
  • _remove_old_repositories() background task
  • _process_folder() helper function
  • _append_line() utility function
• Cleaned up unused imports (asyncio, shutil, time, contextlib, pathlib)

src/server/server_config.py

• Removed DELETE_REPO_AFTER constant (no longer needed)

[github-actions]

⚙️ Preview environment was undeployed.

[NicolasIRAGNE]

lgtm, I think this is not ideal in case of local (eg. dev) non-S3 environments but I don't have a better solution that is cheap to implement so we can go that route now