AI Services
LLM inference, RAG retrieval, audio, vision, and multi-modal pipelines
LLM Services (app/services/llm/ — 20 modules)
The LLM domain handles model discovery, download, inference routing, and multi-backend execution.
| Service | Key Class | Responsibility |
|---|---|---|
llm_manager.py | LLMManager | Model discovery (HuggingFace, Ollama), download, validation, recommendation |
llm_environment.py | get_llm_env() | LLM virtual environment lifecycle |
inference_service.py | InferenceService | Multi-backend chat inference routing (llama.cpp, vLLM, MLX) |
llama_server_manager.py | LlamaServerManager | llama.cpp server lifecycle |
vllm_server_manager.py | VLLMServerManager | vLLM server lifecycle |
mlx_server_manager.py | MLXServerManager | MLX-LM server lifecycle (Apple Silicon) |
huggingface_service.py | HuggingFaceService | HuggingFace Hub API: search, download, metadata |
model_download_service.py | ModelDownloadService | Model download with progress tracking |
model_recommendation.py | ModelRecommendationService | Hardware-aware model recommendations |
model_validation_service.py | ModelValidationService | Model integrity and compatibility checks |
model_router.py | ModelRouter | Select optimal backend per model (format, size, GPU) |
chat_execution_service.py | ChatExecutionService | Full chat completion orchestration |
chat_intent_preflight_service.py | ChatIntentPreflightService | Intent analysis before model routing |
tool_calling.py | ToolCallingService | Function/tool calling integration |
vision_support.py | VisionSupport | Multi-modal image processing for LLMs |
ai_service_orchestrator.py | AIServiceOrchestrator | Top-level orchestration across all AI services |
ai_service_model_manager.py | AIServiceModelManager | Unified model lifecycle across service types |
ai_service_curated_models.py | CuratedModels | Curated model catalog with metadata |
Audio Services
| Service | Route | Description |
|---|---|---|
speech_to_text_service.py | /audio/speech-to-text | Whisper-based transcription |
text_to_speech_service.py | /audio/text-to-speech | Multi-engine TTS (Edge, Google, Coqui, system voices) |
voice_to_voice_service.py | /audio/voice-to-voice | Voice conversion (pitch, time stretch) |
Vision & Media Services
| Service | Route | Description |
|---|---|---|
vision_runtime_service.py | /vision | Vision model inference and processing |
text_to_image_service.py | /text-to-image | Image generation from text prompts |
object_detection_service.py | — | Object detection and classification |
document_runtime_service.py | /document-extraction | Document and media processing |
ml_model_service.py | /ml-models | ML model registry, versioning, validation |
Document Extraction
| Service | Description |
|---|---|
document_extraction_environment.py | Document extraction environment lifecycle |
document_extractor.py | PDF, Office, image parsing and text extraction |
ocr_engine_registry.py | OCR engine registration and discovery |
ocr_engine_tracker.py | OCR engine status and health |
Fine-Tuning
| Service | Description |
|---|---|
finetune/finetune_service.py | Fine-tuning orchestration |
finetune/finetune_environment.py | Fine-tuning environment management |
finetune/finetune_standard/ | Standard fine-tuning implementations |