AI Server
A self-hosted Python AI server that runs local LLMs behind OpenAI- and Anthropic-compatible APIs, with agents, RAG, multi-modal services and MCP management.
AI Server is a Python 3.11+ Flask application that serves large language models through llama.cpp, vLLM and MLX backends. It exposes OpenAI-compatible endpoints for chat, embeddings, audio, images, responses and fine-tuning, and an Anthropic-compatible messages endpoint. Its agent layer runs on LangGraph, DeepAgents or a native runtime and ships 54 specialty agents in four packs, alongside connector agents for 38 enterprise systems. Retrieval-augmented generation uses FAISS or ChromaDB with clustering, graph extraction and grounding checks, next to speech, vision, image generation, MCP server management and enterprise sign-in.
Features
OpenAI-compatible API for chat, embeddings, audio, images, responses and fine-tuning, plus an Anthropic-compatible messages API
Local inference with llama.cpp, vLLM and MLX, plus Hugging Face and Ollama model discovery
54 specialty agents (coding, business analysis, office work, oil and gas) on LangGraph, DeepAgents or a native runtime
Connector agents for 38 enterprise systems, including SAP, Salesforce, ServiceNow, Kafka and SharePoint
RAG with FAISS or ChromaDB, distributed clusters, graph extraction and grounding validation
Speech-to-text, text-to-speech, voice-to-voice, text-to-image, vision and object detection
MCP server lifecycle management, tool catalog and tooling bridge
JWT, API tokens, OAuth2/OIDC, LDAP, Azure AD, MFA and RBAC; SDKs for 14 languages plus Godot and Unity