Roundups of self-hosted private AI building blocks: chat interfaces, LLM app builders, inference servers, vector databases and LLM observability, judged on license, maintenance status and operating effort.
Open WebUI, LibreChat and AnythingLLM compared as company-wide private ChatGPT alternatives: licenses, user management and SSO, document retrieval, agents and MCP, model provider support and the infrastructure each requires.
Dify, Langflow and Flowise compared for building LLM apps and agents visually, with n8n noted for automation-first teams. Flags that Flowise was archived in August 2026 and compares licenses, deployment, databases and extensibility.
vLLM, Ollama and llama.cpp compared for serving open-weight models in your own infrastructure, with SGLang covered as the other high-throughput GPU engine. Looks at hardware support, APIs, batching and throughput features, licensing and operations.
Qdrant, Weaviate and Milvus compared in depth, with pgvector and Chroma covered as the simpler options. Looks at licenses and enterprise features, architecture, scaling and replication, access control and operating effort.
Langfuse, Opik and Arize Phoenix compared for tracing, evaluation and prompt management in your own infrastructure: license type (Phoenix is Elastic License 2.0), feature gating, storage and instrumentation standards.
LiteLLM, Portkey AI Gateway and Kong AI Gateway compared, with Envoy AI Gateway (now Agent Router) named for Kubernetes platform teams: licenses and paid tiers, provider coverage, keys, budgets and rate limits, fallbacks, observability hooks and deployment.
RAGFlow, Onyx and Dify compared for retrieval over company documents, with AnythingLLM named for small teams: document parsing, connectors and permission sync, citations, agents, licenses and paid editions, and the hardware each needs.
For teams that would rather add vector search to a database they already run than deploy a vector database: pgvector, OpenSearch k-NN and Elasticsearch dense_vector compared on index types, dimension limits, filtering, licensing and scaling, with the point where a dedicated store is worth it.
Tell us what you're working on, whether it's a deployment, an audit, a security test or a cyber range. You'll speak with an engineer who can help you scope it.