icon
AI Tools AI Agents MCP Ranking Submit & Advertise
Login

Comprehensive LLM inference Tools for Every Need

Get access to LLM inference solutions that address multiple requirements. One-stop resources for streamlined workflows.

LLM inference

  • rag-services

    rag-services is an open-source microservices framework enabling scalable retrieval-augmented generation pipelines with vector storage, LLM inference, and orchestration.
    0
    0
    What is rag-services?
    rag-services is an extensible platform that breaks down RAG pipelines into discrete microservices. It offers a document store service, a vector index service, an embedder service, multiple LLM inference services, and an orchestrator service to coordinate workflows. Each component exposes REST APIs, allowing you to mix and match databases and model providers. With Docker and Docker Compose support, you can deploy locally or in Kubernetes clusters. The framework enables scalable, fault-tolerant RAG solutions for chatbots, knowledge bases, and automated document Q&A.
Featured