LLMs provides a unified abstraction over various open-source and hosted language models, allowing developers to load and run models through a single interface. It supports model discovery, prompt and pipeline management, batch processing, and fine-grained control over tokens, temperature, and streaming. Users can easily switch between CPU and GPU backends, integrate with local or remote model hosts, and cache responses for performance. The framework includes utilities for prompt templates, response parsing, and benchmarking model performance. By decoupling application logic from model-specific implementations, LLMs accelerates the development of NLP-powered applications such as chatbots, text generation, summarization, translation, and more, without vendor lock-in or proprietary APIs.

OSU NLP Middleware is a lightweight framework built in Python that simplifies the development of AI agent systems. It provides a core agent loop that orchestrates interactions between natural language models and external tool functions defined as plugins. The framework supports popular LLM providers (OpenAI, Hugging Face, etc.), and enables developers to register custom tools for tasks like database queries, document retrieval, web search, mathematical computation, and RESTful API calls. Middleware manages conversation history, handles rate limits, and logs all interactions. It also offers configurable caching and retry policies for improved reliability, making it easy to build intelligent assistants, chatbots, and autonomous workflows with minimal boilerplate code.