Vellum

Vellum offers a comprehensive AI product development platform designed to streamline the creation and management of AI-powered applications. Its features include systematic prompt iteration, a unified dashboard for prompt management, and side-by-side model comparisons, with native support for tools, structured outputs, and OpenAPI specifications.

The platform's Workflows component provides an IDE for constructing steerable, agent-like systems. This includes a visual graph builder for orchestrating complex AI workflows, support for diverse models, custom code, and map/reduce functions, along with built-in features like loops, parallelism, and error handling.

Vellum's Workflows SDK offers a full-stack solution with a GUI for building AI applications with precision and flexibility, featuring bi-directional synchronization between code and UI, fine-grained control flow, global state management, and customization options with Docker and streaming support.

For quality assurance, Vellum provides Evaluations, enabling test-driven development for AI through scalable test suites accessible via UI, API, or CSV. Users can leverage pre-built or custom metrics to evaluate performance, track quality, cost, latency, and regressions over time.

The Retrieval component simplifies RAG (Retrieval-Augmented Generation) infrastructure with user-friendly APIs for uploading and querying unstructured data. It allows for tweaking chunking, embeddings, and search parameters for advanced use cases, supporting various file types, including tables and images.

Vellum's Deployments feature enables seamless updates to AI systems without requiring application redeployment. It offers one-click deployment across various models and providers, staging environments for safe iteration, and scalable inference endpoints for production environments.

Finally, Observability provides complete visibility into AI system performance through audit logs, debugging tools, feedback capture mechanisms, evaluation loops on live traffic, and dashboards for tracking cost, latency, errors, and trends.