Synthetic monitoring tool for LLM inference endpoints that measures TTFT, latency, throughput, and errors across major providers like OpenAI, Anthropic, Google, and Azure. Includes CLI and MCP server with Prometheus and OpenTelemetry export capabilities.
llmprobe is a synthetic monitoring solution designed to track and measure the performance of LLM inference endpoints across multiple providers. It monitors key metrics including Time-to-First-Token (TTFT), latency, throughput, and error rates to ensure optimal LLM service reliability and performance.
Works with OpenAI, Anthropic, Google, Azure, AWS Bedrock, and local inference servers including vLLM, SGLang, and Ollama. This multi-provider support makes it ideal for teams managing heterogeneous LLM infrastructure.
Monday.com MCP Server streamlines board management, item operations, and workflow automation for teams. I…
by NotionFlow
Sentry MCP Server provides comprehensive error tracking and performance monitoring, helping developers id…
by AnalyticsPro
Cloudflare MCP Server simplifies Cloudflare management by providing tools for DNS management, Workers dep…
by PricingBot