Embedding-similarity based filtering that reduces tool context down to the most relevant candidates in under 10ms.
Detailed Introduction
MCP Tool Filter from Portkey AI is an open-source component that provides ultra-fast semantic filtering for Model Context Protocol (MCP) servers. By computing embedding similarity between a query and tool descriptions, it selects the most relevant subset (typically 10–20 tools) from thousands of available tools in real time. Integrating this filter into MCP servers or agent pipelines significantly reduces tool context size, lowering latency and cost while preserving functional coverage.
Main Features
- Embedding-similarity driven semantic filtering with support for multiple embedding models and index backends.
- Extremely low latency: optimized data structures and query paths complete selection in milliseconds.
- Configurable thresholds and candidate counts to trade off precision and coverage.
- MIT-licensed open-source implementation suitable for local/private deployment and auditing.
Use Cases
- MCP servers or agent platforms that need to quickly select tool candidates from large tool registries.
- Latency- and cost-sensitive online services that minimize high-cost model calls by compressing tool context.
- Engineering observability and auditability for retrieval-to-tool-selection pipelines.
Technical Features
- Modular adapters: plug different embedding models and vector index backends (in-memory, Faiss, Annoy, etc.).
- Optimized query path: preprocessing, compressed indices, and parallel scoring to reduce end-to-end latency.
- Observability: emits call metrics and filtering confidences for monitoring and tuning.