Traigent Integrations Directory - Comprehensive Inventory¶
Overview¶
The traigent/integrations/ directory contains the framework integration system that enables zero-code optimization across multiple LLM providers and frameworks. This system uses a hybrid plugin + framework override architecture.
Total Files: 66 Python files, regenerated from the current traigent/integrations/ tree.
Total Integrations: Built-in coverage spans OpenAI, Anthropic, LangChain, Bedrock, Azure OpenAI, Cohere, Gemini, HuggingFace, LiteLLM, LlamaIndex, Mistral, PydanticAI, vector stores, and observability integrations.
Core Architecture Files¶
1. framework_override.py (1,423 lines) - PRIMARY OVERRIDE ENGINE¶
Purpose: Monkey-patching system for automatic parameter injection
Key Classes:
- _BaseOverrideManagerFallback - Fallback implementation mirroring BaseOverrideManager API
- FrameworkOverrideManager - Main orchestrator for framework parameter overrides
Key Methods:
- __init__()
- is_override_active() / set_override_active()
- extract_config_dict()
- get_method_wrapper() / create_sync_wrapper() / create_async_wrapper()
- store_original() / restore_originals()
- merge_parameters()
- is_constructor_overridden() / store_original_constructor() / restore_original_constructor()
- create_overridden_constructor()
- is_method_overridden() / store_original_method() / restore_original_method()
- create_overridden_method()
- start_override_context() / end_override_context()
- cleanup_override() / cleanup_all_overrides()
- override_context() [context manager]
- register_framework_target()
- _create_override_constructor()
- _create_override_method()
- _apply_method_override()
- activate_overrides()
- deactivate_overrides()
- register_parameter_mapping()
- register_method_mapping()
- discover_parameter_mappings()
- get_method_from_target_class()
- enable_framework_overrides()
- disable_framework_overrides()
- register_mock_classes()
- validate_target_class()
- _discover_and_register_from_plugins()
Features: - Dynamic framework parameter discovery - Sync/async wrapper creation - Constructor and method-level overrides - Context-aware override management - Version compatibility handling - Parameter mapping and validation - Mock class registration - Graceful restoration of originals
Hooks/Extension Points:
- register_framework_target() - Register new frameworks
- register_parameter_mapping() - Add custom parameter mappings
- register_mock_classes() - Provide mock implementations
- Enhanced feature flags for discovery and validation
2. base.py (313 lines) - BASE OVERRIDE MANAGER¶
Purpose: Foundation for framework override operations
Key Classes:
- BaseOverrideManager - Abstract base for override operations
Key Methods:
- __init__()
- is_override_active() / set_override_active()
- extract_config_dict()
- get_method_wrapper()
- create_sync_wrapper()
- create_async_wrapper()
- store_original()
- restore_originals()
- merge_parameters()
- is_constructor_overridden() / store_original_constructor() / restore_original_constructor()
- create_overridden_constructor()
- is_method_overridden() / store_original_method() / restore_original_method()
- create_overridden_method()
- start_override_context() / end_override_context()
- cleanup_override() / cleanup_all_overrides()
- override_context() [context manager]
Features: - Thread-safe override state management - Constructor/method storage and restoration - Config extraction and parameter merging - Context manager support for scoped overrides
3. base_plugin.py (441 lines) - PLUGIN INTERFACE & REGISTRY¶
Purpose: Abstract base class and utilities for integration plugins
Key Classes:
- IntegrationPriority (Enum) - Plugin priority levels (LOW, NORMAL, HIGH, CRITICAL)
- PluginMetadata - Plugin metadata container
- ValidationRule - Parameter validation rules
- IntegrationPlugin (ABC) - Abstract base for all plugins
Key Methods (IntegrationPlugin):
- __init__(config_path)
- @abstractmethod _get_metadata()
- @abstractmethod _get_default_mappings()
- @abstractmethod _get_validation_rules()
- @abstractmethod get_target_classes()
- @abstractmethod get_target_methods()
- _normalize_config()
- get_parameter_mappings()
- validate_config()
- apply_overrides()
- _load_config_overrides()
- is_compatible()
- _is_version_supported()
- enable() / disable()
Features: - Plugin metadata management - Parameter mapping definition - Validation rule system - YAML/JSON configuration loading - Version compatibility checking - Enable/disable plugins dynamically - Custom validator support
4. plugin_registry.py (391 lines) - CENTRAL PLUGIN REGISTRY¶
Purpose: Singleton registry for managing all integration plugins
Key Classes:
- PluginRegistry - Singleton plugin management system
Key Methods:
- __new__() / __init__() [Singleton pattern]
- _discover_builtin_plugins()
- register()
- unregister()
- get_plugin()
- get_plugin_for_class()
- get_plugins_for_package()
- list_all_plugins()
- list_plugins_by_package()
- _validate_plugin_metadata()
- _build_lookup_indices()
Built-in Plugins Discovered:
- traigent.integrations.llms.openai_plugin.OpenAIPlugin
- traigent.integrations.llms.langchain_plugin.LangChainPlugin
- traigent.integrations.llms.anthropic_plugin.AnthropicPlugin
- traigent.integrations.llms.llamaindex_plugin.LlamaIndexPlugin
Features: - Auto-discovery of built-in plugins - Plugin registration/unregistration - Class-to-plugin mapping - Package-to-plugin mapping - Configuration file support
Provider Integration Files¶
5. openai_plugin.py (264 lines)¶
Framework: OpenAI SDK
Supported Packages: openai>=1.0.0
Parameter Mappings (Sample):
model -> model
temperature -> temperature
max_tokens -> max_tokens
top_p -> top_p
frequency_penalty -> frequency_penalty
presence_penalty -> presence_penalty
stop -> stop
stream -> stream
tools -> tools
tool_choice -> tool_choice
response_format -> response_format
seed -> seed
logit_bias -> logit_bias
n -> n
timeout -> timeout
max_retries -> max_retries
Supported Models: - gpt-4.1 (all variants: 0314, 0613, 32k, turbo, 1106) - gpt-4.1, gpt-4.1-mini - gpt-4o, gpt-4o-mini (legacy) - text-davinci-003, text-davinci-002
Methods Overridden: - Constructor (init) - chat.completions.create() - completions.create()
6. anthropic_plugin.py (263 lines)¶
Framework: Anthropic SDK
Supported Packages: anthropic>=0.3.0
Parameter Mappings (Sample):
model -> model
max_tokens -> max_tokens
temperature -> temperature
top_p -> top_p
top_k -> top_k
stop_sequences -> stop_sequences
stream -> stream
system -> system
messages -> messages
metadata -> metadata
tools -> tools
tool_choice -> tool_choice
timeout -> timeout
max_retries -> max_retries
anthropic_api_key -> api_key
anthropic_api_url -> base_url
Supported Models: - claude-opus-4-5-20251101 - claude-3-sonnet-20240229 - claude-haiku-4-5-20251001 - claude-2.1, claude-2.0 - claude-instant-1.2, claude-instant-1.1
Methods Overridden: - Constructor (init) - messages.create() - messages.stream()
7. langchain_plugin.py (393 lines)¶
Framework: LangChain (with dynamic discovery)
Supported Packages:
- langchain>=0.1.0
- langchain-core>=0.1
- langchain, langchain-openai, langchain-anthropic, langchain-google, langchain-aws, langchain-huggingface, langchain-community
Parameter Mappings (Extensive - 50+ parameters):
# Core LLM parameters
model -> model_name
temperature -> temperature
max_tokens -> max_tokens
top_p -> top_p
frequency_penalty -> frequency_penalty
presence_penalty -> presence_penalty
stop -> stop
streaming -> streaming
stream -> streaming
# LangChain-specific
verbose -> verbose
callbacks -> callbacks
tags -> tags
metadata -> metadata
run_name -> run_name
memory -> memory
return_messages -> return_messages
max_tokens_limit -> max_tokens_limit
# Agent parameters
agent_type -> agent_type
tools -> tools
handle_parsing_errors -> handle_parsing_errors
max_iterations -> max_iterations
max_execution_time -> max_execution_time
early_stopping_method -> early_stopping_method
# Chain parameters
chain_type -> chain_type
return_intermediate_steps -> return_intermediate_steps
include_run_info -> include_run_info
# Retrieval/RAG
k -> k
fetch_k -> fetch_k
lambda_mult -> lambda_mult
filter -> filter
search_type -> search_type
score_threshold -> score_threshold
# Embedding
embedding_model -> model_name
chunk_size -> chunk_size
chunk_overlap -> chunk_overlap
Target Classes (Dynamic discovery): - langchain_openai.ChatOpenAI - langchain_openai.OpenAI - langchain_anthropic.ChatAnthropic - langchain.llms.OpenAI (legacy) - langchain.llms.Anthropic (legacy) - And all discovered LangChain components
Features: - Dynamic component discovery - Support for chains, agents, retrieval components - RAG-specific parameter mapping - Chain and memory configuration - Provider-specific API key mapping
8. llamaindex_plugin.py (351 lines)¶
Framework: LlamaIndex (formerly GPT Index)
Supported Packages: llama-index>=0.8.0
Target Classes:
# Core LLM classes
llama_index.llms.openai.OpenAI
llama_index.llms.anthropic.Anthropic
llama_index.llms.cohere.Cohere
llama_index.llms.huggingface.HuggingFaceLLM
llama_index.llms.replicate.Replicate
llama_index.llms.palm.PaLM
# Legacy imports
llama_index.llms.OpenAI
llama_index.llms.Anthropic
llama_index.llms.Cohere
# Embedding models
llama_index.embeddings.openai.OpenAIEmbedding
llama_index.embeddings.huggingface.HuggingFaceEmbedding
llama_index.embeddings.cohere.CohereEmbedding
llama_index.embeddings.langchain.LangchainEmbedding
# Service context
llama_index.core.ServiceContext
llama_index.ServiceContext
# Chat engines
llama_index.chat_engine.SimpleChatEngine
llama_index.chat_engine.CondenseQuestionChatEngine
llama_index.chat_engine.ReActChatEngine
Target Methods (Sample):
OpenAI LLM:
- complete()
- acomplete()
- chat()
- achat()
- stream_complete()
- stream_chat()
Embeddings:
- get_text_embedding()
- get_text_embeddings()
- get_query_embedding()
Chat Engines:
- chat()
- stream_chat()
Features: - Multiple LLM provider support (OpenAI, Anthropic, Cohere, HuggingFace) - Embedding model optimization - Service context configuration - Chat engine parameter injection - Legacy version support
9. openai.py (303 lines) - OpenAI SDK Integration¶
Purpose: Enhanced OpenAI SDK integration with advanced features
Key Classes:
- OpenAIIntegration - OpenAI SDK integration handler
Supported Clients: - openai.OpenAI (sync) - openai.AsyncOpenAI (async)
Methods:
- __init__()
- _register_mappings()
- _normalize_client_types()
- enable_openai_overrides()
- disable_openai_overrides()
- get_supported_clients()
- override_client_methods()
- enable_streaming_support()
- enable_tools_support()
Features: - Sync and async client support - Streaming completions - Tool/function calling - Chat completions vs text completions - Client method override - Type validation
Utility & Support Files¶
10. utils/discovery.py (703 lines)¶
Purpose: Dynamic parameter discovery from any framework
Key Classes:
- ParameterDiscovery - Dynamic parameter extraction and mapping
Key Methods:
@staticmethod:
- discover_init_parameters(cls) -> Dict[str, Parameter]
- discover_method_parameters(obj, method_path) -> Dict[str, Parameter]
- find_similar_parameters(source_param, target_params, threshold) -> Optional[str]
- create_universal_mapping(traigent_param) -> List[str]
- auto_map_parameters(params, targets) -> Dict[str, str]
Features: - Inspect-based parameter discovery - Fuzzy matching for parameter names - Common parameter variation detection - Universal parameter mapping - Similarity scoring
Smart Mappings:
model -> [model, model_name, model_id, engine]
temperature -> [temperature, temp]
max_tokens -> [max_tokens, max_length, max_new_tokens, max_tokens_to_sample]
top_p -> [top_p, top_p_sampling]
stop -> [stop, stop_sequences, stop_words]
stream -> [stream, streaming]
11. utils/validation.py (314 lines)¶
Purpose: Parameter validation and type conversion
Key Classes:
- ParameterValidator - Type and value validation
- VersionCompatibilityManager - Version-aware parameter handling
Key Methods (ParameterValidator):
- validate_type()
- validate_value()
- validate_parameter()
- convert_type()
- sanitize_parameters()
- apply_constraints()
Key Methods (VersionCompatibilityManager):
- detect_version()
- get_version_mapping()
- migrate_parameters()
- is_parameter_deprecated()
- get_migration_path()
Features: - Type checking and conversion - Value constraint validation - Deprecated parameter warnings - Version-specific parameter mappings - Parameter sanitization
12. utils/version_compat.py¶
Purpose: Version compatibility utilities
Features: - Version detection - Compatibility checking - Deprecation warnings - Parameter migration paths
13. config.py (388 lines)¶
Purpose: Integration system configuration
Key Classes:
- IntegrationConfig - Global integration configuration
- ParameterConstraints - Parameter constraint definitions
- FrameworkConstraints - Framework constraint definitions
- _IntegrationConfigModule - Module-level configuration proxy
Configuration Options:
# Discovery settings
auto_discover: bool = True
discovery_cache_ttl: int = 3600
cache_discovered_classes: bool = True
# Override behavior
strict_mode: bool = False
fuzzy_matching_enabled: bool = True
fuzzy_matching_threshold: float = 0.8
# Validation
validate_types: bool = True
validate_values: bool = True
auto_convert_types: bool = True
# Compatibility
version_check: bool = True
warn_on_deprecated: bool = True
auto_migrate_parameters: bool = True
# Performance
max_fallback_attempts: int = 4
log_override_details: bool = False
14. Observability Integrations¶
observability/mlflow.py (558 lines)¶
Purpose: MLflow experiment tracking integration
Key Classes:
- TraigentMLflowTracker - MLflow tracking wrapper
- MLflowExperimentManager - Experiment lifecycle management
Key Functions:
- create_mlflow_tracker()
- enable_mlflow_autolog()
- log_traigent_optimization()
- get_best_traigent_run()
- compare_traigent_runs()
Features: - Experiment parameter logging - Trial metrics tracking - Optimization results storage - Run comparison and analysis - Auto-logging support
observability/wandb.py (680 lines)¶
Purpose: Weights & Biases experiment tracking
Key Classes:
- TraigentWandBTracker - W&B tracking wrapper
- WandBIntegration - W&B integration handler
Key Functions:
- create_wandb_tracker()
- init_wandb_run()
- enable_wandb_autolog()
- log_traigent_optimization()
- log_trial_to_wandb()
- log_final_results_to_wandb()
- create_wandb_sweep_config()
Features: - Run configuration logging - Trial metric tracking - Hyperparameter sweep support - Final result reporting - Auto-logging capability
Provider Client Wrappers (Thin Adapters)¶
15. bedrock_client.py (338 lines)¶
Purpose: AWS Bedrock Claude chat wrapper
Key Classes:
- BedrockChatResponse - Response dataclass
- BedrockChatClient - Bedrock client wrapper
Methods:
- __init__(region_name, profile_name, client)
- _ensure_client()
- invoke(model, messages, max_tokens, temperature, extra_params)
- invoke_stream(model, messages, max_tokens, temperature, extra_params)
Features:
- Sync and streaming interfaces
- boto3 optional import
- Mock mode support through TRAIGENT_MOCK_LLM=true or the in-code quickstart helper. Provider-specific *_MOCK environment variables are ignored.
- Anthropic Messages API format
LangChain Sub-integrations¶
16. llms/langchain/base.py (263 lines)¶
Purpose: LangChain base integration utilities
Key Classes:
- LangChainIntegration - LangChain integration handler
Supported LLMs:
langchain_openai.ChatOpenAI
langchain_openai.OpenAI
langchain_anthropic.ChatAnthropic
langchain.llms.OpenAI (legacy)
langchain.llms.Anthropic (legacy)
19. llms/langchain/discovery.py (310 lines)¶
Purpose: Auto-discovery of LangChain components
Key Classes:
- LangChainDiscovery - Component discovery engine
Known Packages Scanned:
langchain
langchain_community
langchain_openai
langchain_anthropic
langchain_google_genai
langchain_cohere
langchain_huggingface
langchain_aws
langchain_azure
langchain_mistralai
langchain_ollama
langchain_together
langchain_groq
langchain_nvidia
Methods:
- discover_all_llms() -> List[type]
- discover_all_embeddings() -> List[type]
- discover_all_retrievers() -> List[type]
- discover_all_chains() -> List[type]
- discover_all_agents() -> List[type]
- discover_component(component_type, package_name) -> List[type]
Vector Stores¶
20. vector_stores/init.py (5 lines)¶
Purpose: Vector store integration package
Status: Implementations are present for ChromaDB, Pinecone, and Weaviate, with a shared base plugin.
Main Package Exports¶
21. init.py (212 lines)¶
Purpose: Public API for integrations module
Exported Functions/Classes:
Framework Override:
- FrameworkOverrideManager
- enable_framework_overrides()
- disable_framework_overrides()
- override_context()
- register_framework_mapping()
- override_openai_sdk()
- override_langchain()
- override_anthropic()
- override_cohere()
- override_huggingface()
- override_all_platforms()
MLflow (optional):
- TraigentMLflowTracker
- create_mlflow_tracker()
- enable_mlflow_autolog()
- log_traigent_optimization()
- compare_traigent_runs()
- get_best_traigent_run()
Weights & Biases (optional):
- TraigentWandBTracker
- init_wandb_run()
- log_traigent_optimization()
- log_trial_to_wandb()
- log_final_results_to_wandb()
- create_wandb_tracker()
- enable_wandb_autolog()
- create_wandb_sweep_config()
- WandBIntegration
LangChain (optional):
- enable_langchain_optimization()
- get_supported_langchain_llms()
- add_langchain_llm_mapping()
- enable_chatgpt_optimization()
- enable_claude_optimization()
- auto_detect_langchain_llms()
OpenAI SDK (optional):
- enable_openai_optimization()
- get_supported_openai_clients()
- enable_sync_openai()
- enable_async_openai()
- openai_context()
- auto_detect_openai()
- enable_streaming_optimization()
- enable_tools_optimization()
Method Feature Matrix¶
Sync Methods Supported¶
| Integration | invoke | call | complete | chat |
|---|---|---|---|---|
| OpenAI | ✅ | N/A | ✅ | ✅ |
| Anthropic | ✅ | N/A | N/A | ✅ |
| LangChain | ✅ | ✅ | ✅ | ✅ |
| LlamaIndex | ✅ | ✅ | ✅ | ✅ |
| Bedrock | ✅ | N/A | N/A | N/A |
| Azure | ✅ | N/A | N/A | N/A |
| Gemini | ✅ | N/A | N/A | N/A |
Async Methods Supported¶
| Integration | ainvoke | async_call | acomplete | achat |
|---|---|---|---|---|
| OpenAI | ✅ | N/A | ✅ | ✅ |
| Anthropic | ✅ | N/A | N/A | ✅ |
| LangChain | ✅ | ✅ | ✅ | ✅ |
| LlamaIndex | ✅ | ✅ | ✅ | ✅ |
| Bedrock | ⚠️ | N/A | N/A | N/A |
| Azure | ✅* | N/A | N/A | N/A |
| Gemini | ✅* | N/A | N/A | N/A |
Streaming Methods Supported¶
| Integration | stream | astream | stream_complete | stream_chat |
|---|---|---|---|---|
| OpenAI | ✅ | ✅ | ✅ | ✅ |
| Anthropic | ✅ | ✅ | N/A | N/A |
| LangChain | ✅ | ✅ | ✅ | ✅ |
| LlamaIndex | ✅ | ✅ | ✅ | ✅ |
| Bedrock | ✅ | N/A | N/A | N/A |
| Azure | ⚠️ | ⚠️ | N/A | N/A |
| Gemini | ⚠️ | ⚠️ | N/A | N/A |
Batch Methods Supported¶
| Integration | batch | abatch |
|---|---|---|
| OpenAI | ✅ | ✅ |
| LangChain | ✅ | ✅ |
| LlamaIndex | ✅ | ✅ |
| Others | ⚠️ | ⚠️ |
Tool/Function Calling¶
| Integration | tools | tool_choice | function_call |
|---|---|---|---|
| OpenAI | ✅ | ✅ | ✅ |
| Anthropic | ✅ | ✅ | N/A |
| LangChain | ✅ | ✅ | ✅ (via OpenAI) |
| LlamaIndex | ✅ | ✅ | ✅ |
| Others | ⚠️ | ⚠️ | ⚠️ |
Vector DB / Embeddings¶
| Integration | query | similarity_search | get_embedding |
|---|---|---|---|
| LangChain | ✅ | ✅ | ✅ |
| LlamaIndex | ✅ | ✅ | ✅ |
| Others | ❌ | ❌ | ❌ |
Known Gaps & TODOs¶
Implemented Plugin Families¶
The current integrations tree includes dedicated plugins for Cohere and HuggingFace, plus vector-store plugins for ChromaDB, Pinecone, and Weaviate. Do not treat those families as missing when planning integration work.
Partial Implementations¶
- Bedrock Async Support - Limited async/await integration
- Azure OpenAI Streaming - Streaming support incomplete
- Gemini Streaming - Streaming support incomplete
Feature Gaps¶
- Batch Endpoints - Limited batch processing support for some providers
- Vision/Multimodal - No vision parameter handling (OpenAI vision, Claude vision)
- Structured Output - No JSON mode / structured output support
- Custom Endpoints - Limited custom endpoint override support
Documentation Gaps¶
- No TODO/FIXME comments found in integration code (clean!)
- Some complex logic lacks inline documentation
- Plugin contribution guide missing
Enhancement Opportunities¶
High Priority¶
- Keep provider-specific plugin docs synchronized with
traigent/integrations/llms/. - Add streaming support documentation.
Medium Priority¶
- Add vision/multimodal parameter support
- Implement structured output (JSON mode) support
- Add custom endpoint configuration
- Enhance batch processing support
Low Priority¶
- Create community plugin marketplace integration
- Add plugin auto-generation from API specs
- Implement hot-reload for plugins
- Add plugin composition support
Testing Status¶
Test Coverage: Estimated ~60% - Core framework override: ✅ Well tested - Plugin system: ✅ Good coverage - Individual plugins: ⚠️ Partial coverage - Observability: ✅ Adequate coverage - Edge cases: ⚠️ Needs improvement
Known Test Issues: None found in integration files
Summary Statistics¶
| Metric | Value |
|---|---|
| Total Python Files | 27 |
| Total Lines of Code | ~8,241 |
| Main Integration Files | 7 |
| Plugin Files | 4 |
| Utility Files | 5 |
| Client Wrappers | 3 |
| Observability | 2 |
| Supported Providers | 7+ |
| Framework Variants | 20+ |
| Parameter Mappings | 100+ |
| Class Overrides | 40+ |
| Method Overrides | 50+ |