Merge pull request #54 from mainframecomputer/sayak/mfe-440-add-better-onoff-switch-to-orchestra-braintrust-logging

Add configurable on/off switch for Braintrust integration

Author: sayakmaity

docs/src/observability.md

@@ -0,0 +1,34 @@
+# Observability
+
+### Braintrust Integration
+
+##### How do I enable or disable Braintrust integration?
+
+Orchestra has built-in support for [Braintrust](https://www.braintrust.dev/), which provides observability and evaluation for LLM applications.
+
+To enable Braintrust integration:
+1. Install the `braintrust` package: `pip install braintrust`
+2. Set your Braintrust API key as an environment variable:
+   ```
+   export BRAINTRUST_API_KEY=your_api_key
+   ```
+3. By default, Braintrust is automatically enabled when the API key is present
+
+To explicitly enable Braintrust integration:
+```
+export BRAINTRUST_ORCHESTRA_ENABLED=true
+```
+
+To disable Braintrust integration inside Orchestra:
+```
+export BRAINTRUST_ORCHESTRA_ENABLED=false
+```
+
+You can also use `0` or `no` as values to disable it, or `1` or `yes` to enable it. Braintrust integration is automatically enabled when:
+1. The `braintrust` package is installed
+2. The `BRAINTRUST_API_KEY` environment variable is set with a valid API key
+3. The `BRAINTRUST_ORCHESTRA_ENABLED` environment variable is either:
+   - Set to enable it (`true`, `1`, or `yes`)
+   - Not set at all (defaults to enabled when API key exists)
+
+When enabled, Braintrust will trace all tool calls and requests to OpenAI, OpenRouter, Groq, Together AI, and Deepseek, providing detailed logs and analytics in your Braintrust dashboard.

packages/python/src/mainframe_orchestra/utils/braintrust_utils.py

@@ -5,17 +5,32 @@
 This module provides fallback decorators when Braintrust is not available.
 """
 
-try:
-    from braintrust import traced, wrap_openai
-    BRAINTRUST_AVAILABLE = True
-except ImportError:
-    BRAINTRUST_AVAILABLE = False
-    def traced(type=None):
-        """No-op decorator when Braintrust is not available"""
-        def decorator(func):
-            return func
+import os
+
+# Check if Braintrust integration is explicitly enabled or disabled via environment variable.
+# Default to enabled if API key is present and no explicit setting
+BRAINTRUST_API_KEY_EXISTS = os.environ.get("BRAINTRUST_API_KEY", "") != ""
+BRAINTRUST_ENABLED = os.environ.get("BRAINTRUST_ORCHESTRA_ENABLED", "").lower() in ("true", "1", "yes") or (
+    BRAINTRUST_API_KEY_EXISTS and os.environ.get("BRAINTRUST_ORCHESTRA_ENABLED", None) is None
+)
+
+# Default implementation of no-op decorators
+def traced(func=None, **kwargs):
+    """No-op decorator when Braintrust is not available"""
+    if func is None:
+        def decorator(f):
+            return f
         return decorator
-    
-    def wrap_openai(func):
-        """No-op decorator when Braintrust is not available"""
-        return func
+    return func
+
+def wrap_openai(func):
+    """No-op decorator when Braintrust is not available"""
+    return func
+
+# Try to import Braintrust if enabled
+if BRAINTRUST_ENABLED:
+    try:
+        from braintrust import traced, wrap_openai
+    except ImportError:
+        # Keep the no-op implementations defined above
+        pass