Python: DevUI: Add OpenAI Responses API proxy support + HIL for Workflows (#1737)

* DevUI: Add OpenAI Responses API proxy support with enhanced UI features

This commit adds support for proxying requests to OpenAI's Responses API,
allowing DevUI to route conversations to OpenAI models when configured to enable testing.

Backend changes:
- Add OpenAI proxy executor with conversation routing logic
- Enhance event mapper to support OpenAI Responses API format
- Extend server endpoints to handle OpenAI proxy mode
- Update models with OpenAI-specific response types
- Remove emojis from logging and CLI output for cleaner text

Frontend changes:
- Add settings modal with OpenAI proxy configuration UI
- Enhance agent and workflow views with improved state management
- Add new UI components (separator, switch) for settings
- Update debug panel with better event filtering
- Improve message renderers for OpenAI content types
- Update types and API client for OpenAI integration

* update ui, settings modal and workflow input form, add register cleanup hooks.

* add workflow HIL support, user mode, other fixes

* feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas

Implement  HIL workflow support allowing workflows to pause for user input
with dynamically generated JSON schemas based on response handler type hints.

Key Features:
- Automatic response schema extraction from @response_handler decorators
- Dynamic form generation in UI based on Pydantic/dataclass response types
- Checkpoint-based conversation storage for HIL requests/responses
- Resume workflow execution after user provides HIL response

Backend Changes:
- Add extract_response_type_from_executor() to introspect response handlers
- Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema()
- Map RequestInfoEvent to response.input.requested OpenAI event format
- Store HIL responses in conversation history and restore checkpoints

Frontend Changes:
- Add HILInputModal component with SchemaFormRenderer for dynamic forms
- Support Pydantic BaseModel and dataclass response types
- Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects
- Display original request context alongside response form

Testing:
- Add  tests for checkpoint storage (test_checkpoints.py)
- Add schema generation tests for all input types (test_schema_generation.py)
- Validate end-to-end HIL flow with spam workflow sample

This enables workflows to seamlessly pause execution and request structured user input
with type-safe, validated forms generated automatically from response type annotations.

* improve HIL support, improve workflow execution view

* ui updates

* ui updates

* improve HIL for workflows, add auth and view modes

* update workflow

* security improvements , ui fixes

* fix mypy error

* update loading spinner in ui

---------

Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
This commit is contained in:
Victor Dibia
2025-11-07 15:28:32 -08:00
committed by GitHub
Unverified
parent 85484c0259
commit 94eae24082
52 changed files with 10178 additions and 1599 deletions
@@ -4,13 +4,14 @@
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Literal
import pytest
# Add parent package to path
sys.path.insert(0, str(Path(__file__).parent.parent))
from agent_framework_devui._utils import generate_input_schema
from agent_framework_devui._utils import extract_response_type_from_executor, generate_input_schema
@dataclass
@@ -132,6 +133,99 @@ def test_schema_generation_error_handling():
pass
def test_extract_response_type_from_executor():
"""Test extraction of response type from @response_handler methods."""
try:
from agent_framework import Executor, WorkflowContext, handler, response_handler
from pydantic import BaseModel, Field
# Define test request and response types
@dataclass
class TestApprovalRequest:
"""Test request for approval."""
prompt: str
context: str
class TestDecision(BaseModel):
"""Test decision response."""
decision: Literal["approve", "reject"] = Field(description="User's decision")
reason: str = Field(description="Reason for decision", default="")
# Create test executor with @response_handler
class TestExecutor(Executor):
"""Test executor with response handler."""
def __init__(self):
super().__init__(id="test_executor")
@handler
async def handle_message(self, message: str, ctx: WorkflowContext) -> None:
"""Regular handler to satisfy executor requirements."""
# Request info that will be handled by response_handler
request = TestApprovalRequest(prompt="Test", context="Test context")
await ctx.request_info(request, TestDecision)
@response_handler
async def handle_approval(
self, original_request: TestApprovalRequest, response: TestDecision, ctx: WorkflowContext
) -> None:
"""Handle approval response."""
pass
# Test extraction
executor = TestExecutor()
extracted_type = extract_response_type_from_executor(executor, TestApprovalRequest)
# Verify correct type was extracted
assert extracted_type is not None, "Should extract response type from @response_handler"
assert extracted_type == TestDecision, f"Expected TestDecision, got {extracted_type}"
# Test full schema generation pipeline
schema = generate_input_schema(extracted_type)
assert schema is not None
assert isinstance(schema, dict)
assert "properties" in schema
assert "decision" in schema["properties"]
assert "enum" in schema["properties"]["decision"]
assert schema["properties"]["decision"]["enum"] == ["approve", "reject"]
except ImportError as e:
pytest.skip(f"Required dependencies not available: {e}")
def test_extract_response_type_no_match():
"""Test that extraction returns None when no matching handler exists."""
try:
from agent_framework import Executor, WorkflowContext, handler
@dataclass
class UnmatchedRequest:
"""Request type with no handler."""
data: str
class MinimalExecutor(Executor):
"""Executor with a handler but no matching response_handler."""
def __init__(self):
super().__init__(id="minimal_executor")
@handler
async def handle_message(self, message: str, ctx: WorkflowContext) -> None:
"""Regular handler."""
pass
executor = MinimalExecutor()
extracted_type = extract_response_type_from_executor(executor, UnmatchedRequest)
assert extracted_type is None, "Should return None when no matching handler exists"
except ImportError as e:
pytest.skip(f"Required dependencies not available: {e}")
if __name__ == "__main__":
# Simple test runner for manual execution
pytest.main([__file__, "-v"])