* fix(devui): add created_at to custom output item events for correct workflow timings (#5545)
CustomResponseOutputItemAddedEvent and CustomResponseOutputItemDoneEvent lacked a
created_at field, causing the frontend to synthesize timestamps using integer-second
precision with a forced +1s minimum gap between events. This made instant workflows
appear to take 3+ seconds in the DevUI timeline.
Fix:
- Add optional created_at: float | None field to both custom event models
- Populate created_at=float(time.time()) in the mapper for executor_invoked,
executor_completed, and executor_failed events
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(devui): use event created_at for accurate workflow timeline timings
workflow-view.tsx synthesized _uiTimestamp using Math.max(baseTimestamp,
lastTimestamp + 1) with integer-second precision, forcing a minimum 1-second
gap between every sequential event. This made instant workflows appear to take
several seconds in the DevUI timeline.
The fix prefers event.created_at (a float Unix timestamp populated by the
backend mapper for all executor events) and only falls back to the synthetic
timestamp when created_at is absent. This matches the pattern already used in
devuiStore.ts:addDebugEvent.
Added a regression test in test_mapper.py verifying that the mapper attaches
created_at to all executor lifecycle events (invoked, completed, failed).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(devui): address review feedback for issue #5545
- Read data.timestamp (ISO string) and response.created_at in addition
to top-level created_at when deriving _uiTimestamp, so
response.workflow_event.completed events get a real server timestamp
instead of a synthesized one
- Change uniqueTimestamp tiebreaker: when a real server timestamp is
available use Math.max(eventTimestamp, lastTimestamp) rather than
lastTimestamp + 1, eliminating artificial 1-second gaps while still
preserving monotonic ordering
- Apply the same fix in the HIL streaming path (second setOpenAIEvents
call in workflow-view.tsx)
- Add assert event.created_at > 0 to regression test to guard against
zero or negative timestamps
- Add test_custom_output_item_event_models_have_created_at_field model-
level test so removing the field produces a clear named failure rather
than a downstream ValidationError
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(#5545): guard NaN timestamps, fix fallback ID uniqueness, add regression tests
- workflow-view.tsx (×2): Wrap data.timestamp ISO→number conversion in a
Number.isFinite() guard. Python's datetime.now().isoformat() emits
microseconds without a trailing 'Z' (e.g. '2024-01-15T12:34:56.123456'),
which some JS engines cannot parse, returning NaN. NaN !== undefined is
true so the eventTimestamp !== undefined guard did not catch it, poisoning
_uiTimestamp and resetting the monotonic ordering seed (NaN || 0 → 0).
- execution-timeline.tsx: Replace uiTimestamp in the fallback syntheticItemId
with the per-executor runNumber counter. Two runs of the same executor
within the same second previously received identical _uiTimestamp values
and therefore identical syntheticItemIds, causing their output buckets,
state, and run entries to collide (execution-timeline.tsx:360–408).
- Add missing test_workflow_timings_bug.py source file (only a stale .pyc
existed). Three regression tests:
· test_custom_event_models_lack_created_at_field – model field guard
· test_workflow_executor_events_lack_created_at – mapper populates created_at
· test_rapid_workflow_events_have_no_top_level_timestamps – confirms
data.timestamp format that requires the frontend NaN guard
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #5545: Python: [Bug]: Workflow timings in DevUI are incorrect
* devui: move timing regression tests into test_mapper.py, remove dedicated bug file
- Delete test_workflow_timings_bug.py; tests belong in existing module files
- The two tests already present in test_mapper.py (test_executor_events_carry_created_at_timestamp
and test_custom_output_item_event_models_have_created_at_field) cover the same ground as the
first two tests in the deleted file
- Add test_executor_completed_maps_to_output_item_done_event to test_mapper.py, replacing the
third test from the deleted file with a generic, issue-agnostic name and docstring
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #5545: review comment fixes
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Refactor Anthropic model option and provider clients
Rename the Anthropic client model option from model_id to model, add provider-specific Anthropic wrappers for Foundry, Bedrock, and Vertex, and expose them through the Anthropic, Foundry, Amazon, and Google namespaces. Update core option handling, docs, samples, and tests accordingly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Anthropic skills sample typing
Cast the Anthropic beta client to Any in the skills sample so the pre-commit sample pyright check no longer fails on beta skills and files endpoints that are not exposed by the current SDK stubs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* undo sample mypy
* Retry CI after transient external failures
Retrigger PR validation after an unrelated Copilot review workflow SAML failure and a transient external tau2 git fetch failure in the Windows Python test setup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback on model option merging
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Anthropic compatibility review feedback
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* moved all to `model`
* fixes for azure ai search
* Python: standardize remaining sample env var names
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: fix foundry-local pyright compatibility
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated env vars in cicd
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Prepare azure-ai-projects 2.0 GA compatibility
Add allow_preview support for internal AIProjectClient creation, keep backward compatibility for renamed SDK model classes, and align Azure AI/core paths and tests for GA validation workflows.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* upgrade to ai-project==2.0.0
* Python: remove azure-ai-projects keyword-guard paths
Assume azure-ai-projects 2.0+ in Azure AI client/provider/responses code paths by removing _supports_keyword_argument gating and related fallback branching.
Also fix pyright typing in FoundryMemoryProvider memory store calls by using ResponseInputItemParam-typed items.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* check fixes
* Python: remove unsupported foundry_features option
Drop foundry_features from Azure AI client and provider surfaces because azure-ai-projects 2.0.0 does not expose that create_version parameter.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: add allow_preview to Foundry memory provider
Propagate allow_preview when FoundryMemoryProvider constructs an AIProjectClient and update tests accordingly.
Also finish wiring allow_preview through AzureAIClient-facing surfaces and related docs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* aligning docstrings
* udpated lock
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Python pyright package scoping and typing remediation
Implements issue #4407 by removing the root pyright include, adding package-level pyright includes, and resolving pyright/mypy typing issues across Python packages. Also cleans unnecessary casts and applies line-level, rule-specific ignores where external libraries are too dynamic.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Reduce pyright cost in handoff cloning
Simplify cloned_options construction in HandoffAgentExecutor to avoid expensive TypedDict narrowing/inference in _handoff.py, which was causing pyright to spend a long time in orchestrations.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix types
* Fix lint and type-check regressions
Resolve current Python package check failures across lint, pyright, and mypy after recent code changes, including purview/declarative pyright issues and multiple ruff simplification findings.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fixed hooks
* Stabilize package tests and test tasks
Resolve cross-package non-integration test failures, simplify streaming type flow, harden locale/culture handling, and standardize package test poe tasks to exclude integration tests where applicable.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* lots of small fixes
* Fix current Python test regressions
Address current failing unit tests in azure-ai, bedrock, and azure-cosmos while keeping Bedrock parsing logic inline (no new static helper methods).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* small fixes
* small fixes
* removed pydantic from json
* final updates
* fix core
* fix tests
* fix obser
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* removed display_name, renamed context_providers, middleware and AggregateContextProvider
* fixes
* fixed test
* testfix
* removed mistakenly put back test
* updated new test
* rename middlewares to middleware
* middleware fixes
* Improve DevUI, add Context Inspector view as new tab under traces
* fix mypy errors
* fix: Handle stale MCP connections in DevUI executor
MCP tools can become stale when HTTP streaming responses end - the underlying
stdio streams close but `is_connected` remains True. This causes subsequent
requests to fail with `ClosedResourceError`.
Add `_ensure_mcp_connections()` to detect and reconnect stale MCP tools before
agent execution. This is a workaround for an upstream Agent Framework issue
where connection state isn't properly tracked.
Fixes MCP tools failing on second HTTP request in DevUI.
fixes #1476#1515#2865
* fix#1572 report import dependency errors more clearly
* Ensure there is streaming toggle where users can select streaming vs non streaming mode in devui . Fixes .NET: [Python] DevUI tool call rendering in non-streaming mode?
* remove unused dead code
* improve ux - workflows with agents show a chat component in execution timelien, also ensure magentic final output shows correctly
* update ui build
* update devui to use instrumentation instead of tracing, other instrumentation and type/instance check fixes
* refactoring and unifying naming schemes of internal methods of chat clients
* set tool_choice to auto
* fix for mypy
* added note on naming and fix#2951
* fix responses
* fixes in azure ai agents client
* DevUI: Add OpenAI Responses API proxy support with enhanced UI features
This commit adds support for proxying requests to OpenAI's Responses API,
allowing DevUI to route conversations to OpenAI models when configured to enable testing.
Backend changes:
- Add OpenAI proxy executor with conversation routing logic
- Enhance event mapper to support OpenAI Responses API format
- Extend server endpoints to handle OpenAI proxy mode
- Update models with OpenAI-specific response types
- Remove emojis from logging and CLI output for cleaner text
Frontend changes:
- Add settings modal with OpenAI proxy configuration UI
- Enhance agent and workflow views with improved state management
- Add new UI components (separator, switch) for settings
- Update debug panel with better event filtering
- Improve message renderers for OpenAI content types
- Update types and API client for OpenAI integration
* update ui, settings modal and workflow input form, add register cleanup hooks.
* add workflow HIL support, user mode, other fixes
* feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas
Implement HIL workflow support allowing workflows to pause for user input
with dynamically generated JSON schemas based on response handler type hints.
Key Features:
- Automatic response schema extraction from @response_handler decorators
- Dynamic form generation in UI based on Pydantic/dataclass response types
- Checkpoint-based conversation storage for HIL requests/responses
- Resume workflow execution after user provides HIL response
Backend Changes:
- Add extract_response_type_from_executor() to introspect response handlers
- Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema()
- Map RequestInfoEvent to response.input.requested OpenAI event format
- Store HIL responses in conversation history and restore checkpoints
Frontend Changes:
- Add HILInputModal component with SchemaFormRenderer for dynamic forms
- Support Pydantic BaseModel and dataclass response types
- Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects
- Display original request context alongside response form
Testing:
- Add tests for checkpoint storage (test_checkpoints.py)
- Add schema generation tests for all input types (test_schema_generation.py)
- Validate end-to-end HIL flow with spam workflow sample
This enables workflows to seamlessly pause execution and request structured user input
with type-safe, validated forms generated automatically from response type annotations.
* improve HIL support, improve workflow execution view
* ui updates
* ui updates
* improve HIL for workflows, add auth and view modes
* update workflow
* security improvements , ui fixes
* fix mypy error
* update loading spinner in ui
* DevUI: Serialize workflow input as string to maintain conformance with OpenAI Responses format
* Phase 1: Add /meta endpoint and fix workflow event naming for .NET DevUI compatibility
* additional fixes for .NET DevUI workflow visualization item ID tracking
**Problem:**
.NET DevUI was generating different item IDs for ExecutorInvokedEvent and
ExecutorCompletedEvent, causing only the first executor to highlight in the
workflow graph. Long executor names and error messages also broke UI layout.
**Changes:**
- Add ExecutorActionItemResource to match Python DevUI implementation
- Track item IDs per executor using dictionary in AgentRunResponseUpdateExtensions
- Reuse same item ID across invoked/completed/failed events for proper pairing
- Add truncateText() utility to workflow-utils.ts
- Truncate executor names to 35 chars in execution timeline
- Truncate error messages to 150 chars in workflow graph nodes
** Details:**
- ExecutorActionItemResource registered with JSON source generation context
- Dictionary cleaned up after executor completion/failure to prevent memory leaks
- Frontend item tracking by unique item.id supports multiple executor runs
- All changes follow existing codebase patterns and conventions
Tested with review-workflow showing correct executor highlighting and state
transitions for sequential and concurrent executors.
* format fixes, remove cors tests
* remove unecessary attributes
---------
Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
Co-authored-by: Reuben Bond <reuben.bond@gmail.com>
* DevUI: Add OpenAI Responses API proxy support with enhanced UI features
This commit adds support for proxying requests to OpenAI's Responses API,
allowing DevUI to route conversations to OpenAI models when configured to enable testing.
Backend changes:
- Add OpenAI proxy executor with conversation routing logic
- Enhance event mapper to support OpenAI Responses API format
- Extend server endpoints to handle OpenAI proxy mode
- Update models with OpenAI-specific response types
- Remove emojis from logging and CLI output for cleaner text
Frontend changes:
- Add settings modal with OpenAI proxy configuration UI
- Enhance agent and workflow views with improved state management
- Add new UI components (separator, switch) for settings
- Update debug panel with better event filtering
- Improve message renderers for OpenAI content types
- Update types and API client for OpenAI integration
* update ui, settings modal and workflow input form, add register cleanup hooks.
* add workflow HIL support, user mode, other fixes
* feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas
Implement HIL workflow support allowing workflows to pause for user input
with dynamically generated JSON schemas based on response handler type hints.
Key Features:
- Automatic response schema extraction from @response_handler decorators
- Dynamic form generation in UI based on Pydantic/dataclass response types
- Checkpoint-based conversation storage for HIL requests/responses
- Resume workflow execution after user provides HIL response
Backend Changes:
- Add extract_response_type_from_executor() to introspect response handlers
- Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema()
- Map RequestInfoEvent to response.input.requested OpenAI event format
- Store HIL responses in conversation history and restore checkpoints
Frontend Changes:
- Add HILInputModal component with SchemaFormRenderer for dynamic forms
- Support Pydantic BaseModel and dataclass response types
- Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects
- Display original request context alongside response form
Testing:
- Add tests for checkpoint storage (test_checkpoints.py)
- Add schema generation tests for all input types (test_schema_generation.py)
- Validate end-to-end HIL flow with spam workflow sample
This enables workflows to seamlessly pause execution and request structured user input
with type-safe, validated forms generated automatically from response type annotations.
* improve HIL support, improve workflow execution view
* ui updates
* ui updates
* improve HIL for workflows, add auth and view modes
* update workflow
* security improvements , ui fixes
* fix mypy error
* update loading spinner in ui
---------
Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
* ensure function aproval is parsed correctly
* udpate ui, add deployment guide button, other debug panel fixes
* feat(devui): Implement lazy loading architecture with enhanced security and state management
Major architectural improvements to DevUI for better performance, security, and developer experience:
Performance & Architecture:
- Implement lazy loading for entity discovery - entities loaded on-demand instead of at startup
- Add hot reload capability for development workflow via new reload endpoint
- Reduce startup time and memory footprint by deferring module imports
Security Enhancements:
- Remove remote entity loading capabilities (POST /v1/entities/add, DELETE endpoints)
- DevUI now strictly local development tool - no remote code execution
- Add explicit security documentation and best practices in README
Frontend Improvements:
- Migrate to Zustand for centralized state management (replacing prop drilling)
- Add lightweight zero-dependency markdown renderer with code block copy support
- Improve gallery UX with setup instructions modal instead of direct URL loading
- Enhanced message UI with copy functionality and better token usage display
Testing & Quality:
- Expand test coverage for lazy loading, type detection, and cache invalidation
- Add comprehensive tests for new behaviors (+231 lines of test code)
- Improve type safety and documentation throughout
Breaking Changes:
- Remote entity loading via URLs is no longer supported
- Entities must be loaded from local filesystem only
* update ui issues, uupdate test descripion
* Python: DevUI - Internal Refactor, Conversations API support, and performance improvements
Comprehensive refactor of DevUI package including samples relocation,
frontend reorganization, OpenAI Conversations API support, and critical
performance and code quality improvements.
Key Changes:
Architecture & Organization
- Moved DevUI samples to python/samples/getting_started/devui/
- Consolidated with other framework samples for better discoverability
- Added .env.example files and comprehensive README
- Restructured frontend components into feature-based folders (agent, workflow, gallery, layout)
- Created new OpenAI-compliant message renderers (devui should render oai responses types primarily)
New Features
- Added _conversations.py (467 lines) - Full conversation storage abstraction, replaces the /threads endpoint to better match oai conversations api
- Implements OpenAI Conversations API for thread management, Supports in-memory and extensible storage backends
API Simplification
- Use 'model' field as entity_id (agent/workflow name) instead of extra_body
- Use standard OpenAI 'conversation' field for conversation context.
Performance & Quality Improvements
- Improved context management in MessageMapper with bounded memory (~500KB max)
- Implemented hybrid LRU + cleanup approach to prevent unbounded memory growth
- General QOL improvement - Eliminated ~150 lines of dead/duplicate code, Consolidated helper functions into _utils.py, Extracted magic numbers to module-level constants, Optimized conversation item lookups with index-based approach
Testing
- Added test_conversations.py (13 tests)
- Added test_performance_fixes.py (9 tests)
- Updated existing tests for code consolidation
- 53 tests passing
Impact: 76 files changed: +4,106 insertions, -2,373 deletions
All linting and formatting checks passing. No breaking changes - backward compatible.
Migration: Samples moved to python/samples/getting_started/devui/
* readme lint fixes
* initial support for function approval and minor ui fixes
Victor DibiaandGitHub
Reuben BondandGitHub
