Commit Graph

16 Commits

  • Python: DevUI, Fix message serialization issue, improve tests (#2674)
    * fix message serialization issue, improve tests for devui
    
    * update tests
  • Python: DevUI fixes : Add multimodal input support for workflows and refactor chat input (#2593)
    * show app version in devui .NET: Python: Improved Versioning for DevUI
    Fixes #2059
    
    * feat: Add multimodal input support for workflows and refactor chat input
    
    This PR adds support for multimodal content (images, files) in workflow
    inputs and refactors the chat input into a reusable component.
    
    ## Multimodal Workflow Support
    - Add `isChatMessageSchema()` to detect ChatMessage input schemas
    - Update `RunWorkflowButton` to use `ChatMessageInput` for ChatMessage workflows
    - Wrap multimodal content in OpenAI message format for backend processing
    - Add `_is_openai_multimodal_format()` to detect OpenAI ResponseInputParam
    - Update `_parse_workflow_input()` to route multimodal input through
      existing `_convert_input_to_chat_message()` converter
    
    ## Reusable ChatMessageInput Component
    - Extract chat input logic from agent-view into `ChatMessageInput` component
    - Support file upload, drag & drop, paste handling, and attachments
    - Add `useDragDrop` hook for parent-level drag handling with full-area
      drop zones
    - Refactor agent-view to use the new shared component
    
    ## Other Improvements
    - Add `isStreaming` prop to executor nodes for animation control
    - Clean up unused imports and state variables in agent-view
    - Add tests for multimodal workflow input handling
    
    Fixes workflow input not receiving images when using AgentExecutor nodes.
    
    * add self loop edge, fix #2470
    
    * fix test
  • Python: make tool call view optional in DevUI + other link fixes (#2243)
    * make tool call view optional in devui + other link fixes
    
    * fix #2310, ensure correct port is shown in command
    
    * fix dialog bug
    
    * ensure executor ids are tracked per items, fix bug where data from concurrent executors where not seperated properly fix #2351
    
    * fix: Enable multi-round human-in-the-loop (HIL) in DevUI workflows
    
    - Backend: Enrich RequestInfoEvents with response schemas in send_responses_streaming path
    - Frontend: Replace old HIL requests with new ones instead of accumulating them
    - Frontend: Fix HIL response state management to prevent sending stale request responses
    
    This allows workflows to properly handle sequential HIL requests, showing only the
    current request to users and progressing through multiple input rounds correctly.
    
    fixes #2334
    
    * fix bug to ensure in memory entities cannot be reloaded in ui
  • Python: Add checkpoint save and restore hooks to executor (#2097)
    * Add checkpoint hooks
    
    * Deprecate get_executor_state and set_executor_state
    
    * Fix tests and samples
    
    * Add doc strings
    
    * Add sample
    
    * Fix import
    
    * Address comments and fix tests
    
    * Address comments
    
    * conditional import
  • Python: Fix HIL regression (#2167)
    * fix devui regression from #2021 where all input is stringified but devui HIL input does not handle stringified json strings correctly.
    
    * update incorrect test
    
    * add devui hil input tests
  • Python: DevUI: Add OpenAI Responses API proxy support + HIL for Workflows (#1737)
    * DevUI: Add OpenAI Responses API proxy support with enhanced UI features
    
    This commit adds support for proxying requests to OpenAI's Responses API,
    allowing DevUI to route conversations to OpenAI models when configured to enable testing.
    
    Backend changes:
    - Add OpenAI proxy executor with conversation routing logic
    - Enhance event mapper to support OpenAI Responses API format
    - Extend server endpoints to handle OpenAI proxy mode
    - Update models with OpenAI-specific response types
    - Remove emojis from logging and CLI output for cleaner text
    
    Frontend changes:
    - Add settings modal with OpenAI proxy configuration UI
    - Enhance agent and workflow views with improved state management
    - Add new UI components (separator, switch) for settings
    - Update debug panel with better event filtering
    - Improve message renderers for OpenAI content types
    - Update types and API client for OpenAI integration
    
    * update ui, settings modal and workflow input form, add register cleanup hooks.
    
    * add workflow HIL support, user mode, other fixes
    
    * feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas
    
    Implement  HIL workflow support allowing workflows to pause for user input
    with dynamically generated JSON schemas based on response handler type hints.
    
    Key Features:
    - Automatic response schema extraction from @response_handler decorators
    - Dynamic form generation in UI based on Pydantic/dataclass response types
    - Checkpoint-based conversation storage for HIL requests/responses
    - Resume workflow execution after user provides HIL response
    
    Backend Changes:
    - Add extract_response_type_from_executor() to introspect response handlers
    - Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema()
    - Map RequestInfoEvent to response.input.requested OpenAI event format
    - Store HIL responses in conversation history and restore checkpoints
    
    Frontend Changes:
    - Add HILInputModal component with SchemaFormRenderer for dynamic forms
    - Support Pydantic BaseModel and dataclass response types
    - Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects
    - Display original request context alongside response form
    
    Testing:
    - Add  tests for checkpoint storage (test_checkpoints.py)
    - Add schema generation tests for all input types (test_schema_generation.py)
    - Validate end-to-end HIL flow with spam workflow sample
    
    This enables workflows to seamlessly pause execution and request structured user input
    with type-safe, validated forms generated automatically from response type annotations.
    
    * improve HIL support, improve workflow execution view
    
    * ui updates
    
    * ui updates
    
    * improve HIL for workflows, add auth and view modes
    
    * update workflow
    
    * security improvements , ui fixes
    
    * fix mypy error
    
    * update loading spinner in ui
    
    ---------
    
    Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
  • Python: DevUI: Use metadata.entity_id instead of model field (#1984)
    * DevUI: Use metadata.entity_id for agent/workflow name instead of model field
    
    * OpenAI Responses: add explicit request validation
    
    * Review feedback
  • Python: DevUI Fix Serialization, Timestamp and Other Issues (#1584)
    * refactor(devui): adopt standard OpenAI lifecycle events for agents and workflows
    
    - Replace custom workflow events with OpenAI Responses API standard lifecycle events
    - Add AgentStartedEvent, AgentCompletedEvent, AgentFailedEvent for clean separation
    - Implement ExecutorActionItem for workflow executor tracking
    - Convert informational events to trace events to reduce noise
    - Update README mapper table with comprehensive event mappings
    - Maintain full backward compatibility with legacy events
    
    * fix(devui): resolve timestamp overwriting and Content serialization errors
    
    - Fix tool call timestamps being overwritten on each render (#1483)
    - Add recursive Content serialization to handle ChatMessage and nested objects (#1548)
    - Implement proper MCP tool cleanup on server shutdown
    - Add timestamp field to function_result.complete events
    - Enhance credential and client resource cleanup
    
    Fixes #1483, #1548
    Partial improvements for #1476
  • Python: Add Function Approval UI to DevUI (#1401)
    * ensure function aproval is parsed correctly
    
    * udpate ui, add deployment guide button, other debug panel fixes
    
    * feat(devui): Implement lazy loading architecture with enhanced security and state management
    
    Major architectural improvements to DevUI for better performance, security, and developer experience:
    
    Performance & Architecture:
    - Implement lazy loading for entity discovery - entities loaded on-demand instead of at startup
    - Add hot reload capability for development workflow via new reload endpoint
    - Reduce startup time and memory footprint by deferring module imports
    
    Security Enhancements:
    - Remove remote entity loading capabilities (POST /v1/entities/add, DELETE endpoints)
    - DevUI now strictly local development tool - no remote code execution
    - Add explicit security documentation and best practices in README
    
    Frontend Improvements:
    - Migrate to Zustand for centralized state management (replacing prop drilling)
    - Add lightweight zero-dependency markdown renderer with code block copy support
    - Improve gallery UX with setup instructions modal instead of direct URL loading
    - Enhanced message UI with copy functionality and better token usage display
    
    Testing & Quality:
    - Expand test coverage for lazy loading, type detection, and cache invalidation
    - Add comprehensive tests for new behaviors (+231 lines of test code)
    - Improve type safety and documentation throughout
    
    Breaking Changes:
    - Remote entity loading via URLs is no longer supported
    - Entities must be loaded from local filesystem only
    
    * update ui issues, uupdate test descripion
  • Python: DevUI - Internal Refactor, Conversations API support, and per… (#1235)
    * Python: DevUI - Internal Refactor, Conversations API support, and performance improvements
    
    Comprehensive refactor of DevUI package including samples relocation,
    frontend reorganization, OpenAI Conversations API support, and critical
    performance and code quality improvements.
    
    Key Changes:
    
    Architecture & Organization
    - Moved DevUI samples to python/samples/getting_started/devui/
    - Consolidated with other framework samples for better discoverability
    - Added .env.example files and comprehensive README
    - Restructured frontend components into feature-based folders (agent, workflow, gallery, layout)
    - Created new OpenAI-compliant message renderers (devui should render oai responses types primarily)
    
    New Features
    - Added _conversations.py (467 lines) - Full conversation storage abstraction, replaces the /threads endpoint to better match oai conversations api
    - Implements OpenAI Conversations API for thread management, Supports in-memory and extensible storage backends
    
    API Simplification
    - Use 'model' field as entity_id (agent/workflow name) instead of extra_body
    - Use standard OpenAI 'conversation' field for conversation context.
    
    Performance & Quality Improvements
    - Improved context management in MessageMapper with bounded memory (~500KB max)
    - Implemented hybrid LRU + cleanup approach to prevent unbounded memory growth
    - General QOL improvement - Eliminated ~150 lines of dead/duplicate code, Consolidated helper functions into _utils.py, Extracted magic numbers to module-level constants, Optimized conversation item lookups with index-based approach
    
    Testing
    - Added test_conversations.py (13 tests)
    - Added test_performance_fixes.py (9 tests)
    - Updated existing tests for code consolidation
    - 53 tests passing
    
    Impact: 76 files changed: +4,106 insertions, -2,373 deletions
    All linting and formatting checks passing. No breaking changes - backward compatible.
    
    Migration: Samples moved to python/samples/getting_started/devui/
    
    * readme lint fixes
    
    * initial support for function approval and minor ui fixes
  • Python: DevUI improvements. (#1091)
    * enable deeplinking in ui, add agent details to entity info, add usage data, add middleware example in samples and foundry agent.
    
    * update ui build
    
    * Update python/packages/devui/frontend/src/components/workflow/workflow-input-form.tsx
    
    Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
    
    * Update python/packages/devui/pyproject.toml
    
    Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
    
    * Update python/packages/devui/pyproject.toml
    
    Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
    
    * imporove mapping for agent nodes and serialiation for agent run events
    
    * lint fixes
    
    * update pyproj toml and ui updates
    
    ---------
    
    Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
  • Python: Default DevUI workflows to string input when start node is auto-wrapped agent (#1143)
    * DevUI workflows default to string input when start node is AgentExecutor
    
    * Remove unnecessary sample
    
    * Fix lint issue
  • Python: Bug fixes for devui serialization and agent structured outputs (#1055)
    * Bug fixes for devui serialization and agent structured outputs
    
    * Fix typing
    
    * Add mapper serialization unit tests
  • Python: [BREAKING] cleanup of thread API and serialization (#893)
    * cleanup of threads and serialization
    
    * fix for sliding window
    
    * fix redis test
    
    * updated from comments
    
    * updated context provider and threads
    
    * updated lock
    
    * add asyncio default
    
    * fix redis tests
    
    * fix tests
    
    * fix tests
    
    * renamed to invoking
    
    * fixed tests
    
    * fix for instructions
  • Python: Add DevUI to AgentFramework (#781)
    * add initial backend service code for devui
    
    * add tests
    
    * add frontendcode
    
    * ui updates
    
    * update readme
    
    * ui updates and tweaks
    
    * update ui bundle
    
    * improve ui, add react flow base
    
    * add react flow ui, fix background
    
    * update ui, fix introspection bug
    
    * update readme
    
    * update ui build
    
    * add support for multimodal input - both backend and frontend
    
    * update ui build
    
    * refactor as main framework package
    
    * backend and tests refactor
    
    * ui build update
    
    * ui build update and refactor
    
    * update pyproject.toml, update uv.lock
    
    * update ui build
    
    * ui update to fit oai responses types
    
    * add backend updat and readme update
    
    * mypy and other fixes
    
    * add intial dev guide
    
    * update ui and fix workflow bug
    
    * update ui build, add thread support
    
    * type fixes
    
    * update workflow view
    
    * update uv.lock
    
    * fix workflow iport errors
    
    * lint and other fixes
    
    * mypy fixes
    
    * minor update
    
    * update ui build
    
    * refactor to use oai dependencies directly, update examples to samples, improve typing
    
    * readme update
    
    * update ui and ui build
    
    * fix workflow pyright error
    
    * update ui, fix issues with run workflow placement, miniamp menu, etc
    
    * make samples integrate serve
    
    ---------
    
    Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
    Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>