Python: DevUI fixes : Add multimodal input support for workflows and refactor chat input (#2593)

* show app version in devui .NET: Python: Improved Versioning for DevUI
Fixes #2059

* feat: Add multimodal input support for workflows and refactor chat input

This PR adds support for multimodal content (images, files) in workflow
inputs and refactors the chat input into a reusable component.

## Multimodal Workflow Support
- Add `isChatMessageSchema()` to detect ChatMessage input schemas
- Update `RunWorkflowButton` to use `ChatMessageInput` for ChatMessage workflows
- Wrap multimodal content in OpenAI message format for backend processing
- Add `_is_openai_multimodal_format()` to detect OpenAI ResponseInputParam
- Update `_parse_workflow_input()` to route multimodal input through
  existing `_convert_input_to_chat_message()` converter

## Reusable ChatMessageInput Component
- Extract chat input logic from agent-view into `ChatMessageInput` component
- Support file upload, drag & drop, paste handling, and attachments
- Add `useDragDrop` hook for parent-level drag handling with full-area
  drop zones
- Refactor agent-view to use the new shared component

## Other Improvements
- Add `isStreaming` prop to executor nodes for animation control
- Clean up unused imports and state variables in agent-view
- Add tests for multimodal workflow input handling

Fixes workflow input not receiving images when using AgentExecutor nodes.

* add self loop edge, fix #2470

* fix test
This commit is contained in:
Victor Dibia
2025-12-03 12:15:51 -08:00
committed by GitHub
Unverified
parent 6835161f2d
commit 411ee7a60f
38 changed files with 3488 additions and 1493 deletions
@@ -87,6 +87,7 @@ interface DevUIState {
deployment: boolean;
};
authRequired: boolean;
serverVersion: string | null;
// Deployment Slice
isDeploying: boolean;
@@ -165,7 +166,7 @@ interface DevUIActions {
toggleOAIMode: () => void;
// Server Meta Actions
setServerMeta: (meta: { uiMode: "developer" | "user"; runtime: "python" | "dotnet"; capabilities: { tracing: boolean; openai_proxy: boolean; deployment: boolean }; authRequired: boolean }) => void;
setServerMeta: (meta: { uiMode: "developer" | "user"; runtime: "python" | "dotnet"; capabilities: { tracing: boolean; openai_proxy: boolean; deployment: boolean }; authRequired: boolean; version?: string }) => void;
// Deployment Actions
startDeployment: () => void;
@@ -252,6 +253,7 @@ export const useDevUIStore = create<DevUIStore>()(
deployment: false,
},
authRequired: false,
serverVersion: null,
// Deployment State
isDeploying: false,
@@ -516,6 +518,7 @@ export const useDevUIStore = create<DevUIStore>()(
runtime: meta.runtime,
serverCapabilities: meta.capabilities,
authRequired: meta.authRequired,
serverVersion: meta.version || null,
}),
// ========================================