[BREAKING] .NET: Workflow Off-Thread Execution Mode (#1233)

* Updates to async run loop.

* fix: Workflow Onwership can be release by nonowner

* fix: Incorrect handling of blockOnPending in StreamingRun

Depending on whether we are running in streaming on non-streaming mode, we may be using the StreamingRun in different ways. Unfortunately, the only place we can really know what is the actual state of execution is in the RunEventStream implementations.

This resulted in blocking where blocking was unneeded and occasionally not-blocking when blocking was needed.

The fix is to move the logic of handling this blocking into RunEventStream implementations.

* fix: Fix cleanup on error and end run

This ensures we clean up the background resources correctly.

* fix: Ensure we let the run loop proceed when shutting down

* fix: Add timeout for Input Waiting

* fix: Make the samples properly clean up `Run`s and `StreamingRun`s

* fix: Simplify Declarative Workflow Run disposal pattern

* Also fixes missing .Disposal() in Integration tests

---------

Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>

dotnet/samples/GettingStarted/Workflows/Declarative/Program.cs

@@ -64,7 +64,7 @@ internal sealed class Program
         InputResponse? response = null;
         do
         {
-            ExternalRequest? inputRequest = await this.MonitorWorkflowRunAsync(run, response);
+            ExternalRequest? inputRequest = await this.MonitorAndDisposeWorkflowRunAsync(run, response);
             if (inputRequest is not null)
             {
                 Notify("\nWORKFLOW: Yield");
@@ -83,6 +83,7 @@ internal sealed class Program
                 // Restore the latest checkpoint.
                 Debug.WriteLine($"RESTORE #{this.LastCheckpoint.CheckpointId}");
                 Notify("\nWORKFLOW: Restore");
+
                 run = await InProcessExecution.ResumeStreamAsync(workflow, this.LastCheckpoint, checkpointManager, run.Run.RunId);
             }
             else
@@ -132,8 +133,10 @@ internal sealed class Program
         this.FoundryClient = new PersistentAgentsClient(this.FoundryEndpoint, new AzureCliCredential());
     }
 
-    private async Task<ExternalRequest?> MonitorWorkflowRunAsync(Checkpointed<StreamingRun> run, InputResponse? response = null)
+    private async Task<ExternalRequest?> MonitorAndDisposeWorkflowRunAsync(Checkpointed<StreamingRun> run, InputResponse? response = null)
     {
+        await using IAsyncDisposable disposeRun = run;
+
         string? messageId = null;
 
         await foreach (WorkflowEvent workflowEvent in run.Run.WatchStreamAsync().ConfigureAwait(false))
@@ -175,7 +178,7 @@ internal sealed class Program
                     }
                     else
                     {
-                        await run.Run.EndRunAsync().ConfigureAwait(false);
+                        await run.Run.DisposeAsync().ConfigureAwait(false);
                         return requestInfo.Request;
                     }
                     break;