Python: [BREAKING] PR2 — Wire context provider pipeline, remove old types, update all consumers (#3850)

* PR2: Wire context provider pipeline and update all internal consumers

- Replace AgentThread with AgentSession across all packages
- Replace ContextProvider with BaseContextProvider across all packages
- Replace context_provider param with context_providers (Sequence)
- Replace thread= with session= in run() signatures
- Replace get_new_thread() with create_session()
- Add get_session(service_session_id) to agent interface
- DurableAgentThread -> DurableAgentSession
- Remove _notify_thread_of_new_messages from WorkflowAgent
- Wire before_run/after_run context provider pipeline in RawAgent
- Auto-inject InMemoryHistoryProvider when no providers configured

* fix: update all tests for context provider pipeline, fix lazy-loaders, remove old test files

* refactor: update all sample files for context provider pipeline (AgentThread→AgentSession, ContextProvider→BaseContextProvider)

* fix: update remaining ag-ui references (client docstring, getting_started sample)

* fix: make get_session service_session_id keyword-only to avoid confusion with session_id

* refactor: rename _RunContext.thread_messages to session_messages

* refactor: remove _threads.py, _memory.py, and old provider files; migrate devui to use plain message lists

* rename: remove _new_ prefix from test files

* refactor: rewrite SlidingWindowChatMessageStore as SlidingWindowHistoryProvider(InMemoryHistoryProvider)

* fix: read full history from session state directly instead of reaching into provider internals

* fix: update stale .pyi stubs, sample imports, and README references for new provider types

* fix: remove stale message_store, _notify_thread_of_new_messages, and session_id.key references in samples

* refactor: merge context_providers and sessions sample folders into sessions, remove aggregate_context_provider

* refactor: UserInfoMemory stores state in session.state instead of instance attributes

* feat: add Pydantic BaseModel support to session state serialization

Pydantic models stored in session.state are now automatically serialized
via model_dump() and restored via model_validate() during to_dict()/from_dict()
round-trips. Models are auto-registered on first serialization; use
register_state_type() for cold-start deserialization.

Also export register_state_type as a public API.

* fix mem0

* Update sample README links and descriptions for session terminology

- Replace 'thread' with 'session' in sample descriptions across all READMEs
- Update file links for renamed samples (mem0_sessions, redis_sessions, etc.)
- Fix Threads section → Sessions section in main samples/README.md
- Update tools, middleware, workflows, durabletask, azure_functions READMEs
- Update architecture diagrams in concepts/tools/README.md
- Update migration guides (autogen, semantic-kernel)

* Fix broken Redis README link to renamed sample

* Fix Mem0 OSS client search: pass scoping params as direct kwargs

AsyncMemory (OSS) expects user_id/agent_id/run_id as direct kwargs,
while AsyncMemoryClient (Platform) expects them in a filters dict.
Adds tests for both client types.

Port of fix from #3844 to new Mem0ContextProvider.

* Fix rebase issues: restore missing _conversation_state.py and checkpoint decode logic

- Add back _conversation_state.py (encode/decode_chat_messages) lost in rebase
- Fix on_checkpoint_restore to decode cache/conversation with decode_chat_messages
- Fix on_checkpoint_restore to use decode_checkpoint_value for pending requests
- Add tests/workflow/__init__.py for relative import support
- Fix test_agent_executor checkpoint selection (checkpoints[1] not superstep)

* Add STORES_BY_DEFAULT ClassVar to skip redundant InMemoryHistoryProvider injection

Chat clients that store history server-side by default (OpenAI Responses API,
Azure AI Agent) now declare STORES_BY_DEFAULT = True. The agent checks this
during auto-injection and skips InMemoryHistoryProvider unless the user
explicitly sets store=False.

* Fix broken markdown links in azure_ai and redis READMEs

* Fix getting-started samples to use session API instead of removed thread/ContextProvider API

* updates to workflow as agent

* fix group chat import

* Rename Thread→Session throughout, fix service_session_id propagation, remove stale AGUIThread

- Fix: Propagate conversation_id from ChatResponse back to session.service_session_id
  in both streaming and non-streaming paths in _agents.py
- Rename AgentThreadException → AgentSessionException
- Remove stale AGUIThread from ag_ui lazy-loader
- Rename use_service_thread → use_service_session in ag-ui package
- Rename test functions from *_thread_* to *_session_*
- Rename sample files from *_thread* to *_session*
- Update docstrings and comments: thread → session
- Update _mcp.py kwargs filter: add 'session' alongside 'thread'
- Fix ContinuationToken docstring example: thread=thread → session=session
- Fix _clients.py docstring: 'Agent threads' → 'Agent sessions'

* Fix broken markdown links after thread→session file renames

* fix azure ai test

python/packages/durabletask/tests/integration_tests/test_01_dt_single_agent.py

@@ -39,9 +39,9 @@ class TestSingleAgent:
     def test_single_interaction(self):
         """Test a single interaction with the agent."""
         agent = self.agent_client.get_agent("Joker")
-        thread = agent.get_new_thread()
+        session = agent.create_session()
 
-        response = agent.run("Tell me a short joke about programming.", thread=thread)
+        response = agent.run("Tell me a short joke about programming.", session=session)
 
         assert response is not None
         assert response.text is not None
@@ -50,33 +50,33 @@ class TestSingleAgent:
     def test_conversation_continuity(self):
         """Test that conversation context is maintained across turns."""
         agent = self.agent_client.get_agent("Joker")
-        thread = agent.get_new_thread()
+        session = agent.create_session()
 
         # First turn: Ask for a joke about a specific topic
-        response1 = agent.run("Tell me a joke about cats.", thread=thread)
+        response1 = agent.run("Tell me a joke about cats.", session=session)
         assert response1 is not None
         assert len(response1.text) > 0
 
         # Second turn: Ask a follow-up that requires context
-        response2 = agent.run("Can you make it funnier?", thread=thread)
+        response2 = agent.run("Can you make it funnier?", session=session)
         assert response2 is not None
         assert len(response2.text) > 0
 
         # The agent should understand "it" refers to the previous joke
 
-    def test_multiple_threads(self):
-        """Test that different threads maintain separate contexts."""
+    def test_multiple_sessions(self):
+        """Test that different sessions maintain separate contexts."""
         agent = self.agent_client.get_agent("Joker")
 
-        # Create two separate threads
-        thread1 = agent.get_new_thread()
-        thread2 = agent.get_new_thread()
+        # Create two separate sessions
+        session1 = agent.create_session()
+        session2 = agent.create_session()
 
-        assert thread1.session_id != thread2.session_id
+        assert session1.durable_session_id != session2.durable_session_id
 
-        # Send different messages to each thread
-        response1 = agent.run("Tell me a joke about dogs.", thread=thread1)
-        response2 = agent.run("Tell me a joke about birds.", thread=thread2)
+        # Send different messages to each session
+        response1 = agent.run("Tell me a joke about dogs.", session=session1)
+        response2 = agent.run("Tell me a joke about birds.", session=session2)
 
         assert response1 is not None
         assert response2 is not None

python/packages/durabletask/tests/integration_tests/test_02_dt_multi_agent.py

@@ -47,9 +47,9 @@ class TestMultiAgent:
     def test_weather_agent_with_tool(self):
         """Test weather agent with weather tool execution."""
         agent = self.agent_client.get_agent(WEATHER_AGENT_NAME)
-        thread = agent.get_new_thread()
+        session = agent.create_session()
 
-        response = agent.run("What's the weather in Seattle?", thread=thread)
+        response = agent.run("What's the weather in Seattle?", session=session)
 
         assert response is not None
         assert response.text is not None
@@ -66,9 +66,9 @@ class TestMultiAgent:
     def test_math_agent_with_tool(self):
         """Test math agent with calculation tool execution."""
         agent = self.agent_client.get_agent(MATH_AGENT_NAME)
-        thread = agent.get_new_thread()
+        session = agent.create_session()
 
-        response = agent.run("Calculate a 20% tip on a $50 bill.", thread=thread)
+        response = agent.run("Calculate a 20% tip on a $50 bill.", session=session)
 
         assert response is not None
         assert response.text is not None
@@ -85,11 +85,11 @@ class TestMultiAgent:
     def test_multiple_calls_to_same_agent(self):
         """Test multiple sequential calls to the same agent."""
         agent = self.agent_client.get_agent(WEATHER_AGENT_NAME)
-        thread = agent.get_new_thread()
+        session = agent.create_session()
 
         # Multiple weather queries
-        response1 = agent.run("What's the weather in Chicago?", thread=thread)
-        response2 = agent.run("And what about Los Angeles?", thread=thread)
+        response1 = agent.run("What's the weather in Chicago?", session=session)
+        response2 = agent.run("And what about Los Angeles?", session=session)
 
         assert response1 is not None
         assert response2 is not None

python/packages/durabletask/tests/integration_tests/test_03_dt_single_agent_streaming.py

@@ -70,7 +70,7 @@ class TestSampleReliableStreaming:
 
     async def _stream_from_redis(
         self,
-        thread_id: str,
+        session_key: str,
         cursor: str | None = None,
         timeout: float = 30.0,
     ) -> tuple[str, bool, str]:
@@ -78,7 +78,7 @@ class TestSampleReliableStreaming:
         Stream responses from Redis using the sample's RedisStreamResponseHandler.
 
         Args:
-            thread_id: The conversation/thread ID to stream from
+            session_key: The conversation/thread ID to stream from
             cursor: Optional cursor to resume from
             timeout: Maximum time to wait for stream completion
 
@@ -92,7 +92,7 @@ class TestSampleReliableStreaming:
 
         async with await self._get_stream_handler() as stream_handler:  # type: ignore[reportUnknownMemberType]
             try:
-                async for chunk in stream_handler.read_stream(thread_id, cursor):  # type: ignore[reportUnknownMemberType]
+                async for chunk in stream_handler.read_stream(session_key, cursor):  # type: ignore[reportUnknownMemberType]
                     if time.time() - start_time > timeout:
                         break
 
@@ -124,15 +124,15 @@ class TestSampleReliableStreaming:
         assert travel_planner is not None
         assert travel_planner.name == "TravelPlanner"
 
-        # Create a new thread
-        thread = travel_planner.get_new_thread()
-        assert thread.session_id is not None
-        assert thread.session_id.key is not None
-        thread_id = str(thread.session_id.key)
+        # Create a new session
+        session = travel_planner.create_session()
+        assert session.durable_session_id is not None
+        assert session.durable_session_id.key is not None
+        session_key = str(session.durable_session_id.key)
 
         # Start agent run with wait_for_response=False for non-blocking execution
         travel_planner.run(
-            "Plan a 1-day trip to Seattle in 1 sentence", thread=thread, options={"wait_for_response": False}
+            "Plan a 1-day trip to Seattle in 1 sentence", session=session, options={"wait_for_response": False}
         )
 
         # Poll Redis stream with retries to handle race conditions
@@ -146,7 +146,7 @@ class TestSampleReliableStreaming:
 
         while retry_count < max_retries and not is_complete:
             text, is_complete, last_cursor = asyncio.run(
-                self._stream_from_redis(thread_id, cursor=cursor, timeout=10.0)
+                self._stream_from_redis(session_key, cursor=cursor, timeout=10.0)
             )
             accumulated_text += text
             cursor = last_cursor  # Resume from last position on next read
@@ -166,7 +166,7 @@ class TestSampleReliableStreaming:
 
         # Verify we got content
         assert len(accumulated_text) > 0, (
-            f"Expected text content but got empty string for thread_id: {thread_id} after {retry_count} retries"
+            f"Expected text content but got empty string for session_key: {session_key} after {retry_count} retries"
         )
         assert "seattle" in accumulated_text.lower(), f"Expected 'seattle' in response but got: {accumulated_text}"
         assert is_complete, "Expected stream to be complete"
@@ -175,13 +175,13 @@ class TestSampleReliableStreaming:
         """Test streaming with cursor-based resumption."""
         # Get the TravelPlanner agent
         travel_planner = self.agent_client.get_agent("TravelPlanner")
-        thread = travel_planner.get_new_thread()
-        assert thread.session_id is not None
-        assert thread.session_id.key is not None
-        thread_id = str(thread.session_id.key)
+        session = travel_planner.create_session()
+        assert session.durable_session_id is not None
+        assert session.durable_session_id.key is not None
+        session_key = str(session.durable_session_id.key)
 
         # Start agent run
-        travel_planner.run("What's the weather like?", thread=thread, options={"wait_for_response": False})
+        travel_planner.run("What's the weather like?", session=session, options={"wait_for_response": False})
 
         # Wait for agent to start writing
         time.sleep(3)
@@ -194,7 +194,7 @@ class TestSampleReliableStreaming:
                 chunk_count = 0
 
                 # Read just first 2 chunks
-                async for chunk in stream_handler.read_stream(thread_id):  # type: ignore[reportUnknownMemberType]
+                async for chunk in stream_handler.read_stream(session_key):  # type: ignore[reportUnknownMemberType]
                     last_entry_id = chunk.entry_id  # type: ignore[reportUnknownMemberType]
                     if chunk.text:  # type: ignore[reportUnknownMemberType]
                         accumulated_text += chunk.text  # type: ignore[reportUnknownMemberType]
@@ -207,7 +207,7 @@ class TestSampleReliableStreaming:
         partial_text, cursor = asyncio.run(get_partial_stream())
 
         # Resume from cursor
-        remaining_text, _, _ = asyncio.run(self._stream_from_redis(thread_id, cursor=cursor))
+        remaining_text, _, _ = asyncio.run(self._stream_from_redis(session_key, cursor=cursor))
 
         # Verify we got some initial content
         assert len(partial_text) > 0