Python: [BREAKING] PR2 — Wire context provider pipeline, remove old types, update all consumers (#3850)

* PR2: Wire context provider pipeline and update all internal consumers

- Replace AgentThread with AgentSession across all packages
- Replace ContextProvider with BaseContextProvider across all packages
- Replace context_provider param with context_providers (Sequence)
- Replace thread= with session= in run() signatures
- Replace get_new_thread() with create_session()
- Add get_session(service_session_id) to agent interface
- DurableAgentThread -> DurableAgentSession
- Remove _notify_thread_of_new_messages from WorkflowAgent
- Wire before_run/after_run context provider pipeline in RawAgent
- Auto-inject InMemoryHistoryProvider when no providers configured

* fix: update all tests for context provider pipeline, fix lazy-loaders, remove old test files

* refactor: update all sample files for context provider pipeline (AgentThread→AgentSession, ContextProvider→BaseContextProvider)

* fix: update remaining ag-ui references (client docstring, getting_started sample)

* fix: make get_session service_session_id keyword-only to avoid confusion with session_id

* refactor: rename _RunContext.thread_messages to session_messages

* refactor: remove _threads.py, _memory.py, and old provider files; migrate devui to use plain message lists

* rename: remove _new_ prefix from test files

* refactor: rewrite SlidingWindowChatMessageStore as SlidingWindowHistoryProvider(InMemoryHistoryProvider)

* fix: read full history from session state directly instead of reaching into provider internals

* fix: update stale .pyi stubs, sample imports, and README references for new provider types

* fix: remove stale message_store, _notify_thread_of_new_messages, and session_id.key references in samples

* refactor: merge context_providers and sessions sample folders into sessions, remove aggregate_context_provider

* refactor: UserInfoMemory stores state in session.state instead of instance attributes

* feat: add Pydantic BaseModel support to session state serialization

Pydantic models stored in session.state are now automatically serialized
via model_dump() and restored via model_validate() during to_dict()/from_dict()
round-trips. Models are auto-registered on first serialization; use
register_state_type() for cold-start deserialization.

Also export register_state_type as a public API.

* fix mem0

* Update sample README links and descriptions for session terminology

- Replace 'thread' with 'session' in sample descriptions across all READMEs
- Update file links for renamed samples (mem0_sessions, redis_sessions, etc.)
- Fix Threads section → Sessions section in main samples/README.md
- Update tools, middleware, workflows, durabletask, azure_functions READMEs
- Update architecture diagrams in concepts/tools/README.md
- Update migration guides (autogen, semantic-kernel)

* Fix broken Redis README link to renamed sample

* Fix Mem0 OSS client search: pass scoping params as direct kwargs

AsyncMemory (OSS) expects user_id/agent_id/run_id as direct kwargs,
while AsyncMemoryClient (Platform) expects them in a filters dict.
Adds tests for both client types.

Port of fix from #3844 to new Mem0ContextProvider.

* Fix rebase issues: restore missing _conversation_state.py and checkpoint decode logic

- Add back _conversation_state.py (encode/decode_chat_messages) lost in rebase
- Fix on_checkpoint_restore to decode cache/conversation with decode_chat_messages
- Fix on_checkpoint_restore to use decode_checkpoint_value for pending requests
- Add tests/workflow/__init__.py for relative import support
- Fix test_agent_executor checkpoint selection (checkpoints[1] not superstep)

* Add STORES_BY_DEFAULT ClassVar to skip redundant InMemoryHistoryProvider injection

Chat clients that store history server-side by default (OpenAI Responses API,
Azure AI Agent) now declare STORES_BY_DEFAULT = True. The agent checks this
during auto-injection and skips InMemoryHistoryProvider unless the user
explicitly sets store=False.

* Fix broken markdown links in azure_ai and redis READMEs

* Fix getting-started samples to use session API instead of removed thread/ContextProvider API

* updates to workflow as agent

* fix group chat import

* Rename Thread→Session throughout, fix service_session_id propagation, remove stale AGUIThread

- Fix: Propagate conversation_id from ChatResponse back to session.service_session_id
  in both streaming and non-streaming paths in _agents.py
- Rename AgentThreadException → AgentSessionException
- Remove stale AGUIThread from ag_ui lazy-loader
- Rename use_service_thread → use_service_session in ag-ui package
- Rename test functions from *_thread_* to *_session_*
- Rename sample files from *_thread* to *_session*
- Update docstrings and comments: thread → session
- Update _mcp.py kwargs filter: add 'session' alongside 'thread'
- Fix ContinuationToken docstring example: thread=thread → session=session
- Fix _clients.py docstring: 'Agent threads' → 'Agent sessions'

* Fix broken markdown links after thread→session file renames

* fix azure ai test

python/samples/getting_started/sessions/redis/redis_sessions.py

@@ -0,0 +1,249 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Redis Context Provider: Thread scoping examples
+
+This sample demonstrates how conversational memory can be scoped when using the
+Redis context provider. It covers three scenarios:
+
+1) Global thread scope
+   - Provide a fixed thread_id to share memories across operations/threads.
+
+2) Per-operation thread scope
+   - Enable scope_to_per_operation_thread_id to bind the provider to a single
+     thread for the lifetime of that provider instance. Use the same thread
+     object for reads/writes with that provider.
+
+3) Multiple agents with isolated memory
+   - Use different agent_id values to keep memories separated for different
+     agent personas, even when the user_id is the same.
+
+Requirements:
+  - A Redis instance with RediSearch enabled (e.g., Redis Stack)
+  - agent-framework with the Redis extra installed: pip install "agent-framework-redis"
+  - Optionally an OpenAI API key for the chat client in this demo
+
+Run:
+  python redis_threads.py
+"""
+
+import asyncio
+import os
+import uuid
+
+from agent_framework.openai import OpenAIChatClient
+from agent_framework.redis import RedisContextProvider
+from redisvl.extensions.cache.embeddings import EmbeddingsCache
+from redisvl.utils.vectorize import OpenAITextVectorizer
+
+# Please set the OPENAI_API_KEY and OPENAI_CHAT_MODEL_ID environment variables to use the OpenAI vectorizer
+# Recommend default for OPENAI_CHAT_MODEL_ID is gpt-4o-mini
+
+
+async def example_global_thread_scope() -> None:
+    """Example 1: Global thread_id scope (memories shared across all operations)."""
+    print("1. Global Thread Scope Example:")
+    print("-" * 40)
+
+    global_thread_id = str(uuid.uuid4())
+
+    client = OpenAIChatClient(
+        model_id=os.getenv("OPENAI_CHAT_MODEL_ID", "gpt-4o-mini"),
+        api_key=os.getenv("OPENAI_API_KEY"),
+    )
+
+    provider = RedisContextProvider(
+        redis_url="redis://localhost:6379",
+        index_name="redis_threads_global",
+        application_id="threads_demo_app",
+        agent_id="threads_demo_agent",
+        user_id="threads_demo_user",
+        thread_id=global_thread_id,
+        scope_to_per_operation_thread_id=False,  # Share memories across all sessions
+    )
+
+    agent = client.as_agent(
+        name="GlobalMemoryAssistant",
+        instructions=(
+            "You are a helpful assistant. Personalize replies using provided context. "
+            "Before answering, always check for stored context containing information"
+        ),
+        tools=[],
+        context_providers=[provider],
+    )
+
+    # Store a preference in the global scope
+    query = "Remember that I prefer technical responses with code examples when discussing programming."
+    print(f"User: {query}")
+    result = await agent.run(query)
+    print(f"Agent: {result}\n")
+
+    # Create a new session - memories should still be accessible due to global scope
+    new_session = agent.create_session()
+    query = "What technical responses do I prefer?"
+    print(f"User (new session): {query}")
+    result = await agent.run(query, session=new_session)
+    print(f"Agent: {result}\n")
+
+    # Clean up the Redis index
+    await provider.redis_index.delete()
+
+
+async def example_per_operation_thread_scope() -> None:
+    """Example 2: Per-operation thread scope (memories isolated per session).
+
+    Note: When scope_to_per_operation_thread_id=True, the provider is bound to a single session
+    throughout its lifetime. Use the same session object for all operations with that provider.
+    """
+    print("2. Per-Operation Thread Scope Example:")
+    print("-" * 40)
+
+    client = OpenAIChatClient(
+        model_id=os.getenv("OPENAI_CHAT_MODEL_ID", "gpt-4o-mini"),
+        api_key=os.getenv("OPENAI_API_KEY"),
+    )
+
+    vectorizer = OpenAITextVectorizer(
+        model="text-embedding-ada-002",
+        api_config={"api_key": os.getenv("OPENAI_API_KEY")},
+        cache=EmbeddingsCache(name="openai_embeddings_cache", redis_url="redis://localhost:6379"),
+    )
+
+    provider = RedisContextProvider(
+        redis_url="redis://localhost:6379",
+        index_name="redis_threads_dynamic",
+        # overwrite_redis_index=True,
+        # drop_redis_index=True,
+        application_id="threads_demo_app",
+        agent_id="threads_demo_agent",
+        user_id="threads_demo_user",
+        scope_to_per_operation_thread_id=True,  # Isolate memories per session
+        redis_vectorizer=vectorizer,
+        vector_field_name="vector",
+        vector_algorithm="hnsw",
+        vector_distance_metric="cosine",
+    )
+
+    agent = client.as_agent(
+        name="ScopedMemoryAssistant",
+        instructions="You are an assistant with thread-scoped memory.",
+        context_providers=[provider],
+    )
+
+    # Create a specific session for this scoped provider
+    dedicated_session = agent.create_session()
+
+    # Store some information in the dedicated session
+    query = "Remember that for this conversation, I'm working on a Python project about data analysis."
+    print(f"User (dedicated session): {query}")
+    result = await agent.run(query, session=dedicated_session)
+    print(f"Agent: {result}\n")
+
+    # Test memory retrieval in the same dedicated session
+    query = "What project am I working on?"
+    print(f"User (same dedicated session): {query}")
+    result = await agent.run(query, session=dedicated_session)
+    print(f"Agent: {result}\n")
+
+    # Store more information in the same session
+    query = "Also remember that I prefer using pandas and matplotlib for this project."
+    print(f"User (same dedicated session): {query}")
+    result = await agent.run(query, session=dedicated_session)
+    print(f"Agent: {result}\n")
+
+    # Test comprehensive memory retrieval
+    query = "What do you know about my current project and preferences?"
+    print(f"User (same dedicated session): {query}")
+    result = await agent.run(query, session=dedicated_session)
+    print(f"Agent: {result}\n")
+
+    # Clean up the Redis index
+    await provider.redis_index.delete()
+
+
+async def example_multiple_agents() -> None:
+    """Example 3: Multiple agents with different thread configurations (isolated via agent_id) but within 1 index."""
+    print("3. Multiple Agents with Different Thread Configurations:")
+    print("-" * 40)
+
+    client = OpenAIChatClient(
+        model_id=os.getenv("OPENAI_CHAT_MODEL_ID", "gpt-4o-mini"),
+        api_key=os.getenv("OPENAI_API_KEY"),
+    )
+
+    vectorizer = OpenAITextVectorizer(
+        model="text-embedding-ada-002",
+        api_config={"api_key": os.getenv("OPENAI_API_KEY")},
+        cache=EmbeddingsCache(name="openai_embeddings_cache", redis_url="redis://localhost:6379"),
+    )
+
+    personal_provider = RedisContextProvider(
+        redis_url="redis://localhost:6379",
+        index_name="redis_threads_agents",
+        application_id="threads_demo_app",
+        agent_id="agent_personal",
+        user_id="threads_demo_user",
+        redis_vectorizer=vectorizer,
+        vector_field_name="vector",
+        vector_algorithm="hnsw",
+        vector_distance_metric="cosine",
+    )
+
+    personal_agent = client.as_agent(
+        name="PersonalAssistant",
+        instructions="You are a personal assistant that helps with personal tasks.",
+        context_providers=[personal_provider],
+    )
+
+    work_provider = RedisContextProvider(
+        redis_url="redis://localhost:6379",
+        index_name="redis_threads_agents",
+        application_id="threads_demo_app",
+        agent_id="agent_work",
+        user_id="threads_demo_user",
+        redis_vectorizer=vectorizer,
+        vector_field_name="vector",
+        vector_algorithm="hnsw",
+        vector_distance_metric="cosine",
+    )
+
+    work_agent = client.as_agent(
+        name="WorkAssistant",
+        instructions="You are a work assistant that helps with professional tasks.",
+        context_providers=[work_provider],
+    )
+
+    # Store personal information
+    query = "Remember that I like to exercise at 6 AM and prefer outdoor activities."
+    print(f"User to Personal Agent: {query}")
+    result = await personal_agent.run(query)
+    print(f"Personal Agent: {result}\n")
+
+    # Store work information
+    query = "Remember that I have team meetings every Tuesday at 2 PM."
+    print(f"User to Work Agent: {query}")
+    result = await work_agent.run(query)
+    print(f"Work Agent: {result}\n")
+
+    # Test memory isolation
+    query = "What do you know about my schedule?"
+    print(f"User to Personal Agent: {query}")
+    result = await personal_agent.run(query)
+    print(f"Personal Agent: {result}\n")
+
+    print(f"User to Work Agent: {query}")
+    result = await work_agent.run(query)
+    print(f"Work Agent: {result}\n")
+
+    # Clean up the Redis index (shared)
+    await work_provider.redis_index.delete()
+
+
+async def main() -> None:
+    print("=== Redis Thread Scoping Examples ===\n")
+    await example_global_thread_scope()
+    await example_per_operation_thread_scope()
+    await example_multiple_agents()
+
+
+if __name__ == "__main__":
+    asyncio.run(main())