jameswt-oai
ff50b47dce
## Why Plugin analytics overloaded `plugin_id`: most events used the Codex `<plugin>@<marketplace>` identity, while remote install events used the backend plugin ID. That makes the same field change meaning across event types and complicates downstream identity resolution. This change makes the contract unambiguous: - `plugin_id`: the local Codex `<plugin>@<marketplace>` identity, when resolved - `remote_plugin_id`: the backend plugin identity, when available For a remote install failure that happens before plugin details resolve, `plugin_id` is `null` and `remote_plugin_id` remains populated. ## What changed All six plugin analytics events use the same identity contract: - `codex_plugin_installed` - `codex_plugin_install_failed` - `codex_plugin_uninstalled` - `codex_plugin_enabled` - `codex_plugin_disabled` - `codex_plugin_used` Remote identity is resolved from the current installed-plugin snapshot first, with persisted install metadata as fallback. The telemetry metadata type keeps local identity optional for failures that occur before remote details are available. The app-server test client's manual analytics smokes now find remote mutation events through `remote_plugin_id` and validate that `plugin_id` remains local. ## Remote uninstall Resolve and capture telemetry metadata before removing the local plugin cache, then emit `codex_plugin_uninstalled` after the backend confirms success. The event is also emitted when backend uninstall succeeds but local cache cleanup reports `CacheRemove`. If a concurrent remote-cache refresh removes the local bundle before telemetry capture, the already-fetched remote plugin detail supplies fallback capability metadata. ## Validation - `just test -p codex-analytics` — 82 passed - `just test -p codex-core-plugins` — 271 passed - `just test -p codex-app-server-test-client` — 5 passed - `just test -p codex-plugin` — 3 passed - `just test -p codex-app-server plugin_install` — 37 passed - `just test -p codex-app-server plugin_uninstall` — 10 passed The production app-server install/uninstall flow was also exercised against `plugins~Plugin_f1b845ac33888191ac156169c58733c2` (`build-ios-apps@openai-curated-remote`), and the plugin's original uninstalled state was restored.
5.3 KiB
App Server Test Client
Quickstart for running and hitting codex app-server.
Quickstart
Run from <reporoot>/codex-rs.
# 1) Build debug codex binary
cargo build -p codex-cli --bin codex
# 2) Start websocket app-server in background
cargo run -p codex-app-server-test-client -- \
--codex-bin ./target/debug/codex \
serve --listen ws://127.0.0.1:4222 --kill
# 3) Call app-server (defaults to ws://127.0.0.1:4222)
cargo run -p codex-app-server-test-client -- model-list
send-message and send-message-v2 handle request_user_input server requests interactively.
When Codex asks a question, choose a numbered option (or o for a free-form answer when offered)
and the client will send the response and continue streaming the same turn.
Testing Plugin Analytics
The plugin-analytics-smoke command exercises plugin/installed, plugin
enable/disable config writes, and a structured plugin mention through one
app-server connection. Analytics are captured to a local JSONL file and are
not sent to the analytics backend. The model turn uses a loopback Responses
API server.
The selected plugin must already be installed and enabled remotely, and the active Codex profile must be authenticated. On a fresh local cache, the command retries ephemeral turns while the installed remote bundle finishes syncing.
# Build a debug Codex binary; analytics capture is unavailable in release builds.
cargo build -p codex-cli --bin codex
cargo run -p codex-app-server-test-client -- \
--codex-bin ./target/debug/codex \
plugin-analytics-smoke \
--plugin-id linear@openai-curated-remote
Use --capture-file /tmp/plugin-analytics.jsonl to select the output path.
The command validates one codex_plugin_disabled, codex_plugin_enabled, and
codex_plugin_used event with the expected local and remote plugin identities
and capability metadata. Each event includes the local ID in plugin_id and the
backend ID in remote_plugin_id. The enabled and disabled events come from
successful writes to the temporary config; the command does not mutate the
remote enabled state. It prints the events and leaves the JSONL file in place
for inspection. It does not install or uninstall plugins and does not modify
the profile's persistent config.
Testing remote install and uninstall analytics
plugin-analytics-mutation-smoke is a manually invoked live smoke test. It
contacts the configured remote plugin API and temporarily changes the active
account's installed-plugin state. It is not run by cargo test, just test,
or CI.
Choose a remote plugin that is available to the active account and is not
currently installed. The command refuses to run when the plugin is already
installed, installs it, validates codex_plugin_installed, uninstalls it, and
validates codex_plugin_uninstalled, and verifies that the original
uninstalled state was restored.
The mutation events include the local Codex ID in plugin_id and the backend ID
in remote_plugin_id.
--remote-plugin-id takes the backend ID, such as plugins~Plugin_..., not the
local <plugin>@<marketplace> ID.
cargo run -p codex-app-server-test-client -- \
--codex-bin ./target/debug/codex \
plugin-analytics-mutation-smoke \
--remote-plugin-id <REMOTE_PLUGIN_ID> \
--confirm-account-mutation \
--capture-file /tmp/plugin-mutation-analytics.jsonl
Analytics use the normal queue, reduction, batching, and serialization path, but the debug capture destination suppresses analytics network delivery. The command prints one of these final states:
PASS: the install and uninstall events validated and the plugin is uninstalled.FAIL-CLEAN: validation failed, but the original uninstalled state was restored.FAIL-LOCAL-CACHE: the backend is uninstalled, but local cleanup reported an error.FAIL-DIRTY: cleanup failed and the plugin still appears installed.FAIL-UNKNOWN: the command could not verify the final installed state.
For a dirty or uncertain result, retry cleanup with:
cargo run -p codex-app-server-test-client -- \
--codex-bin ./target/debug/codex \
plugin-remote-uninstall \
--remote-plugin-id <REMOTE_PLUGIN_ID> \
--confirm-account-mutation
Cleanup does not require analytics capture or a debug Codex binary. When the
smoke uses global --config overrides, its printed recovery command preserves
them so cleanup targets the same backend and account.
Watching Raw Inbound Traffic
Initialize a connection, then print every inbound JSON-RPC message until you stop it with
Ctrl+C:
cargo run -p codex-app-server-test-client -- watch
Testing Thread Rejoin Behavior
Build and start an app server using commands above. The app-server log is written to /tmp/codex-app-server-test-client/app-server.log
1) Get a thread id
Create at least one thread, then list threads:
cargo run -p codex-app-server-test-client -- send-message-v2 "seed thread for rejoin test"
cargo run -p codex-app-server-test-client -- thread-list --limit 5
Copy a thread id from the thread-list output.
2) Rejoin while a turn is in progress (two terminals)
Terminal A:
cargo run --bin codex-app-server-test-client -- \
resume-message-v2 <THREAD_ID> "respond with thorough docs on the rust core"
Terminal B (while Terminal A is still streaming):
cargo run --bin codex-app-server-test-client -- thread-resume <THREAD_ID>