Files
codex/codex-rs/app-server-test-client/src/plugin_analytics_capture_tests.rs
T
jameswt-oai ff50b47dce Separate local and remote plugin analytics IDs (#29495)
## Why

Plugin analytics overloaded `plugin_id`: most events used the Codex
`<plugin>@<marketplace>` identity, while remote install events used the
backend plugin ID. That makes the same field change meaning across event
types and complicates downstream identity resolution.

This change makes the contract unambiguous:

- `plugin_id`: the local Codex `<plugin>@<marketplace>` identity, when
resolved
- `remote_plugin_id`: the backend plugin identity, when available

For a remote install failure that happens before plugin details resolve,
`plugin_id` is `null` and `remote_plugin_id` remains populated.

## What changed

All six plugin analytics events use the same identity contract:

- `codex_plugin_installed`
- `codex_plugin_install_failed`
- `codex_plugin_uninstalled`
- `codex_plugin_enabled`
- `codex_plugin_disabled`
- `codex_plugin_used`

Remote identity is resolved from the current installed-plugin snapshot
first, with persisted install metadata as fallback. The telemetry
metadata type keeps local identity optional for failures that occur
before remote details are available.

The app-server test client's manual analytics smokes now find remote
mutation events through `remote_plugin_id` and validate that `plugin_id`
remains local.

## Remote uninstall

Resolve and capture telemetry metadata before removing the local plugin
cache, then emit `codex_plugin_uninstalled` after the backend confirms
success. The event is also emitted when backend uninstall succeeds but
local cache cleanup reports `CacheRemove`.

If a concurrent remote-cache refresh removes the local bundle before
telemetry capture, the already-fetched remote plugin detail supplies
fallback capability metadata.

## Validation

- `just test -p codex-analytics` — 82 passed
- `just test -p codex-core-plugins` — 271 passed
- `just test -p codex-app-server-test-client` — 5 passed
- `just test -p codex-plugin` — 3 passed
- `just test -p codex-app-server plugin_install` — 37 passed
- `just test -p codex-app-server plugin_uninstall` — 10 passed

The production app-server install/uninstall flow was also exercised
against `plugins~Plugin_f1b845ac33888191ac156169c58733c2`
(`build-ios-apps@openai-curated-remote`), and the plugin's original
uninstalled state was restored.
2026-06-23 12:27:14 -07:00

98 lines
3.2 KiB
Rust

use super::PluginEventIdentity;
use super::read_events_for_remote_plugin;
use super::validate_mutation_events;
use serde_json::Value;
use serde_json::json;
use std::fs;
use std::path::PathBuf;
use std::process;
use std::time::SystemTime;
const REMOTE_PLUGIN_ID: &str = "plugins~Plugin_test";
#[test]
fn reads_and_validates_remote_plugin_mutation_events() {
let path = unique_capture_path("valid");
let installed = mutation_event("codex_plugin_installed");
let uninstalled = mutation_event("codex_plugin_uninstalled");
let unrelated = json!({
"event_type": "codex_plugin_installed",
"event_params": {
"plugin_id": "other@openai-curated-remote",
"remote_plugin_id": "plugins~Plugin_other"
}
});
let contents = [
json!({"events": [unrelated]}),
json!({"events": [installed, uninstalled]}),
]
.into_iter()
.map(|payload| serde_json::to_string(&payload).expect("serialize capture payload"))
.collect::<Vec<_>>()
.join("\n");
fs::write(&path, contents).expect("write capture file");
let events = read_events_for_remote_plugin(&path, REMOTE_PLUGIN_ID)
.expect("read matching plugin events");
let validated =
validate_mutation_events(events, expected_identity()).expect("validate mutation events");
assert_eq!(validated, vec![installed, uninstalled]);
fs::remove_file(path).expect("remove capture file");
}
#[test]
fn rejects_duplicate_mutation_events() {
let installed = mutation_event("codex_plugin_installed");
let error = validate_mutation_events(vec![installed.clone(), installed], expected_identity())
.expect_err("duplicate install events should fail validation");
assert!(error.to_string().contains("found 2"));
}
#[test]
fn rejects_missing_capability_metadata() {
let mut installed = mutation_event("codex_plugin_installed");
installed["event_params"]["has_skills"] = Value::Null;
let error = validate_mutation_events(vec![installed], expected_identity())
.expect_err("missing capability metadata should fail validation");
assert!(error.to_string().contains("has_skills"));
}
fn mutation_event(event_type: &str) -> Value {
json!({
"event_type": event_type,
"event_params": {
"plugin_id": "sample@openai-curated-remote",
"remote_plugin_id": REMOTE_PLUGIN_ID,
"plugin_name": "sample",
"marketplace_name": "openai-curated-remote",
"has_skills": true,
"mcp_server_count": 0,
"connector_ids": [],
"product_client_id": "test-client"
}
})
}
fn expected_identity() -> PluginEventIdentity<'static> {
PluginEventIdentity {
plugin_id: "sample@openai-curated-remote",
remote_plugin_id: REMOTE_PLUGIN_ID,
plugin_name: "sample",
marketplace_name: "openai-curated-remote",
}
}
fn unique_capture_path(name: &str) -> PathBuf {
let nonce = SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.expect("system clock should be after Unix epoch")
.as_nanos();
std::env::temp_dir().join(format!(
"codex-plugin-analytics-capture-{name}-{}-{nonce}.jsonl",
process::id()
))
}