Files
codex/codex-rs/app-server-test-client/src/plugin_analytics_capture.rs
T
jameswt-oai 8a40200880 [codex-app-server-test-client] Plugin Install/Uninstall Analytics Smoke Test (#27100)
## This PR

The original [combined remote plugin analytics PR
#26281](https://github.com/openai/codex/pull/26281) mixed reusable
analytics test infrastructure, two manual smoke workflows, a metadata
refactor, and the final identity behavior. This PR adds the
account-mutating validation workflow separately so its cleanup and
recovery guarantees can be reviewed without the final analytics behavior
change.

- Add a manually invoked remote plugin install/uninstall smoke workflow.
- Require explicit account-mutation confirmation and an initially
uninstalled plugin.
- Validate the current `codex_plugin_installed` contract, where
`plugin_id` is the backend ID.
- Restore and verify the original uninstalled state, with a dedicated
recovery command.

This baseline intentionally does not require `codex_plugin_uninstalled`,
because production does not emit that event yet. The final PR will
update this smoke to require local `plugin_id`, `remote_plugin_id`, and
uninstall emission. Review this PR as the net diff against #27099.

## Testing

- `just test -p codex-app-server-test-client` (3 focused
capture/validation tests passed)
- The live workflow was previously exercised on the green combined
reference branch, and the original uninstalled account state was
restored.
- CI is green across the required platform matrix.

## Split Overview

```text
main
├── #27093  Debug analytics capture
│   └── #27099  Non-mutating plugin smoke
│       └── #27100  Remote install/uninstall smoke  ← you are here
└── #27102  Plugin telemetry metadata refactor

After #27093, #27099, #27100, and #27102 merge:
└── Final PR: add remote_plugin_id to plugin analytics
```

Review order and dependencies:

1. [#27093 Add debug-only analytics event
capture](https://github.com/openai/codex/pull/27093) (based on `main`)
2. [#27099 Add a plugin analytics smoke
workflow](https://github.com/openai/codex/pull/27099) (stacked on
#27093)
3. [#27100 Add a remote plugin analytics mutation smoke
workflow](https://github.com/openai/codex/pull/27100) **(this PR,
stacked on #27099)**
4. [#27102 Centralize plugin telemetry metadata
construction](https://github.com/openai/codex/pull/27102) (independent,
based on `main`)
5. Final remote-ID behavior PR (created after PRs 1-4 merge)

The original [#26281](https://github.com/openai/codex/pull/26281)
remains open as the green aggregate reference until the final PR is
published.
2026-06-16 12:28:45 -07:00

103 lines
3.0 KiB
Rust

use anyhow::Context;
use anyhow::Result;
use anyhow::bail;
use serde_json::Value;
use std::fs;
use std::io;
use std::path::Path;
pub(super) fn read_events_for_remote_plugin(
path: &Path,
remote_plugin_id: &str,
) -> Result<Vec<Value>> {
let contents = match fs::read_to_string(path) {
Ok(contents) => contents,
Err(err) if err.kind() == io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(err) => {
return Err(err).with_context(|| format!("read capture file {}", path.display()));
}
};
let mut matching = Vec::new();
for (index, line) in contents.lines().enumerate() {
if line.trim().is_empty() {
continue;
}
let payload: Value = serde_json::from_str(line).with_context(|| {
format!(
"parse analytics capture line {} from {}",
index + 1,
path.display()
)
})?;
let events = payload["events"]
.as_array()
.context("analytics capture payload is missing events")?;
matching.extend(
events
.iter()
.filter(|event| event["event_params"]["plugin_id"] == remote_plugin_id)
.cloned(),
);
}
Ok(matching)
}
pub(super) struct PluginEventIdentity<'a> {
pub(super) plugin_id: &'a str,
pub(super) plugin_name: &'a str,
pub(super) marketplace_name: &'a str,
}
pub(super) fn validate_mutation_events(
events: Vec<Value>,
expected: PluginEventIdentity<'_>,
) -> Result<Vec<Value>> {
let event_type = "codex_plugin_installed";
let matching = events
.iter()
.filter(|event| event["event_type"] == event_type)
.collect::<Vec<_>>();
let [event] = matching.as_slice() else {
bail!(
"expected exactly one `{event_type}` event for `{}`, found {}",
expected.plugin_id,
matching.len()
);
};
validate_event(event, &expected)?;
Ok(vec![(*event).clone()])
}
fn validate_event(event: &Value, expected: &PluginEventIdentity<'_>) -> Result<()> {
let params = &event["event_params"];
require_string(params, "plugin_id", expected.plugin_id)?;
require_string(params, "plugin_name", expected.plugin_name)?;
require_string(params, "marketplace_name", expected.marketplace_name)?;
for field in [
"has_skills",
"mcp_server_count",
"connector_ids",
"product_client_id",
] {
if params.get(field).is_none_or(Value::is_null) {
bail!(
"{} event has null or missing `{field}`",
event["event_type"]
);
}
}
Ok(())
}
fn require_string(params: &Value, field: &str, expected: &str) -> Result<()> {
let actual = params.get(field).and_then(Value::as_str);
if actual != Some(expected) {
bail!("expected `{field}` to be `{expected}`, got {actual:?}");
}
Ok(())
}
#[cfg(test)]
#[path = "plugin_analytics_capture_tests.rs"]
mod tests;