## Summary
Adds a FeatureFlag to enforce UTF8 encoding in powershell, particularly
Windows Powershell v5. This should help address issues like #7290.
Notably, this PR does not include the ability to parse `apply_patch`
invocations within UTF8 shell commands (calls to the freeform tool
should not be impacted). I am leaving this out of scope for now. We
should address before this feature becomes Stable, but those cases are
not the default behavior at this time so we're okay for experimentation
phase. We should continue cleaning up the `apply_patch::invocation`
logic and then can handle it more cleanly.
## Testing
- [x] Adds additional testing
Previous to this PR, we used a hand-rolled PowerShell parser in
`windows_safe_commands.rs` to take a `&str` of PowerShell script see if
it is equivalent to a list of `execvp(3)` invocations, and if so, we
then test each using `is_safe_powershell_command()` to determine if the
overall command is safe:
https://github.com/openai/codex/blob/6e6338aa876bb4258abe25b02ac6417b8ea9dff0/codex-rs/core/src/command_safety/windows_safe_commands.rs#L89-L98
Unfortunately, our PowerShell parser did not recognize `@(...)` as a
special construct, so it was treated as an ordinary token. This meant
that the following would erroneously be considered "safe:"
```powershell
ls @(calc.exe)
```
The fix introduced in this PR is to do something comparable what we do
for Bash/Zsh, which is to use a "proper" parser to derive the list of
`execvp(3)` calls. For Bash/Zsh, we rely on
https://crates.io/crates/tree-sitter-bash, but there does not appear to
be a crate of comparable quality for parsing PowerShell statically
(https://github.com/airbus-cert/tree-sitter-powershell/ is the best
thing I found).
Instead, in this PR, we use a PowerShell script to parse the input
PowerShell program to produce the AST.
I am trying to tighten up some of our logic around PowerShell over in
https://github.com/openai/codex/pull/7607 and it would be helpful to be
more precise about `pwsh.exe` versus `powershell.exe`, as they do not
accept the exact same input language.
To that end, this PR introduces utilities for detecting each on the
system. I think we also want to update `get_user_shell_path()` to return
PowerShell instead of `None` on Windows, but we'll consider that in a
separate PR since it may require more testing.
## Summary
Enables shell_command for windows users, and starts adding some basic
command parsing here, to at least remove powershell prefixes. We'll
follow this up with command parsing but I wanted to land this change
separately with some basic UX.
**NOTE**: This implementation parses bash and powershell on both
platforms. In theory this is possible, since you can use git bash on
windows or powershell on linux. In practice, this may not be worth the
complexity of supporting, so I don't feel strongly about the current
approach vs. platform-specific branching.
## Testing
- [x] Added a bunch of tests
- [x] Ran on both windows and os x
Dylan Hurd ·
Michael Bolin ·