pwsh fails at startup in AttemptExecPwshLogin (sysctl) on Unix platforms other than Linux and macOS, and child pwsh processes repeat the check #28283

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.

Steps to reproduce

The setup is PowerShell 7.6.7 (powershell-7.6.7-linux-x64-musl-noopt-fxdependent) on a Unix-like system that .NET identifies as neither Linux nor macOS: both OperatingSystem.IsLinux() and OperatingSystem.IsMacOS() are false. In our case this is WitOS, a .NET port in the Unix form over musl's libc. Any other Unix that PowerShell can run on, such as FreeBSD, would take the same path.

1. Start pwsh with __PWSH_LOGIN_CHECKED=1 in its environment. Without the variable, even this first pwsh fails as shown below.
2. From that session, start another pwsh (for example pwsh -c Get-Date), or run a program that starts commands through $env:SHELL (a file manager running a typed command, for instance).

Expected behavior

The second pwsh starts. On a platform where the login-shell detection cannot work (no /proc, and no sysctl with KERN_PROCARGS2), the detection should be skipped instead of failing.

Actual behavior

The second pwsh exits at once:

Unhandled exception. System.EntryPointNotFoundException: Unable to find an entry point named 'sysctl' in shared library 'libc'.
   at Microsoft.PowerShell.ManagedPSEntry.SysCtl(Int32* mib, Int32 mibLength, Void* oldp, Int32* oldlenp, IntPtr newp, Int32 newlenp)
   at Microsoft.PowerShell.ManagedPSEntry.AttemptExecPwshLogin(String[] args)
   at Microsoft.PowerShell.ManagedPSEntry.Main(String[] args)

Error details

There are two causes, both in AttemptExecPwshLogin in src/powershell/Program.cs:

1. The macOS branch runs on every non-Linux Unix. Linux is handled through /proc/self/cmdline. Every other platform then falls into the code commented "At this point, we are on macOS", which calls sysctl(KERN_PROCARGS2). musl does not export sysctl, so on a non-Linux, non-macOS Unix the call throws before any login detection happens.
2. The workaround variable is not passed on. __PWSH_LOGIN_CHECKED avoids the call, but the method then removes the variable from the process environment (Environment.SetEnvironmentVariable(LOGIN_ENV_VAR_NAME, null)). As a result, every pwsh started by a pwsh goes through the detection again and fails. This happens whether the second pwsh is started directly or by a program the first one runs. Setting the variable globally therefore helps only the outermost shell.

A possible change is to run the sysctl branch only when Platform.IsMacOS, and on any other platform to return without the login exec, as on a non-login start.

Environment data

PSVersion        7.6.7
.NET             10.0.12
OS               WitOS (a Unix-form .NET port over musl), x64

Visuals

No response