Windows AI structured JSON generation intermittently returns no progress or completion #6871

Describe the bug

Microsoft.Windows.AI.Text.LanguageModel.GenerateStructuredJsonResponseAsync intermittently returns its async operation promptly but then produces neither progress callbacks nor a completed result within a 25-second observation window. The same question can complete through ordinary GenerateResponseAsync with streaming progress.

This reproduces in a standalone, MSIX-packaged WinUI C# app using only Windows App SDK and .NET APIs. There are no MAUI, Microsoft.Extensions.AI, chat adapters, or test-runner dependencies.

Repro project: https://github.com/mattleibow/windows-ai-structured-output-repro

Pinned source: https://github.com/mattleibow/windows-ai-structured-output-repro/tree/f269673e9cea6cfbdcfba1fdc3dc4c07e90d7e20

Access note: the repro repository is currently private. Access will need to be arranged before the team can browse it. The prompt, native API calls, schema and measured results are also included below.

Steps to reproduce the bug

1. Use a supported Windows AI device with the language model ready. The repro's LanguageModel.GetReadyState() returned Ready.
2. Build and run the repro using .\Run.ps1 -SdkVariant current on Windows with Developer Mode enabled.
3. Leave the prompt as who is the fastest gun in the west?, with the default User: prefix enabled, and select Structured JSON.
4. Observe that LanguageModel.CreateAsync() completes and GenerateStructuredJsonResponseAsync() returns its async object, but the operation can remain incomplete with zero progress callbacks until the app's 25-second wait bound.
5. Restart the process and select Plain text for comparison. Repeat trials in fresh processes; success in one trial does not rule out the intermittent problem.
6. Repeat using -SdkVariant previous to test the previous experimental SDK package set. The launch script verifies the registered MSIX layout and the loaded native AI DLL SHA256 so that Windows does not silently activate a previously registered build.

Automatic examples (one new process per trial):

.\Run.ps1 -SdkVariant current -Case plain -Label current-plain-1
.\Run.ps1 -SdkVariant current -Case structured -Label current-json-1 -SkipBuild
.\Run.ps1 -SdkVariant previous -Case structured -Label previous-json-1

The relevant code directly calls the native APIs on a worker thread so a blocking native entry point cannot freeze WinUI:

using var model = await LanguageModel.CreateAsync();
string prompt = "User: who is the fastest gun in the west?";
var operation = model.GenerateStructuredJsonResponseAsync(
    prompt, schema, new LanguageModelOptions());
operation.Progress = (_, text) => { /* record timestamp and text length */ };
var result = await operation;

Plain-text comparison:

using var context = model.CreateContext();
var operation = model.GenerateResponseAsync(
    context, prompt, new LanguageModelOptions());
operation.Progress = (_, text) => { /* record timestamp and text length */ };
var result = await operation;

The four-field schema is passed as a literal string, without runtime transformation:

{
  "type": "object",
  "properties": {
    "summary": {
      "description": "A brief, accurate answer to the user's request. Do not invent details not supported by the prompt.",
      "type": "string"
    },
    "keyPoints": {
      "description": "Specific points supporting the summary. Use an empty list when there are no distinct supporting points.",
      "type": "array",
      "items": { "type": "string" }
    },
    "category": {
      "description": "A short, broad category for the response, such as General, Technical, or Creative.",
      "type": "string"
    },
    "sentiment": {
      "description": "Classify the overall tone. Use Neutral for objective or factual replies.",
      "type": "string",
      "enum": ["Neutral", "Positive", "Negative", "Mixed"]
    }
  },
  "additionalProperties": false,
  "required": ["summary", "keyPoints", "category", "sentiment"]
}

Expected behavior

The native operation should complete with a response or report a failure status/error. If inference is still progressing, progress callbacks would help distinguish slow constrained decoding from a stalled operation.

The 25-second wait bound is a diagnostic limit in the repro, not an assertion that the API promises a 25-second completion deadline or proof of an infinite deadlock.

Screenshots

Not included. The repro contains 13 timestamped JSON reports in Evidence/, recording entry/return boundaries, native statuses, progress counts, OS version and loaded native DLL fingerprints. Evidence/README.md summarizes them.

NuGet package version

Loaded native Microsoft.Windows.AI.Text.dll:

• Current: file version 2.5.3.10969, SHA256 DACC7657BBD5880EAF1EC23B4A1F3CB7E65A11F4C1F27186F453087409258EF0.
• Previous: file version 2.4.7.28936, SHA256 77E09F777C6129E918CAE2C3FCA4664F6E675A2E57615BE353E29D2F210688A4.

Packaging type

Packaged (MSIX), self-contained Windows App SDK runtime, systemAIModels capability declared. x64.

Windows version

Windows 11 26200.9448, AMD Ryzen AI 7 PRO 350. .NET SDK 10.0.401.

IDE

Other: dotnet build and PowerShell registration/activation scripts.

Additional context

Measured standalone WinUI results:

• Current SDK, prefixed plain text: completed in 7.88 seconds, 42 progress callbacks.
• Current SDK, prefixed four-field JSON: four trials reached the 25-second bound with zero progress callbacks and no result.
• Previous SDK, prefixed four-field JSON: two trials reached the same bound with zero progress callbacks and no result.
• Current SDK, unprefixed four-field JSON: also reached the bound.
• Previous SDK, unprefixed four-field JSON: completed in 5.14 seconds, with no intermediate progress.
• A minimal one-string schema without descriptions, arrays, enum or additionalProperties completed in 3.08 seconds on the current SDK and 5.46 seconds on the previous SDK. The repro supports -MinimalSchema -Raw for this comparison.
• Later plain-text trials on both versions also blocked in native CreateContext() until the wait bound, so the problem is not necessarily confined to structured decoding once the underlying runtime is in a failing state.

The repro does not retry, rewrite prompts, fabricate progress, or treat timed-out waits as successful responses. It requests native cancellation when the bounded wait expires and disables further requests if the worker has not finished; each subsequent comparison uses a fresh process.

Rolling back the NuGet SDK did not eliminate the four-field prefixed case on this machine. This is not a claim that the latest SDK introduced the issue or that the underlying cause has been established. OS model packages and shared inference services were not reset between trials; changing the SDK does not roll those back.