[BUG] IPC timeout on unbind in signal-start-stop test

[kv2019i]

Describe the bug
Starting with 11th June 2025 daily test on Intel PTL, we see failures in check-signal-stop-start-playback-50.sh test.

@ujfalusi did a test with today's SOF kernel and yesterday's FW and issue is not seen, so it would seem this is a regression caused by Zephyr baseline update in #10046

To Reproduce
check-signal-stop-start-playback-50.sh

Reproduction Rate
High

Expected behavior
No IPC timeout

Impact
Loss of audio functionality

Environment

1. Branch name and commit hash of the 2 repositories: sof (firmware/topology) and linux (kernel driver).
   • Kernel: 80c081426c0c
   • SOF: 97f869cda8cb
2. Name of the topology file
   • Topology: multiple, HDA, nocodec, SDW
3. Name of the platform(s) on which the bug is observed.
   • Platform: Intel ptl

Screenshots or console output

[ 2692.795135] kernel: snd_sof:sof_widget_free_unlocked: sof-audio-pci-intel-ptl 0000:00:1f.3: widget host-copier.31.playback freed
[ 2692.795147] kernel: snd_sof:sof_ipc4_route_free: sof-audio-pci-intel-ptl 0000:00:1f.3: unbind modules gain.15.1:0 -> mixin.15.1:0
[ 2692.795157] kernel: snd_sof:sof_widget_free_unlocked: sof-audio-pci-intel-ptl 0000:00:1f.3: widget gain.15.1 freed
[ 2692.795168] kernel: snd_sof:sof_ipc4_route_free: sof-audio-pci-intel-ptl 0000:00:1f.3: unbind modules mixin.15.1:0 -> mixout.1.1:0
[ 2692.795180] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc tx      : 0x4600100a|0x100b: MOD_UNBIND
[ 2693.297990] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc timed out for 0x4600100a|0x100b
[ 2693.298017] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: IPC timeout
[ 2693.298059] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: failed to unbind modules mixin.15.1:0 -> mixout.1.1:0
[ 2693.298091] kernel: snd_sof:sof_ipc4_log_header: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc tx      : 0x12000000|0x0: GLB_DELETE_PIPELINE
[ 2693.801982] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc timed out for 0x12000000|0x0
[ 2693.802010] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: IPC timeout
[ 2693.802054] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: failed to free pipeline widget pipeline.15

[kv2019i]

FYI @tmleman @ujfalusi @lgirdwood

[lgirdwood]

@serhiy-katsyuba-intel any PM flows around stream start/stop with your Zephyr update ?

[serhiy-katsyuba-intel]

@serhiy-katsyuba-intel any PM flows around stream start/stop with your Zephyr update ?

I did not make any recent changes to Zephyr or update Zephyr rev in SOF. #10046 is made by @tmleman . Tomek may know more if there were some PM related changes in Zephyr recently.

[kv2019i]

@lyakh and @ujfalusi looking at this. We can reassign once bisect is complete.

Currently bug seems to have appeared between:

bff7cdbfbe54 (Fix alsa and topology building
97f869cda8cb (seppo/main, origin/main, origin/HEAD, main) intel_adsp: switch to default PM policy for ACE configurations
707ccae1f4a7 west.yml: update zephyr to 8c6b2c8c887
ad809eb21f95 topology2: add sof-sdw-generic.tplg
a50bde5f40a7 module-adapter: remove a superfluous check
d2a8a8972858 audio: ipc4: enable xrun notifications by default
e4e600137a14 mixer: Add support for underrun notifications
96f841af60e8 ipc4: notification: Add function to prepare mixer underrun notification
fb1d34b821c4 ipc4: module: interface: module_adapter: Add support for Config Get/Set ipc
cca4d8707a56 ipc4: module: Add definitions for Config Get / Set ipc
9018bc4a5cdf zephyr: dai: Add support for xrun notifications in dai
c11934dba567 zephyr: host: Add support for xrun notifications in host
7815e318adec chain_dma: xrun notification update
e95a313de44a ipc4: notification: Add functions to prepare ipc xrun notifications
82f85e404209 boards: ace30: enable telemetry and io perf measurements
7110f4e3496e boards: ace30: increase Debug Window size
287d174806a3 audio: mic_privacy: Fix DSP panic during D3 entry with HW managed mode

[kv2019i]

@lyakh made a fix #10055 that is merged and seems to help. Let's monitor daily reports for a few days and then close.

[ekurdybx]

@lyakh @ujfalusi
Issue seems to resurface again on test multiple-pipeline-all-50:

[ 3227.286729] kernel:  Jack Out: ASoC error (-110): trigger FE cmd: 0 failed
[ 3227.791653] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc timed out for 0x12040000|0x0
[ 3227.791661] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: IPC timeout
[ 3227.791692] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: failed to free pipeline widget pipeline.15
[ 3227.791702] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: Failed to free connected widgets
[ 3227.791706] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: sof_pcm_stream_free: pcm31 (Deepbuffer Jack Out), dir 0: sof_widget_list_free failed -110
[ 3227.791714] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: ASoC error (-110): at snd_soc_pcm_component_hw_free() on 0000:00:1f.3
[ 3228.053510] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: FW reported error: 12 - Required resource is in invalid state
[ 3228.053525] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: ipc error for msg 0x12020000|0x0
[ 3228.053532] kernel: sof-audio-pci-intel-ptl 0000:00:1f.3: failed to free pipeline widget pipeline.0

Kernel commit: 3e755313f509
SOF commit: 117e460b47a6
Platform: PTL

[lyakh]

Issue seems to resurface again on test multiple-pipeline-all-50:

@ekurdybx where does this log excerpt come from? Is this from a CI run or from your local testing?

[ekurdybx]

@lyakh The logs came from CI fail, however after disscussing this fail, we decided to create inner jira ticket for this issue.