Skip to content

Add stereo PCM rendering, semantic buses and DSP effects - #608

Merged
freezy merged 10 commits into
masterfrom
feat/audio-processing
Oct 25, 2026
Merged

freezy merged 10 commits into
masterfrom
feat/audio-processing

Conversation

@freezy

@freezy freezy commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Table audio needs a shared processing stage for EQ, dynamics and room effects. This adds an opt-in native stereo PCM renderer with seven semantic buses, followed by live stereo DSP. Existing configurations keep standard playback. Supported voices share one carrier without duplicate playback; custom mixer routes and unsupported or over-budget clips retain their existing route with diagnostics.

Each bus and the stereo output have eight EQ bands and a stereo-linked soft-knee compressor. A shared Signalsmith reverb return accepts independent post-processing bus sends; ROM defaults to no reverb. A final stereo-linked sample-peak limiter uses a fixed 1 ms lookahead with matching dry delay. Parameters are normalized off the audio thread, graph updates and retirement use bounded queues, and rendering performs no allocation, locks or file access. Pause, mute, unload and device reset clear or restore state safely, including saved master gain before first playback.

The renderer owns clip cursors, loops, pitch, panning and gain ramps. Optional bank bus IDs survive .vpe packaging, including unknown IDs and legacy defaults. ROM takeover requires an exclusive playback lease and explicit integration gain metadata; raw PCM remains available to other subscribers. Player-owned processing preferences are excluded from tables. Signalsmith sources are pinned, two local fixes are documented, and runtime resources include MIT/Boost notices.

An additive optional recorded-clip preview API auditions three bounded flipper strokes through the selected bus and DSP while game voices remain paused. Clip and bus are published atomically; stop and sample-rate changes cancel it safely. Legacy ABI and preview calls remain compatible. The companion Player bundles the recording and exposes clear effect sections, per-table scope, retained private drafts and an explicit Apply footer.

Validation:

  • Core and Player compilation passed using Unity reference assemblies, including the exact proposed commits with unrelated working edits excluded.
  • Windows x64/x86 and Linux native routing, PCM and DSP suites passed, including stereo linking, limiter ceilings, reverb reset, handoff, zero callback allocations and recorded-preview pause/resampling/stereo/stop/duration/rate-change cases.
  • Eight Unity processing-settings tests passed during the DSP implementation; 37 final standalone production-package UI/runtime checks passed, including the real atomic Player store, Apply rollback/retry, retained scope drafts, cabinet scrolling and bidirectional native EQ curves.
  • Standalone stereo/live-effects/pause/mute/reset/teardown checks passed. T2 retains 41 banks and 214 readable clips; table assignments and packaged audio banks are unchanged.
  • Fable 5.1 and Astra 6 reviewed before commit. Their navigation, painter-lifetime and curve findings were corrected and verified; no actionable findings remain.

Companion Player PR #11 provides controls, presets and table overrides; PinMAME PR #48 supplies integration gain metadata. Native binaries are generated by CI rather than committed. Physical SSF, adaptive ROM drift and non-Windows playback qualification remain future work. Processing remains opt-in stereo.

Low-latency output and plugin audio

Three further commits measure and cut the mixer's output latency, then route plugin audio through it.

  • audio: play the central PCM mixer through a native WASAPI stream. Unity's output path added its four-block ring plus FMOD's device buffer, about 40 to 50 ms on top of what Windows needs. The graph now renders on its own event-driven WASAPI shared stream (MMCSS thread, one engine period plus 3 ms queued, follows default-device changes, widens its margin per underrun once settled), with Unity's carrier as fallback.
  • audio: add exclusive WASAPI and ASIO output modes for the central mixer. Exclusive WASAPI negotiates the device's own format. ASIO lives in a separate GPLv3 library (VisualPinball.Native~/Asio) that links the Steinberg ASIO SDK; one owner thread drives the driver, reset requests and rate changes reinitialize it, and the driver's control panel can be opened with or without a table.
  • audio: route plugin audio through the central mixer. Seven more live inputs (PcmAudioStream), SoundPlayback.Play(source, bus) for plugin clips (listener-relative pan, Unity-style rolloff, mono copy for 3D voices), and GamelogicAudioPump for emulators that pulled their queue in a Unity filter. Buffered inputs have a jitter buffer sized from both sides' blocks and are steered against clock drift. The ROM stream previously drifted late without bound (141 ms queued on ASIO); it now holds about 22 ms.

Measured from a flipper key to the audio interface's own output (its loopback input, recorded through ASIO): Unity output 174 ms, native shared WASAPI 132 ms, exclusive 133 ms, ASIO at 64 frames 14 ms. ASIO4ALL on the same interface stayed at 115 to 122 ms, since it goes through the device's Windows driver. Plugin audio through ASIO: a stream push reaches the device in about 14 ms, a plugin clip in 8 ms.

Native routing and PCM suites (3,051 checks, including live WASAPI output, stream reuse, drift and burst cases) pass on Windows x64. Code review rounds found and fixed: a carrier/device handover race, ASIO reinitialization on reset and rate change, pause on fallback, stream slot reuse and producer races, VLC timing, and video channel layouts. VpeAsio.dll is not built by CI yet; without it the Player's ASIO choice falls back to normal output.

Companion PRs: Player #11 (output settings), Kiki, P-ROC and PinUP (plugin audio routing), and the B2S backglass.

@greptile-apps

greptile-apps Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The latest changes appear safe to merge and address all three numbered previous findings.

Summary

Adds opt-in stereo PCM playback with semantic buses, shared effects, native Windows output options, and plugin audio inputs. Standard playback remains available.

  • Supported table sounds can play through a shared stereo mixer and device output.
  • Each sound bus and the stereo mix can use live effects.
  • ROM and plugin audio can join the shared mixer as live PCM.
  • Audio developers can build an isolated probe for routing PCM channels.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Clips[Supported clips] --> Buses[Seven sound buses]
  Inputs[ROM and plugin PCM] --> Buses
  Buses --> Effects[Bus and output effects]
  Effects --> Output{Output route}
  Output --> ASIO[ASIO]
  Output --> WASAPI[WASAPI]
  Output --> Unity[Unity carrier]
  Fallback[Unsupported clips and pause-menu sounds] --> Reference[Unity playback]
Loading

Reviews (7) · Last reviewed commit: "audio: keep the mixer's clock running th..." · Reviewed by Greptile

@freezy freezy changed the title Add opt-in stereo PCM rendering and semantic audio buses Add stereo PCM rendering, semantic buses and DSP effects Oct 23, 2026
@freezy
freezy force-pushed the feat/audio-processing branch from a1738a5 to f94c8b3 Compare October 24, 2026 13:05
Unity's output path adds its four-block mixer ring and FMOD's own device
buffer, about 40-50 ms on top of what Windows needs. The central graph now
renders on its own event-driven WASAPI shared-mode stream on an MMCSS
thread, keeping one engine period plus 3 ms queued. The stream follows
default-device changes, reopens lost devices while keeping the graph's
clock running, and widens its margin by 1 ms per underrun once settled.

PcmAudioRenderer prefers the device stream (PreferDeviceOutput) and keeps
Unity's carrier as the fallback and on other platforms. Pause state is
applied before the stream's prefill renders.

Flipper key to Windows loopback went from 109-112 ms to 62-71 ms.
The central PCM graph can now reach the device three ways on Windows:
shared WASAPI (the default), exclusive WASAPI, or an ASIO driver.

Exclusive mode negotiates the device's own format (float, 32-, 24- or
16-bit) at the graph's rate, with a period from the requested buffer size
or the device minimum. A live switch to it fails while Unity's own output
uses the device, so the status says it is retried at the next table load.

ASIO lives in a separate GPLv3 library (VisualPinball.Native~/Asio) that
links the Steinberg ASIO SDK and renders the graph from the driver's
buffer callback. One owner thread drives the driver; reset requests and
rate changes reinitialize it at the graph's rate. The driver's control
panel can be opened with or without a table playing.

PcmAudioRenderer gets OutputMode, AsioDriver and OutputBufferFrames,
falls back ASIO/exclusive -> shared -> Unity, reopens the output live,
and gates Unity's carrier callback before a device stream takes over.

On a USB interface, flipper key to device output measured 139 ms shared,
133 ms exclusive and 14 ms through the vendor's ASIO driver; ASIO4ALL
went through the device's Windows driver and stayed at 115-122 ms.
The native graph gets seven more live PCM inputs besides the ROM stream,
each on a bus with one producer (VpePcmStreamOpen and friends); a slot is
only reused once the renderer saw every close. Buffered inputs wait for a
jitter buffer sized from the producer's and the renderer's largest recent
blocks, and are steered to it: up to 0.3% faster or slower against clock
drift, and a jump back after a burst. Overflows only flush unbuffered
inputs.

The ROM stream uses the adaptive mode, which plays at once until its
first underflow but is steered from the start. Before, an emulator that
ran ahead kept its audio late until the ring overflowed (141 ms queued on
ASIO); it now holds about 22 ms.

For plugins: PcmAudioStream (OpenStream on the renderer) for audio they
produce themselves, SoundPlayback.Play(source, bus) for their clips, with
listener-relative pan, Unity-style rolloff and a mono copy for 3D voices,
and GamelogicAudioPump, which drains a gamelogic engine's emulator queue
into the ROM stream while the mixer holds the playback lease.

Through ASIO at 64 frames, a stream push reaches the device in about
14 ms and a plugin clip in 8 ms.
…ix the macOS build

A fading loop handed its full level to the central mixer before the fade began, so it could start loud for a frame. The Unity carrier now restarts after an audio reset while paused too, so previews keep playing. The device output's error code is unused off Windows.
Comment thread VisualPinball.Native~/Asio/VpeAsio.cpp Resolved
… reads by the lease

While an ASIO reset cannot reopen the driver, the owner thread renders the elapsed time so one-shots finish. The gamelogic pump reads and publishes a block under the lease gate, so no block is lost when the mixer hands playback back. Plugin sounds that ignore the listener's pause stay on Unity playback.
@freezy
freezy merged commit 05605ca into master Oct 25, 2026
21 checks passed
@freezy
freezy deleted the feat/audio-processing branch October 25, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant