Clay Sound Plugin

Hybrid synth + sampler + tracker audio plugin for Clayground. One coherent engine, two wrappers for pre-existing assets (Sound, Music), and a small instrument/sequencer layer on top (SynthInstrument, SampleInstrument, SongPlayer, ChipMood/MoodPlayer). Offline render is first-class and deterministic. See docs/docs/manual/dojo.md for how to hot-reload song files without resetting the playhead (.dojoignore).

Getting Started


import Clayground.Sound

Components

Sound

For short sound effects (fire-and-forget). Supports overlapping playback with instance pooling for efficiency. 

Sound {
    id: jumpSound
    source: "sounds/jump.wav"
    volume: 0.8
    lazyLoading: false  // preload by default
}

// Usage
onJumped: jumpSound.play()
onReset: jumpSound.stop()  // stops all playing instances

Music

For background music with full playback controls. 

Music {
    id: bgMusic
    source: "music/theme.mp3"
    volume: 0.5
    loop: true
}

// Usage
onGameStarted: bgMusic.play()
onGamePaused: bgMusic.pause()
onGameOver: bgMusic.stop()

Properties

Sound

| Property | Type | Default | Description | |----------|------|---------|-------------| | source | url | "" | Audio file URL (local or remote) | | volume | real | 1.0 | Volume level (0.0-1.0) | | lazyLoading | bool | false | If true, load on first play() | | loaded | bool | readonly | Whether audio data is loaded | | status | enum | readonly | Null/Loading/Ready/Error |

Music

| Property | Type | Default | Description | |----------|------|---------|-------------| | source | url | "" | Audio file URL | | volume | real | 1.0 | Volume level (0.0-1.0) | | lazyLoading | bool | false | If true, load on first play() | | loaded | bool | readonly | Whether audio data is loaded | | playing | bool | readonly | Currently playing | | paused | bool | readonly | Currently paused | | loop | bool | false | Loop playback | | position | int | readonly | Current position (ms) | | duration | int | readonly | Total duration (ms) |

SynthInstrument

Real-time oscillator voice with ADSR, pitch envelope, LFO. Trigger notes by MIDI number or Hz. Also bakes to WAV. 

SynthInstrument {
    id: lead
    waveform: "square"      // sine | square | triangle | sawtooth | noise
    attack: 0.005; decay: 0.08; sustain: 0.5; release: 0.15
    volume: 0.8
}
// Fire a note
lead.triggerNote(69, 0.9, 0.25)
// Synth-to-sample bounce
var wavPath = lead.bake(69, 0.4)

SampleInstrument

PCM sample playback with loop points, root note, ADSR on top of samples. 

SampleInstrument {
    id: drum
    source: "kick.wav"
    rootNote: 60
    volume: 0.9
}
drum.triggerOneShot(0.8)

SongPlayer

Plays a .song.json against QML instruments resolved by objectName. Supports play/pause/stop/seek/loop and hot-reload without playhead reset. 

SynthInstrument { id: lead; objectName: "demoLead" }
SynthInstrument { id: bass; objectName: "demoBass" }

SongPlayer {
    source: "songs/demo.song.json"
    instruments: [lead, bass]
    loop: true
    onHotReloaded: console.log("song file changed, kept playing")
}
Song file format (v1): json { "tempo": 110, "tracks": { "lead": { "instrument": "demoLead" }, "bass": { "instrument": "demoBass" } }, "patterns": { "A": { "lead": [ { "t": 0, "note": "C5", "dur": 0.5 }, { "t": 0.5, "note": "E5", "dur": 0.5 } ], "bass": [{ "t": 0, "note": "C3", "dur": 2.0 }] } }, "sections": [ { "pattern": "A", "repeat": 4 } ] } Notes can be MIDI numbers or scientific pitch strings (C4, F#3, Bb5). Defaults: dur=0.5 beats, vel=0.8.

Platform Support

  • WASM: full support — the same QAudioSink-driven engine as desktop,
    • through Qt Multimedia's emscripten backend. See "What Music cannot do on WASM" below.
  • Desktop/Mobile: Full support — all types above work end-to-end.

    • What Music cannot do on WASM

    Qt's WebAssembly media backend plays a Music source through an HTML element and reports almost nothing back, so on the web (and only there):
  • status and loaded stay at their initial values until the track ends
  • duration and position stay 0
  • loop has no effect — the track plays once
    • A background loop on the web therefore needs a Sound re-triggered by a Timer at the clip length, the way Music was worked around before it played at all (see #216).

    Technical Notes

  • Audio is fully preloaded before playback (no streaming)
  • WASM: Web Audio API requires user gesture to start AudioContext
  • WASM: Remote URLs must be CORS-enabled
  • Desktop: Uses QAudioSink for all in-engine types, QMediaPlayer for Music
  • Hot-reload: SongPlayer watches its source file; drop a .dojoignore
    • (songs/ or *.song.json) next to your Sandbox.qml to prevent the dojo from reloading the whole scene on song edits