[Stk] MIDI event timing relative to audio stream

[Tim Grant]

Thanks, Arthur.

Not all commercial synths discretize events to the frame boundaries. For
example, it's possible to trigger Native Instruments Battery with sample
accuracy when sequenced via a DAW, and even when being triggered via
external MIDI it does so with sub-frame accuracy within the limits of the
host MIDI driver. I have no illusions of achieving sample accuracy, but if
I can achieve sub-frame accuracy I will be happy. My intended application
is percussion-oriented so low jitter is a very desirable property for me. I
understand that RtMidi has been designed for ease of use, and not
necessarily for the highest performance.

I'm measuring the delta between MIDI events in the callback using a
high_resolution_clock, and the delta times differ from those reported by
the callback somewhat significantly, sometimes > 1ms. I suppose this is due
to the low precision of the timestamps with the Windows MM API? Or is the
timestamp reliable, and there is a variable time between the MIDI event and
the callback invocation? I haven't measured the jitter in the audio
callback yet. If the time between audio callbacks is reasonably stable then
the system clock might be a reliable-enough reference.

Also, perhaps this is a bug, but the delta time for the first event is not
zero, but some (often large) value like 4291232.2280 seconds. If that's
unexpected then I can file an issue on GitHub.

Thanks,

Tim

On Tue, Apr 12, 2022 at 4:38 PM Arthur Benilov <arthur.benilov at gmail.com>
wrote:

> Hi Tim,
> You can timestamp the MIDI events relative to the next audio frame using
> the system steady clock. The very first MIDI event received will have delta
> time zero, all following events will have delta time measured relative to
> the first event. By relating these to the system clock and the system time
> recorded at the audio callback you can estimate for a particular MIDI event
> its time within the next audio frame. Also you don't need to be
> sample-perfect, all commercial software synths I know will discretize the
> MIDI events processing within an audio frame to 16 or 32
> samples @44.1-48kHz. You may also let the jitter be and just decrease the
> audio frame size to minimize it (Raspberry Pi will be able to do 256
> samples per frame @44.1kHz with its onboard audio), it may not be that
> critical for an instrument you are developing.
>
> Cheers,
>
> Arthur.
>
>
> On Tue, Apr 12, 2022 at 10:06 PM Tim Grant <tim at rootdown.net> wrote:
>
>> Hello!
>>
>> I've recently started prototyping a synthesizer project using RtAudio and
>> RtMidi. I'm doing the initial development in Windows and I plan to
>> eventually port the project to something like a Raspberry Pi if the
>> prototype goes well.
>>
>> I have the audio output and MIDI input working, and I'm starting to think
>> about triggering the synth voices with MIDI notes. I would like to keep the
>> jitter between the notes and the triggered sounds low, but I'm not quite
>> sure how to achieve this with these two APIs.
>>
>> A diagram might be helpful here:
>> MIDI      -----N----------------------N-------------------------->
>> Input          |                      |
>>                |                      |
>> Audio     |----*-----|----P-----|-----*----|-----P----|---------->
>> Stream       Frame 1    Frame 2    Frame 3    Frame 4    Frame 5
>>
>> My thinking is that I need to know at what point in the current audio
>> frame an MIDI note arrives. From the diagram, assume a note (N) arrives at
>> the moment corresponding to the 5th sample in Frame 1; knowing this, I can
>> begin playback at the 5th sample on the next frame (P). This way, the
>> latency is held constant at 1 frame and the amount of jitter is minimized.
>>
>> So with that long-winded introduction out of the way, my question is:
>>
>> Is it possible to recover the timing of MIDI events relative to the audio
>> stream? I can't see how to do this using the delta time provided with the
>> MIDI note and the stream time provided by RtAudio, but my hope is that I'm
>> missing something.
>>
>> Thanks!
>>
>> Tim
>> _______________________________________________
>> Stk mailing list
>> Stk at ccrma.stanford.edu
>> https://cm-mail.stanford.edu/mailman/listinfo/stk
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://cm-mail.stanford.edu/pipermail/stk/attachments/20220412/cc4f8ff8/attachment-0001.html>