From bil at ccrma.Stanford.EDU Sun Aug 6 07:12:52 2023 From: bil at ccrma.Stanford.EDU (bil at ccrma.Stanford.EDU) Date: Sun, 06 Aug 2023 07:12:52 -0700 Subject: [CM] Snd 23.6 Message-ID: <6fc5f5ad0e780285dd6a92d50b1b7b3e@ccrma.stanford.edu> Snd 23.6: various small improvements and bugfixes in Snd and s7. checked: sbcl 2.3.7 Thanks!: Kenneth Flak From anders at avinjar.no Tue Aug 15 07:40:53 2023 From: anders at avinjar.no (Anders Vinjar) Date: Tue, 15 Aug 2023 16:40:53 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) Message-ID: <87pm3ofjwq.fsf@avinjar.no> Hi everyone, perhaps someone here has suggestions, og refs to techniques, examples or further info? I want to remove as much-as-possible harmonic content from sound input, leaving only the noise part. Think acoustic guitar, or close-miked piano, leaving the 'mechanics' only. Ultimately i'll need a real-time solution. Perhaps something like a LPC-based approach which may be used in real-time? Or perhaps trying to subtract a certain part of the result of a FFT-analysis? Or some other approach? Anyone tried? All suggestions or refs are very welcome! :-) Thanks! -anders From juanig at ccrma.Stanford.EDU Tue Aug 15 10:20:38 2023 From: juanig at ccrma.Stanford.EDU (Juan Reyes) Date: Tue, 15 Aug 2023 10:20:38 -0700 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: <87pm3ofjwq.fsf@avinjar.no> References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: Hi Anders, If I recall Juan Pampin's ATS [1] would do what you are describing. Lisp version was good but I have not used it in fifteen years (CMUCL) and not so sure it will work on SBCL. Meaning that it would probably need some toggling, if someone has some free time. Juan and Pablo de Liscia also had some C++ binaries that worked 'realtime' but have not tried them recently. They might need recompilatation. If you get ATS working I'm positive you'll get what you want. --* Juan Reyes [1] https://dxarts.washington.edu/wiki/analysis-transformation-and-synthesis-ats > I want to remove as much-as-possible harmonic content from sound > input, leaving only the noise part. Think acoustic guitar, or > close-miked piano, leaving the 'mechanics' only. > > Ultimately i'll need a real-time solution. > > Perhaps something like a LPC-based approach which may be used in > real-time? > > Or perhaps trying to subtract a certain part of the result of a > FFT-analysis? Or some other approach? Anyone tried? > > All suggestions or refs are very welcome! :-) > From TestCase at asu.edu Tue Aug 15 10:54:59 2023 From: TestCase at asu.edu (Todd Ingalls) Date: Tue, 15 Aug 2023 17:54:59 +0000 Subject: [CM] Pysndlib on pypi Message-ID: <6BB0622F-7BB5-42AB-A8D5-8C67F49FAF54@asu.edu> Hello Everyone After finding the initial version too slow, I am now using cython to generate code that seems to run 4x or more faster that what I had announced earlier. I have also put a package with pre-built wheels on pypi so getting started if you are on linux or macOS should be as easy as doing pip install pysndlib The page is here https://pypi.org/project/pysndlib/ I also have generated docs which are also hopefully helpful https://testcase.github.io/pysndlib/index.html This is all very much a work in progress but things are at least at the stage that I think people might start to find useful . Source code - https://github.com/testcase/pysndlib Todd Ingalls Associate Director School of Arts, Media and Engineering ame.asu.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From snd at oddodd.org Tue Aug 15 12:12:44 2023 From: snd at oddodd.org (NA) Date: Tue, 15 Aug 2023 21:12:44 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: <87pm3ofjwq.fsf@avinjar.no> References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: Hi Anders, Some Harmonic-Percussive Source Separation could help you out. Check the Fluid Corpus Manipulation Toolkit. https://learn.flucoma.org/reference/hpss/ best, Niklas Adam On Tue Aug 15, 2023 at 4:40 PM CEST, Anders Vinjar wrote: > Hi everyone, > > perhaps someone here has suggestions, og refs to techniques, examples or > further info? > > I want to remove as much-as-possible harmonic content from sound input, > leaving only the noise part. Think acoustic guitar, or close-miked > piano, leaving the 'mechanics' only. > > Ultimately i'll need a real-time solution. > > Perhaps something like a LPC-based approach which may be used in > real-time? > > Or perhaps trying to subtract a certain part of the result of a > FFT-analysis? Or some other approach? Anyone tried? > > All suggestions or refs are very welcome! :-) > > Thanks! > > -anders > _______________________________________________ > Cmdist mailing list > Cmdist at ccrma.stanford.edu > https://cm-mail.stanford.edu/mailman/listinfo/cmdist From anders at avinjar.no Tue Aug 15 13:05:34 2023 From: anders at avinjar.no (Anders Vinjar) Date: Tue, 15 Aug 2023 22:05:34 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: (Juan Reyes's message of "Tue, 15 Aug 2023 10:20:38 -0700") References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: <87h6p0f4vl.fsf@avinjar.no> J> Juan and Pablo de Liscia also had some C++ binaries that worked J> 'realtime' but have not tried them recently. They might need J> recompilatation. Any links or suggestions as to where to look for these? It seems ATS in essence finds and tracks the harmonic content as sinusoids, and subtracts this content from the original to get the residual. Seems to be a valid approach to achieve what i need indeed. Do you think it might be possible to use a (simplified?) ATS-approach in realtime, in fex SuperCollider or PD: ie. find and track (harmonic) partials, subtract from original to get the residuals? Perhaps an approach based on autocorrelation could be useful? -anders From anders at avinjar.no Tue Aug 15 13:08:27 2023 From: anders at avinjar.no (Anders Vinjar) Date: Tue, 15 Aug 2023 22:08:27 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: (NA's message of "Tue, 15 Aug 2023 21:12:44 +0200") References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: <87cyzof4qs.fsf@avinjar.no> >>>>> 15 Aug 2023 - "NA" : N> Hi Anders, Some Harmonic-Percussive Source Separation could help N> you out. Check the Fluid Corpus Manipulation Toolkit. N> https://learn.flucoma.org/reference/hpss/ This looks very interesting! I'll try to read me up. Do you know if the approach to signal/noise separation can be used in real-time contexts? -anders From snd at oddodd.org Tue Aug 15 15:19:50 2023 From: snd at oddodd.org (NA) Date: Wed, 16 Aug 2023 00:19:50 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: <87cyzof4qs.fsf@avinjar.no> References: <87pm3ofjwq.fsf@avinjar.no> <87cyzof4qs.fsf@avinjar.no> Message-ID: Yes its very possible and cheap! Of course some latency will be incurred. Check out the FluidHPSS class in the SuperCollider extension Niklas Adam On Tue Aug 15, 2023 at 10:08 PM CEST, Anders Vinjar wrote: > >>>>> 15 Aug 2023 - "NA" : > > N> Hi Anders, Some Harmonic-Percussive Source Separation could help > N> you out. Check the Fluid Corpus Manipulation Toolkit. > N> https://learn.flucoma.org/reference/hpss/ > > This looks very interesting! I'll try to read me up. > > Do you know if the approach to signal/noise separation can be used in > real-time contexts? > > -anders From anders at avinjar.no Wed Aug 16 03:19:13 2023 From: anders at avinjar.no (Anders Vinjar) Date: Wed, 16 Aug 2023 12:19:13 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: (NA's message of "Wed, 16 Aug 2023 00:19:50 +0200") References: <87pm3ofjwq.fsf@avinjar.no> <87cyzof4qs.fsf@avinjar.no> Message-ID: <878rabffxa.fsf@avinjar.no> Hi Niklas, >>>>> 16 Aug 2023 - "NA" : N> Yes its very possible and cheap! Of course some latency will be N> incurred. Check out the FluidHPSS class in the SuperCollider N> extension Thanks! I see it links to FitzGeralds paper fom 2010, as does the code in the existing MedianSeparation class in SC. Seems to be more or less the same basic approach? Do you know where/if they differ? -anders From orm.finnendahl at selma.hfmdk-frankfurt.de Wed Aug 16 06:55:30 2023 From: orm.finnendahl at selma.hfmdk-frankfurt.de (Orm Finnendahl) Date: Wed, 16 Aug 2023 15:55:30 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: Hi Anders, I made the Common Lisp version of ATS work last Christmas. It now works with newest CLM. In addition I reimplemented the resynthesis part as a realtime module in incudine. You can find it here: https://github.com/ormf/ats-cuda The performance is comparable to the original non-realtime CLM version (even slightly more efficient on my machine). I use sbcl on linux and only tested this combination. It might not be all that helpful for your purpose as ATS has an analysis stage that is non realtime. In addition tweaking the analysis params (especially in your case) is probably crucial in order to get the residual signal you'd like to obtain and you'd probably want to adjust the params dynamically, which is not feasible considering the architecture of ATS. I made a browser based gui to interactively scan across the sonagram of the ATS analysis with a mouse. You can narrow down the bandwidth to listen to, modulate the frequencies of the ATS tracks individually and/or crossfade between the sine and residual component in realtime. If you want to try it out let me know and I can give you directions on installation. The interactive part is pre alpha, but it works well and I'm already using it for compositions. Once the ATS file has been calculated it's like an interactive realtime version of Spear with modulation capabilities. Best, Orm ---------------------------------------------------------------------- Prof. Orm Finnendahl Komposition Hochschule f?r Musik und Darstellende Kunst Eschersheimer Landstr. 29-39 60322 Frankfurt am Main https://www.youtube.com/watch?v=2rWha1HTfFE&list=PLiGfneJSWmNw6dTUvcTHbTkCYOOTiB_N6 Am Dienstag, den 15. August 2023 um 10:20:38 Uhr (-0700) schrieb Juan Reyes: > > Hi Anders, > > If I recall Juan Pampin's ATS [1] would do what you are describing. > > Lisp version was good but I have not used it in fifteen years (CMUCL) > and not so sure it will work on SBCL. Meaning that it would probably > need some toggling, if someone has some free time. > > Juan and Pablo de Liscia also had some C++ binaries that worked > 'realtime' but have not tried them recently. They might need > recompilatation. > > If you get ATS working I'm positive you'll get what you want. > > --* Juan Reyes > > > [1] > https://dxarts.washington.edu/wiki/analysis-transformation-and-synthesis-ats > > > > I want to remove as much-as-possible harmonic content from sound > > input, leaving only the noise part. Think acoustic guitar, or > > close-miked piano, leaving the 'mechanics' only. > > > > Ultimately i'll need a real-time solution. > > > > Perhaps something like a LPC-based approach which may be used in > > real-time? > > > > Or perhaps trying to subtract a certain part of the result of a > > FFT-analysis? Or some other approach? Anyone tried? > > > > All suggestions or refs are very welcome! :-) > > > > _______________________________________________ > Cmdist mailing list > Cmdist at ccrma.stanford.edu > https://cm-mail.stanford.edu/mailman/listinfo/cmdist From anders at avinjar.no Wed Aug 16 13:50:30 2023 From: anders at avinjar.no (Anders Vinjar) Date: Wed, 16 Aug 2023 22:50:30 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: (Orm Finnendahl's message of "Wed, 16 Aug 2023 15:55:30 +0200") References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: <87zg2q67ah.fsf@avinjar.no> Hi Orm, thanks for nice suggestions! Yes, ATS in various incarnations has been very useful, and some of its siblings such as sms, spear, pvanal, svp, ceres etc. For off-line processing i'd normally try one of those, or perhaps LPC-based techniques. My problem is the real-time requirements (no off-line pre-analysis), and not-very-large latency. Do you think ATS could provide solutions for those? The Median Filtering approach of FitzGerald is perfect! But needs several frames to track partials to distinguish from non-harmonic content (as ats etc.). Probably as close as i'll come... I was hoping there might be some 'simpler' solution - autocorrelation, wavelets (wild suggestions, but why not...?) Perhaps I could extract typical samples of noise-type qualities expected to come in the input sounds, and 'teach' some analysis technique to recognize something similar? -anders From juanig at ccrma.Stanford.EDU Wed Aug 16 14:15:42 2023 From: juanig at ccrma.Stanford.EDU (Juan Reyes) Date: Wed, 16 Aug 2023 14:15:42 -0700 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: References: <87pm3ofjwq.fsf@avinjar.no> Message-ID: <66341dea-5dbc-4c59-b089-0ebcdfabe536@ccrma.stanford.edu> Hi Orm, Sorry I forgot you had ATS CL version working. Had it on my mind and my unconscious betrayed me. You also beat me on explaining "what you need to know while using spectral modeling". In other words, and like years ago we used to hear, a good analysis and a good FFT of a sound consequently renders a great resynthesis. Getting the ATS analysis file was not definitely realtime. I second your thoughts: > Once the ATS file has been calculated it's like an interactive > realtime version of Spear with modulation capabilities. Digging a bit deeper on the ATS webpage [1], found that Pablo DiLiscia has working Pd binaries (only for windoze though) [2]. There is even source code on GitLab [3]. > I was hoping there might be some 'simpler' solution - > autocorrelation, wavelets (wild suggestions, but why not...?) On speech recognition topics there might be some hints. I don't know of anything in concrete but recall some high density talks on a conference that aimed at accomplishing some of the stuff you are referring. And for what is worth, AI "borrowed" some of these techniques. --* Juan Reyes [1] https://dxarts.washington.edu/wiki/analysis-transformation-and-synthesis-ats#overview [2] https://puredata.info/Members/pdiliscia/ats-pd [3] https://gitlab.com/dxarts/projects/ats From orm.finnendahl at selma.hfmdk-frankfurt.de Thu Aug 17 01:27:06 2023 From: orm.finnendahl at selma.hfmdk-frankfurt.de (Orm Finnendahl) Date: Thu, 17 Aug 2023 10:27:06 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: <66341dea-5dbc-4c59-b089-0ebcdfabe536@ccrma.stanford.edu> References: <87pm3ofjwq.fsf@avinjar.no> <66341dea-5dbc-4c59-b089-0ebcdfabe536@ccrma.stanford.edu> Message-ID: Hi Juan, Am Mittwoch, den 16. August 2023 um 14:15:42 Uhr (-0700) schrieb Juan Reyes: > > Digging a bit deeper on the ATS webpage [1], found that Pablo > DiLiscia has working Pd binaries (only for windoze though) [2]. for the record: There are also Csound opcodes and SuperCollider plugins available. AFAIK the pd binaries don't contain the resynthesis part (a bit like sigmund~, but reading from an ATS file rather than realtime). The Csound and SuperCollider plugins do resynthesis but lack the option to modulate amp or frequency for each sinusoidal track individually (which for me makes all the difference). -- Orm From orm.finnendahl at selma.hfmdk-frankfurt.de Thu Aug 17 01:57:48 2023 From: orm.finnendahl at selma.hfmdk-frankfurt.de (Orm Finnendahl) Date: Thu, 17 Aug 2023 10:57:48 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: <87zg2q67ah.fsf@avinjar.no> References: <87pm3ofjwq.fsf@avinjar.no> <87zg2q67ah.fsf@avinjar.no> Message-ID: Hi Anders, Am Mittwoch, den 16. August 2023 um 22:50:30 Uhr (+0200) schrieb Anders Vinjar: > > My problem is the real-time requirements (no off-line pre-analysis), and > not-very-large latency. Do you think ATS could provide solutions for > those? I don't think so. > The Median Filtering approach of FitzGerald is perfect! But needs > several frames to track partials to distinguish from non-harmonic > content (as ats etc.). Probably as close as i'll come... I completely agree. Although I heard very convincing AI based extractions of drum parts from instrumental tracks and imagine this could work very well for your application case, I would expect you need a fair bit of training data and expertise to make it work. I'd be quite interested to hear about your experiences and results (just post offline). -- Orm From juan at pampin.org Sat Aug 19 09:01:44 2023 From: juan at pampin.org (Juan Pampin) Date: Sat, 19 Aug 2023 12:01:44 -0400 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: References: <87pm3ofjwq.fsf@avinjar.no> <87zg2q67ah.fsf@avinjar.no> Message-ID: Hi All, sorry for not chiming in before, I'm in NY sending my son off to college and have been very busy. Here are are some ideas regarding for Anders request: 1) if you are working with harmonic instruments/sources, you could do this by tracking the F0 and using a bank of notch filters, similar to the way hum memoval algorithms work. I got good results doing this in real time in SC with relatively stable harmonic sound sources. 2) You could also do the same as in #1 using the FFT. You would still need to detect the F0, then remove (or attenuate) the bins of the magnitude spectrum corresponding to the harmonic spectrum, and finally use the IFFT to resynthesize the sound. This should be done on a frame by frame basis using OLA. 3) If your sound doesn't have a harmonic spectrum (or you don't know) you could use a technique similar to #2 except that instead of F0 tracking you do peak detection on the magnitude spectrum, then once the main peaks are detected (you could use a threshold in dB to define them) you remove or attenuate them from the magnitude spectrum and then use then use the IFFT to resynthesize the sound. This is done on a frame by frame basis, using OLA, no tracking is required. You can look at the peak-detection algorithm used in ATS (or SMS for that matter). I had some very good results using this method in real time with strings and wind instruments. 4) Same as #3 but doing peak trekking and resynthesis. In this case you could try to track the peaks across a certain number of frames, then remove them from the FFT magnitude spectrum before resynthesis. This could work fine if you track stable sinusoids over a few frames, you could try the ATS tracking algorithm for this method. 5) you could also use a frequency and time-domain algorithm with #3 and #4, using phase information. In this case once peaks are detected you would remove them by synthesizing them using the phase information and then remove them by subtracting them from the time domain signal (frame). For #4 this would require phase tracking using high-order interpolation but it is doable in real time. This residual generation method is the one used by ATS (and SMS) offline. OK, I hope this is helpful. Cheers, JUAN On Thu, Aug 17, 2023 at 4:58?AM Orm Finnendahl < orm.finnendahl at selma.hfmdk-frankfurt.de> wrote: > Hi Anders, > > Am Mittwoch, den 16. August 2023 um 22:50:30 Uhr (+0200) schrieb > Anders Vinjar: > > > > My problem is the real-time requirements (no off-line pre-analysis), and > > not-very-large latency. Do you think ATS could provide solutions for > > those? > > I don't think so. > > > The Median Filtering approach of FitzGerald is perfect! But needs > > several frames to track partials to distinguish from non-harmonic > > content (as ats etc.). Probably as close as i'll come... > > I completely agree. Although I heard very convincing AI based > extractions of drum parts from instrumental tracks and imagine this > could work very well for your application case, I would expect you > need a fair bit of training data and expertise to make it work. > > I'd be quite interested to hear about your experiences and results > (just post offline). > > -- > Orm > _______________________________________________ > Cmdist mailing list > Cmdist at ccrma.stanford.edu > https://cm-mail.stanford.edu/mailman/listinfo/cmdist > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From snd at oddodd.org Mon Aug 28 03:15:50 2023 From: snd at oddodd.org (NA) Date: Mon, 28 Aug 2023 12:15:50 +0200 Subject: [CM] removing harmonic content from sound (solutions for real-time use) In-Reply-To: <878rabffxa.fsf@avinjar.no> References: <87pm3ofjwq.fsf@avinjar.no> <87cyzof4qs.fsf@avinjar.no> <878rabffxa.fsf@avinjar.no> Message-ID: Yeah I think its a direct implementation. Except for possibly the other filtering modes that are offered in the Fluid class. n On Wed Aug 16, 2023 at 12:19 PM CEST, Anders Vinjar wrote: > Hi Niklas, > > >>>>> 16 Aug 2023 - "NA" : > > N> Yes its very possible and cheap! Of course some latency will be > N> incurred. Check out the FluidHPSS class in the SuperCollider > N> extension > > Thanks! > > I see it links to FitzGeralds paper fom 2010, as does the code in the > existing MedianSeparation class in SC. > > Seems to be more or less the same basic approach? Do you know where/if > they differ? > > -anders