Audacity- Need a math geek

[ghazzer]

The only code I have written in the last 20 years was some lengthy Visual Basic macros for EXCEL, so almost no help here.

Are there any parameters you could list or sub functions you need that others might help with? I am trying to imagine some kind of flow diagram (if they are still used) that might help you define what you are attempting to do.

Seems there is an app for everything these days. Are there any apps for converting code between languages or platforms?

 

[ConradH]

Got my PowerBASIC program working with S-G filters last night. Way easier than I thought as one can just use the pre-calculated coefficients commonly available. No need to reinvent the wheel. The filter action does exactly what I want on a section of digitized LP that was damaged, though I'm not sure it's better than a simple triangle moving average. I'll clean the demo program up a bit just because, but now it's on to learning about the Nyquist language and/or maybe the LADSPA dlls. The attached demo file lets you see the effectiveness of various filters and how many passes one has to make. (somebody let me know if the demo unzips and runs OK, as I'm still using an old steam powered XP machine here.)

 

[ghazzer]

The noisetest.wav file is only 0.023 seconds long.

When I run the demo executable it prompts me to load 5,7,9,11, or 13 point S-G files, but they all appear to be the same. I apparently don't know how to get sound out of the demo.

 

[ConradH]

Yes, the demo file is short so you can see the waveform. The filters will vary in effect, with the 5 point being very mild and the 13 point quite aggressive, depending on the number of times you hit the smooth button. I've uploaded a newer version that locks out the buttons and exits if the file is bad- a couple things that would crash it. It doesn't have a play function because data that's short enough to view is too short to listen to.

 

[loopstick]

I was able to run the demo and go through the different filters and revert back to the original. A good filter follows the Hippocratic Oath - do no harm. This may explain why the higher order SG filters aren't "mashing" the waveform - but still seem to be removing crud. I'm guessing that the higher order SG's will be better at zapping out actual impulse type noise. IOW the least squares fitting that the SG apparently does is better able to ignore anomalies than the simple "triangle" - which I'm assuming is a gaussian approximation with a footprint of three.

edit - actually I don't think it's the "least squares" thing that reduces sensitivity to anomalies (vinyl pops and crackles) - it's the fact that the order of the filter is too low to "track" the anomalies and the anomalies therefore get ignored more than they would in a weighted average approach.

 

[ConradH]

The source is easy to follow in spite of using some pointer techniques specific to PowerBASIC, rather than conventional arrays. Here's the triangle sub and the 13 point S-G sub for comparison-

SUB TriangularSmooth(nSamples AS LONG, dLen1 AS LONG)           'triangular smoothing
    LOCAL i AS LONG
    FOR i = 3 TO nSamples - 3
        @sData[i] = (@pData[i - 2] + 2 * @pData[i - 1] + 3 * @pData [i]+ 2 * @pData[i + 1] + @pData[i + 2]) / 9
    NEXT
    MEMORY COPY sData, pData, dLen1                             'copy filtered data to reference array for next run
END SUB 

SUB SG_Smooth13(nSamples AS LONG, dLen1 AS LONG)                'Savitzky-Golay filter, # of points = 13
    LOCAL i AS LONG
    FOR i = 6 TO nSamples - 6
        @sData[i] = (-11 * @pData[i - 6] + 0 * @pData[i - 5] + 9 * @pData[i - 4] + 16 * @pData[i - 3] + 21 * @pData[i - 2] _
                    + 24 * @pData[i - 1] + 25 * @pData[i] + 24 * @pData[i + 1] + 21 * @pData[i + 2] _
                    + 16 * @pData[i + 3] + 9 * @pData[i + 4] + 0 * @pData[i + 5] - 11 * @pData[i + 6]) / 143
    NEXT
    MEMORY COPY sData, pData, dLen1                             'copy filtered data to reference array for next run
END SUB

It's not at all clear to me how to do this with the Nyquist language, but like all things it should eventually make some sense. I hope.

edit- Fixed some code weirdness in the post with code tags. Doesn't like brackets without it.

 

[ghazzer]

Tried it again and the demo seems to function the same on my WIN7 and WIN10 laptops. The only problem was a subtle reminder that I have exceeded the 40 day trial period on the RAR file reader currently installed.

Your S-G filter seems very subtle, and precise. Most of the Pop & Click removers I have seen in the past just take too much of the "music" out of a WAV file, even for my decrepit hearing. I have avoided them and only work on the worst clicks & pops by hand. Tedious, but it preserves the "music".

Well done!

 

[audiotemp]

Reference 14 in the paper that Conrad linked to in post #8 gives quite a good explanation of S-G, what it is doing, and how changing the parameters affects the frequency domain response of the filter. (It contains math, so if necessary avert your eyes and just look at the pictures.) It is available (free) here, https://inst.cs.berkeley.edu/~ee123/fa12/docs/SGFilter.pdf

 

[loopstick]

The source is easy to follow in spite of using some pointer techniques specific to PowerBASIC, rather than conventional arrays. Here's the triangle sub and the 13 point S-G sub for comparison-

SUB TriangularSmooth(nSamples AS LONG, dLen1 AS LONG)           'triangular smoothing
    LOCAL i AS LONG
    FOR i = 3 TO nSamples - 3
        @sData[i] = (@pData[i - 2] + 2 * @pData[i - 1] + 3 * @pData [i]+ 2 * @pData[i + 1] + @pData[i + 2]) / 9
    NEXT
    MEMORY COPY sData, pData, dLen1                             'copy filtered data to reference array for next run
END SUB

SUB SG_Smooth13(nSamples AS LONG, dLen1 AS LONG)                'Savitzky-Golay filter, # of points = 13
    LOCAL i AS LONG
    FOR i = 6 TO nSamples - 6
        @sData[i] = (-11 * @pData[i - 6] + 0 * @pData[i - 5] + 9 * @pData[i - 4] + 16 * @pData[i - 3] + 21 * @pData[i - 2] _
                    + 24 * @pData[i - 1] + 25 * @pData[i] + 24 * @pData[i + 1] + 21 * @pData[i + 2] _
                    + 16 * @pData[i + 3] + 9 * @pData[i + 4] + 0 * @pData[i + 5] - 11 * @pData[i + 6]) / 143
    NEXT
    MEMORY COPY sData, pData, dLen1                             'copy filtered data to reference array for next run
END SUB

It's not at all clear to me how to do this with the Nyquist language, but like all things it should eventually make some sense. I hope.

edit- Fixed some code weirdness in the post with code tags. Doesn't like brackets without it.

Interesting - it looks like the triangle is five-wide. Which means you could replace the linear weighting with perhaps less artifacty gaussian weights. BTW - the Wikipedia article on SG filters seems to want to hype convolution as the "secret sauce" of the method. But convolution is just the method of application of any kernel-based filter. The real secret sauce is the way the weights are developed - IOW how the filter itself is developed - not the mechanism of how it's applied.

I'm not familiar with the Nyquist language but I'm sure you'll figure it out. BTW - notice the weights in the SG. Not only are they not linear but there are changes in sign. This indicates sophistication and high-frequency properties.

 

[ConradH]

Turns out Nyquist language has a convolve instruction. Just need to figure out how to use it. It's a high level language with lots of fancy instructions, but seems to be a bit lacking when manipulating individual samples.

 

[audiotemp]

Turns out Nyquist language has a convolve instruction. Just need to figure out how to use it. It's a high level language with lots of fancy instructions, but seems to be a bit lacking when manipulating individual samples.

Don't manipulate the individual samples. Take your music sample of (length n), and convolve it with the list of S-G weights (length m). You may have to pad the latter to length n with zeros and/or convert it to look like a sound sample first (dunno, I didn't look closely). Then take that m+n (or 2n, if you have to pad) length thing and pick out the length n-m section that is what you want by dropping terms off the front and back. Welcome to Lisp, my friend.

 

[audiotemp]

Nyquist convolution usage demo (using SAL) here:

https://sourceforge.net/p/nyquist/code/HEAD/tree/trunk/nyquist/demos/convolution.htm
https://sourceforge.net/p/nyquist/code/HEAD/tree/trunk/nyquist/demos/src/convolve.sal

 

[ConradH]

Makes sense. I found a demo of an FIR filter that's somewhat enlightening. Having used various Basics over about three decades, starting with the original IBM compiler, and never having had any attraction to C or anything else, I'm a bit set in my ways. For me the LISP welcome mat is printed, "get off my lawn", "beware of dog" and "abandon ye all hope".