It's all about the Software

FABADA : "FABADA is a novel non-parametric noise reduction technique which arise from the point of view of Bayesian inference that iteratively evaluates possible smoothed models of the data, obtaining an estimation of the underlying signal that is statistically compatible with the noisy measurements. Iterations stop based on the evidence $E$ and the $\chi^2$ statistic of the last smooth model, and we compute the expected value of the signal as a weighted average of the smooth models. You can find the entire paper describing the new method in (link will be available soon)."

FABADA by https://github.com/PabloMSanAla , https://github.com/paranoya

I have taken it upon myself to implement FABADA for audio. With the help of justin engel to improve stream handling, I now have a very safe version of this program with automatic audio device serialization. The code, I have spent the last week optimizing and improving, and now it is perfected. I have also spoken with one of the FABADA algorithm authors and perhaps some more improvements are possible there. The program does not crash, and it completes its work on my machine within the cpu time allocated. A lot of the code is JIT compiled. This allows the program to process the audio in near realtime.

There is some delay due to the size of buffer it processes. There is also a small problem with some clicking idk how to stop.

This program accepts audio in a variety of bandwidths, however, you must set sample_rate to the sample rate of your audio device, ie 44100, 48000, etc,
and if you change this you must also optimize the time it is allocated to complete each cycle.

Instructions:
https://github.com/falseywinchnet/fabad ... _buffer.py
Download and follow the instructions in this python script.
The binary release is actually slower than this.

By stacking it with other noise reduction methods that do not do very well with noise that is sporadic and diverse, i have achieved nice results.
That is to say, this program doesn't reduce noise as much as it smooths it out. I am now done with it and someone else can take it from here.


I guess i wasn't done with it.
1/12/2022 I still havnt managed to reduce the clicking but the code is now highly, highly, highly vectorized and JIT'd to LLVM behind the scenes.
CPU wise it is not as intensive as it was.
1/15/2022 im back to consuming all of your CPU, now with a GUI.

Aren't the filter settings differemt?

I am willing to bet Simon is correct and fabada simply does not understand the formats that go beyond 2 gigabytes.
{o.o}

I'm willing to bet you're not a programmer jdow .-.

LSB8E wrote: Thu Dec 23, 2021 2:05 pm Aren't the filter settings differemt?

Won't change the waterfall. I was testing to see how FABADA handles the distortion caused by Microsoft RT on AM signals.

- delete- please see first post

delete - see first post

It doesn't work on IQ samples.
.

rainstar wrote: Thu Dec 23, 2021 6:48 pm I'm willing to bet you're not a programmer jdow .-.

Yup, I fooled 'em completely for dozens of years. I also rather unfairly have a little more experience nurse maiding RIFF (and IFF before that) based audio and video applications.

I also notice you are intent on moving us back into the 80s for audio sample rates. People who wear big boy and girl clothes use 48ksps (or multiples for SDR needs or snakeoil purposes.)

{^_-} (You started it.)

I posted this gauging interest. I am poor, unpaid, dumb, disabled, and have not coded seriously in over a decade. This is the most amount of non-javascript code I've woven together in 10 years, and it's mostly other people's code. Merely accomplishing this has been really difficult for me. I cannot get anyone else to just make it for me, and I have enough competing interests and challenges and anxiety over how computers work and complications mentally associated with my disability that I cannot commit the attention span or determination needed to truthfully master the higher logic required to fully understand this. To me, it's magic. All of it.

I do NOT care for any criticism or commentary on my effort, jdow. NONE. Not "helpful pointers in the right directions" not "unpack my efforts" not "you're doing x wrong". I have my family to load me up with all of that which I will ever need. It is degrading to compare me to more experienced people, and does not motivate me to "do better". I like compliments and help. I like it when people do things for me, I think everyone does. If you want to help, I have some suggestions. If you want to compliment my efforts, I would appreciate it. For everything else, you can send a letter to santa.

I tried to apply a rfft/irfft bandpass filter to the output, and while the filter works, the problem is that the code doesn't run fast enough. At larger sample sizes, the code doesn't run fast enough. I would, of course, be interested in the code running faster. Then we could see the effects of real-time processing of various algorithms. If you want to contribute, maybe take a look at the FABADA source code and see if there is anything in there that could be optimized, perhaps by using C. It is an open source project and the main function consists of a very small amount of algorithmic logic.

On the other hand, if you, just as myself, no longer really code, and since you do claim to be more experienced, I could use some help replacing the buffer. I want to either use https://github.com/justengel/np_rw_buffer or https://pypi.org/project/dvg-ringbuffer/ I don't know which one would be faster. I have not been successful in using either, but I am poking along at it.