4 ms·

Except with the right (obvious) non linear filter you can easily extract orders of magnitude smaller signals randomly inserted in a white noise That is the bas

Except with the right (obvious) non linear filter you can easily extract orders of magnitude smaller signals randomly inserted in a white noise That is the basics of how people in astronomy clean their signal. http://beauty-of-imagination.blogspot.fr/2012/09/fun-with-signal-processing-and.html http://beauty-of-imagination.blogspot.fr/2012/09/fun-with-si...

raverbashing 9y ago
It's interesting how the blog post think this is surprising, and how some basic techniques are rediscovered The average filter is basically correlating the input signal with a (window sized) square wave It's one of the most basic denoising techniques available (for AWGN)
- SFjulie1 9y ago
  My experience is that it is hard convincing most CTO and developers (including one with a background in sound engineering) I met that moving average were not fit for most time series aggregation.
- kortex 9y ago
  "The moving average is the most common filter in DSP, mainly because it is the easiest digital filter to understand and use. In spite of its simplicity, the moving average filter is optimal for a common task: reducing random noise while retaining a sharp step response. This makes it the premier filter for time domain encoded signals. However, the moving average is the worst filter for frequency domain encoded signals, with little ability to separate one band of frequencies from another." It definitely depends on your domain, but moving average/median can be extremely effective. (Or terrible, if mis-applied). http://www.analog.com/media/en/technical-documentation/dsp-book/dsp_book_Ch15.pdf http://www.analog.com/media/en/technical-documentation/dsp-b...
cornholio 9y ago
A good pseudorandom generator will pass any such test with flying colours, in fact it's a basic requirement. Bu that is no guarantee that, given a certain length of output, an external attacker couldn't sync up and predict all future outputs. And I don't believe there is a general way to distinguish such a device from an actual random source without looking inside. For example, they could use AES256 output in counter mode on a low entopy chip ID + high entropy backdoor key. Trivial to bruteforce given a few words of output, but you would essentially need to break AES to detect it.

raverbashing 9y ago

It's interesting how the blog post think this is surprising, and how some basic techniques are rediscovered The average filter is basically correlating the input signal with a (window sized) square wave It's one of the most basic denoising techniques available (for AWGN)

SFjulie1 9y ago
My experience is that it is hard convincing most CTO and developers (including one with a background in sound engineering) I met that moving average were not fit for most time series aggregation.
kortex 9y ago
"The moving average is the most common filter in DSP, mainly because it is the easiest digital filter to understand and use. In spite of its simplicity, the moving average filter is optimal for a common task: reducing random noise while retaining a sharp step response. This makes it the premier filter for time domain encoded signals. However, the moving average is the worst filter for frequency domain encoded signals, with little ability to separate one band of frequencies from another." It definitely depends on your domain, but moving average/median can be extremely effective. (Or terrible, if mis-applied). http://www.analog.com/media/en/technical-documentation/dsp-book/dsp_book_Ch15.pdf http://www.analog.com/media/en/technical-documentation/dsp-b...

cornholio 9y ago

A good pseudorandom generator will pass any such test with flying colours, in fact it's a basic requirement. Bu that is no guarantee that, given a certain length of output, an external attacker couldn't sync up and predict all future outputs. And I don't believe there is a general way to distinguish such a device from an actual random source without looking inside. For example, they could use AES256 output in counter mode on a low entopy chip ID + high entropy backdoor key. Trivial to bruteforce given a few words of output, but you would essentially need to break AES to detect it.