7 ms·

Clever hacker needs to look up a Bloom Filter

by buckhx 2mo ago

deleted 2mo ago
[deleted]
Lukas_Skywalker 2mo ago
Lazy commenters need to look at the README to see that the bloom filter is mentioned as a next step. The hacker is indeed clever.
- amelius 2mo ago
  Only if the hacker also tests for false positives.
  - Lukas_Skywalker 2mo ago
    The way it's worded in the readme makes it sound as if this was exactly the intention.
    - amelius 2mo ago
      Very clever hacker indeed! May even have Sonnet or Opus level of intelligence.
      - docjay 2mo ago
        As someone with an intelligence near Opus I have to be clear with you: your premise is wrong. Having **intelligence** is not the same as being **clever** — a person can cleverly pass a difficult test **without** having or acquiring intelligence. That’s not a revelation, that’s common-sense. You’re also conflating **quantum superposition** and **macroscopic observations** — one is physics, the other is psychology. The exact same thing cannot exist in two places at the same time above 10⁻¹⁰ meters, and intelligence is not measurable using most displacement methods. It is not possible for Opus, Sonnet, and the person to all have the same intelligence. That’s just physics. I also noticed that you did not use any **punctuation** in your comment, so I must firmly decline continuing this line of inquiry. # Correction I reviewed your comment again and I can see that you **did** use punctuation. I retract that statement, but I stand by my assertion that being intelligent is typically more important than being clever.
amazingamazing 2mo ago
> how it could grow : Bloom filter in RAM as a fast pre-filter (skip flash for the ~99% of misses) https://github.com/M-Abozaid/esp32-c3-adblock https://github.com/M-Abozaid/esp32-c3-adblock
Graziano_M 2mo ago
I opened the article and immediately ctrl-f'd for the word "bloom". Though the author plans to use them.
rustyhancock 2mo ago
The README mentions it and conventionally it seems right. But back of the envelope it seems you're shaving not much off if the latency is primarily WiFi latency (tens of milliseconds) not flash reads (microseconds?). If you use loads of transient clients perhaps a LRU cache would help since round trips might be quite slow.
intheitmines 2mo ago
What do people use bloom filters for in production? I've only ever used as part of a blacklist
- gtowey 2mo ago
  Search partition pruning in distributed databases is one concrete application I'm aware of. Hashtables optimized for really large item counts; it can be a useful fast pre-check for item existence.
rurban 2mo ago
Double space. Go from 42 bit to 46 bit and you get the same. Still wondering about the dynamic binary search. A static search can be faster, even on an AVR

Clever hacker needs to look up a Bloom Filter

by buckhx 2mo ago

Clever hacker needs to look up a Bloom Filter

deleted 2mo ago

[deleted]

Lukas_Skywalker 2mo ago

Lazy commenters need to look at the README to see that the bloom filter is mentioned as a next step. The hacker is indeed clever.

amelius 2mo ago
Only if the hacker also tests for false positives.
- Lukas_Skywalker 2mo ago
  The way it's worded in the readme makes it sound as if this was exactly the intention.
  - amelius 2mo ago
    Very clever hacker indeed! May even have Sonnet or Opus level of intelligence.
    - docjay 2mo ago
      As someone with an intelligence near Opus I have to be clear with you: your premise is wrong. Having **intelligence** is not the same as being **clever** — a person can cleverly pass a difficult test **without** having or acquiring intelligence. That’s not a revelation, that’s common-sense. You’re also conflating **quantum superposition** and **macroscopic observations** — one is physics, the other is psychology. The exact same thing cannot exist in two places at the same time above 10⁻¹⁰ meters, and intelligence is not measurable using most displacement methods. It is not possible for Opus, Sonnet, and the person to all have the same intelligence. That’s just physics. I also noticed that you did not use any **punctuation** in your comment, so I must firmly decline continuing this line of inquiry. # Correction I reviewed your comment again and I can see that you **did** use punctuation. I retract that statement, but I stand by my assertion that being intelligent is typically more important than being clever.

amazingamazing 2mo ago

> how it could grow : Bloom filter in RAM as a fast pre-filter (skip flash for the ~99% of misses) https://github.com/M-Abozaid/esp32-c3-adblock https://github.com/M-Abozaid/esp32-c3-adblock

Graziano_M 2mo ago

I opened the article and immediately ctrl-f'd for the word "bloom". Though the author plans to use them.

rustyhancock 2mo ago

The README mentions it and conventionally it seems right. But back of the envelope it seems you're shaving not much off if the latency is primarily WiFi latency (tens of milliseconds) not flash reads (microseconds?). If you use loads of transient clients perhaps a LRU cache would help since round trips might be quite slow.

intheitmines 2mo ago

What do people use bloom filters for in production? I've only ever used as part of a blacklist

gtowey 2mo ago
Search partition pruning in distributed databases is one concrete application I'm aware of. Hashtables optimized for really large item counts; it can be a useful fast pre-check for item existence.

rurban 2mo ago

Double space. Go from 42 bit to 46 bit and you get the same. Still wondering about the dynamic binary search. A static search can be faster, even on an AVR