6 ms·

It's really difficult to see what technologies will be enablers of state power. IMO, the techs ushering in 1984-levels of control are wifi, cheap cameras, and

It's really difficult to see what technologies will be enablers of state power. IMO, the techs ushering in 1984-levels of control are wifi, cheap cameras, and LLMs. The three technologies on their own are beneficial, but taken together they're going to make Big Brother seem impotent.

conception 29d ago
And drones.
xpct 29d ago
I think of it as, we've had a hurdle where computers were too slow to organize the data humans produced, but have since crossed it. LLMs and CV also enable analysis of fuzzy data, which hasn't been possible before. This makes me uneasy, to say the least.
- alansaber 29d ago
  Precisely, modern multimodal LLMs are a breakthrough in ultra high volume data processing, for all their flaws
- matheusmoreira 29d ago
  I asked Claude to analyze and datamine all of my eleven thousand HN comments, plus my reddit comments, lobsters comments, mailing list emails, stackexchange content, etc. With this database it managed to create an entire dossier about me. It could psychologically profile me, and it had a better and more complete view of me than my literal psychologist. We identified things like weaknesses, social engineering susceptibility, likely scenarios. It got to the point it started digging into open source projects I made when I was a teenager and started making eerily accurate inferences about my childhood. I can't even imagine what the NSAs and CIAs of the world are capable of with their limitless black budgets. Anthropic only balked at domestic surveillance, it had zero objections towards allowing the US government to surveil foreign persons like me.
  - xpct 29d ago
    I completely agree. I've done a similar experiment on this rather young HN account and it was already able to deduce some things that I never explicitly stated. It probably doesn't require much more than a megabyte of someone's Slack messages or emails to build a profile on them, and there's much more data out there than that. People can be linked by their footprints in big tech products, or more crudely by techniques like stylometry, which is fully accessible to foreign governments. LLMs will likely enable even more aggressive techniques for linking people's public data, let alone to analyze it.
dogleash 29d ago
Hard disagree. Plenty of new technologies have readily apparent straight-forward application for government use. You're arguing your point by naming a technology that speculative fiction regularly foresees advancements in, and loves using in stories as the enabler of more intrusive state power. More surveillance records, better processing and recall of those records are like THE THING when it comes to more law enforcement and passive coercion by the state. There is honest to goodness disruptive tech that takes time for society to adapt to and reveal where state power wants to go. But those are the vast minority. I think it's a pasttime of industry to play ignorant to how their products will interplay with the government's obvious motivations.

conception 29d ago

And drones.

xpct 29d ago

I think of it as, we've had a hurdle where computers were too slow to organize the data humans produced, but have since crossed it. LLMs and CV also enable analysis of fuzzy data, which hasn't been possible before. This makes me uneasy, to say the least.

alansaber 29d ago
Precisely, modern multimodal LLMs are a breakthrough in ultra high volume data processing, for all their flaws
matheusmoreira 29d ago
I asked Claude to analyze and datamine all of my eleven thousand HN comments, plus my reddit comments, lobsters comments, mailing list emails, stackexchange content, etc. With this database it managed to create an entire dossier about me. It could psychologically profile me, and it had a better and more complete view of me than my literal psychologist. We identified things like weaknesses, social engineering susceptibility, likely scenarios. It got to the point it started digging into open source projects I made when I was a teenager and started making eerily accurate inferences about my childhood. I can't even imagine what the NSAs and CIAs of the world are capable of with their limitless black budgets. Anthropic only balked at domestic surveillance, it had zero objections towards allowing the US government to surveil foreign persons like me.
- xpct 29d ago
  I completely agree. I've done a similar experiment on this rather young HN account and it was already able to deduce some things that I never explicitly stated. It probably doesn't require much more than a megabyte of someone's Slack messages or emails to build a profile on them, and there's much more data out there than that. People can be linked by their footprints in big tech products, or more crudely by techniques like stylometry, which is fully accessible to foreign governments. LLMs will likely enable even more aggressive techniques for linking people's public data, let alone to analyze it.

dogleash 29d ago

Hard disagree. Plenty of new technologies have readily apparent straight-forward application for government use. You're arguing your point by naming a technology that speculative fiction regularly foresees advancements in, and loves using in stories as the enabler of more intrusive state power. More surveillance records, better processing and recall of those records are like THE THING when it comes to more law enforcement and passive coercion by the state. There is honest to goodness disruptive tech that takes time for society to adapt to and reveal where state power wants to go. But those are the vast minority. I think it's a pasttime of industry to play ignorant to how their products will interplay with the government's obvious motivations.