6 ms·

Show HN: Is AI hijacking your intent? A formal control algorithm to measure it

10 points by daikikadowaki 8mo ago

I’m an independent researcher proposing State Discrepancy, a public-domain metric to quantify how much an AI system changes a user’s intent (“the Ghost”). The goal: replace vague legal and philosophical notions of “manipulation” with a concrete engineering variable. Without clear boundaries, AI faces regulatory fog, social distrust, and the risk of being rejected entirely. Algorithm 1 (on pp.16–17 of the linked white paper) formally defines the metric: 1. D = CalculateDistance(VisualState, LogicalState) 2. IF D < α : optimization (Reduce Update Rate) 3. ELSE IF α ≤ D < β : warning (Apply Visual/Haptic Modifier proportional to D) 4. ELSE IF β ≤ D < γ : intervention (Modulate Input / Synchronization) 5. ELSE : security (Execute Defensive Protocol) The full paper is available on Zenodo: https://doi.org/10.5281/zenodo.18206943 https://doi.org/10.5281/zenodo.18206943

daikikadowaki 8mo ago
Hi HN, I recently submitted a white paper on State Discrepancy (D) to the EU AI Office (CNECT-AIOFFICE). This paper, "The Judgment Transparency Principle (JTP)," is my attempt to provide a mathematical foundation for the right to human autonomy in the age of black-box AI. Philosophy: Protecting the Future While Enabling Speed • Neutral Stance: I side with neither corporations nor regulators. I advocate for the healthy coexistence of technology and humanity. • Preventing Rupture: History shows that perceiving new tech as a “controllable threat” often triggers violent Luddite movements. If AI continues to erode human agency in a black box, society may eventually reject it entirely. This framework is meant to prevent that rupture. Logic of Speed: Brakes Are for Racing • A Formula 1 car reaches top speed because it has world-class brakes. Similarly, AI progress requires precise boundaries between “assistance” and “manipulation.” • State Discrepancy (D) provides a math-based Safe Harbor, letting developers push UX innovation confidently while building system integrity by design. The Call for Collective Intelligence: Why I Need Your Strength I have defined the formal logic of Algorithm V1. However, providing this theoretical foundation is where my current role concludes. The true battle lies in its realization. Translating this framework into high-dimensional, real-world systems is a monumental challenge—one that necessitates the specialized brilliance of the global engineering community. I am not stepping back out of uncertainty, but to open the floor. I have proposed V1 as a catalyst, but I am well aware that a single mind cannot anticipate every edge case of such a critical infrastructure. Now, I am calling for your expertise to stress-test it, tear it apart, and refine it right here. I want this thread to be the starting point for a living standard. If you see a flaw, point it out. If you see a better path, propose it. The practical brilliance that can translate this "what" into a robust, scalable "how" is essential to this mission. Whether it be refining the logic or engineering the reality, your strength is necessary to build a better future for AI. Let’s use this space to iterate on V1 until we build something that truly safeguards our collective future. Anticipating Pushback: • “Too complex?” If AI is safe, why hide its correction delta? • “Bad for UX?” A non-manipulative UX only benefits from exposing user intent. Calling it “too complex” admits a lack of control; calling it “bad for UX” admits reliance on hiding human-machine boundaries. If this framework serves as a mere stepping stone for you to create something superior—an algorithm that surpasses my own—it would be my greatest fulfillment. Beyond this point, the path necessitates the contribution of all of you. Let us define the path together.
- daikikadowaki 8mo ago
  For example, a critical engineering challenge lies in the high-dimensional mapping of 'Logical State'. While Algorithm 1 defines the logic, implementing CalculateDistance() for a modern LLM involves normalizing vectors from a massive latent space in real-time. Doing this without adding significant latency to the inference loop is a non-trivial optimization problem. I invite ideas on how to architect this 'Observer' layer efficiently.
- kingkongjaffa 8mo ago
  > If AI continues to erode human agency in a black box What do you mean by this? Is there evidence this has happened? > I advocate for the healthy coexistence of technology and humanity. This means whatever you want it to mean at any given time, I don't understand this point without further elaboration.
  - daikikadowaki 8mo ago
    Thanks for the direct push. Let me ground those statements in the framework of the paper: 1. On "eroding human agency in a black box": I am referring to "Agency Misattribution". When Generative AI transitions from a passive tool to an active agent, it silently corrects and optimizes human input without explicit consent. The evidence is observable in the psychological shift where users internalize system-mediated successes as personal mastery. For example, when an LLM silently polishes a draft, the writer claims authorship over nuances they did not actually conceive. 2. On "healthy coexistence": In this paper, this is defined as "Seamful Agency". It is a state where the human can quantify the "D" (Discrepancy) between their raw intent and the system's output. Coexistence is "healthy" only when the locus of judgment remains visible at the moment of intervention. For a more rigorous definition of JTP and the underlying problem of "silent delegation," I highly recommend reading Chapter 1 of the white paper. Does this technical framing of "agency as a measurable gap" make more sense to you?
QuadmasterXLII 8mo ago
Hi, I think I saw you on slate star codex the other day!
- daikikadowaki 8mo ago
  Wow, good catch! I was just lurking in the shadows of that open thread. I didn't think anyone was actually reading my comments there. If you've been following my train of thought since then, this white paper is basically my attempt to formalize those chaotic ideas into a concrete metric. I’d love to know if you think this 'State Discrepancy' approach actually holds water compared to the usual high-level AI ethics talk.
amarcheschi 8mo ago
Is this paper written with heavy aid by ai? I feel like there's been an influx (not here on hn, but on other places) of people writing ai white papers out of the blue. /r/llmphysics has a lot of these
- daikikadowaki 8mo ago
  [flagged]
  - a-dub 8mo ago
    did you use ai to write this as well?
    - daikikadowaki 8mo ago
      To be consistent with my own principle: Yes, I am using AI to help structure these responses and refine the phrasing. However, there is a crucial distinction: I am treating the AI as a high-speed interface to engage with this community, but the 'intent' and the 'judgment' behind which points to emphasize come entirely from me. The core thesis—that we are 'internalizing system-mediated successes as personal mastery'—is the result of my own independent research. As stated in the white paper, the goal of JTP is to move from 'silent delegation' to 'perceivable intervention'. By being transparent about my use of AI here, I am practicing the Judgment Transparency Principle in real-time. I am not hiding the 'seams' of this conversation. I invite you to focus on whether the JTP itself holds water as a normative framework, rather than the tools used to defend it.
      - durch 8mo ago
        I am 100% in agreement, AI is a tool and it does not rob us of our core facilities , if anything it enhances them 100x if used "correctly", ie intentionally and with judgement. I will borrow your argument for JTP since it deals with exactly the kind of superficial objections I'm used to seeing everywhere these days, and that don't move the discussion in any meaningful way.
stuartjohnson12 8mo ago
https://www.lesswrong.com/posts/rarcxjGp47dcHftCP/your-llm-assisted-scientific-breakthrough-probably-isn-t https://www.lesswrong.com/posts/rarcxjGp47dcHftCP/your-llm-a... Hi author, this isn't personal, but I think your AI may be deceiving you into thinking you've made a breakthrough.
- daikikadowaki 8mo ago
  [flagged]
  - durch 8mo ago
    If you have a few minutes I invite you to check what we're doing over at Open Horizon Labs, its exactly the type of thinking we have around the current state of the world. Apologies I feel like I'm stalking you in the comments, but what you're saying absolutely resonates with what I've been thinking, and what I've been trying to build, and its refreshing to finally feel that I'm not insane. https://github.com/open-horizon-labs/superego https://github.com/open-horizon-labs/superego is probably the most useful tool we have, but I'm hoping that we can package it and bring it to the people, as it does make all these LLMs orders of magnitude more useful
    - daikikadowaki 8mo ago
      No apologies needed—I'm just glad to find I'm not the only 'insane' person here. It's easy to feel that way when obsessing over these problems, so knowing my ideas resonate with what you're building at superego is a huge relief. I’m diving into your repo now. Please keep me posted on your progress or any new thoughts—I'd love to hear them.
    - judahmeek 8mo ago
      > as it does make all these LLMs orders of magnitude more useful That seems like something that should be really easy to prove statistically.
      - 8mo ago
durch 8mo ago
This is exciting, I hope you manage to get traction for the idea! I currently have rely on a sort of supervisor LLM to check and detect if we're drifting, or overcomplicating or similar (https://github.com/open-horizon-labs/superego https://github.com/open-horizon-labs/superego). While I still to figure out who watches the watchers, they're are pretty reliable given the constrained mandate they have, and the base model actually (usually) pays attention to the feedback.
- daikikadowaki 8mo ago
  [flagged]
  - durch 8mo ago
    Thank you! I really hope we can make some headway here :)
    - daikikadowaki 8mo ago
      Thanks! I'm glad you feel the same. Unfortunately, the thread was just flagged, so I've messaged the mods to appeal it. I hope it gets restored so we can continue the debate. Let’s see what happens!
frizlab 8mo ago
> the risk of being rejected entirely I would have phrased it the hope of being rejected entirely, but to each his own I guess.
- daikikadowaki 8mo ago
  'Hope' might be a more honest word in an era of infinite noise. If my logic is just another hallucination, then I agree—it deserves to be rejected entirely. I have no interest in contributing to the 'AI-generated debris' either. But that’s exactly why I’m here. I’m betting that the 'State Discrepancy' metric and the JTP hold up under actual scrutiny. If you find they don't, then by all means, fulfill your 'hope' and tear the paper down. I'd rather be rejected for a flawed idea than ignored for a fake one."
daikikadowaki 8mo ago
[flagged]
- daikikadowaki 8mo ago
  [flagged]
  - deleted 8mo ago
    [deleted]
satisfice 8mo ago
“perceptibility of judgement” is not rigorously defined in these papers, as far as I can tell. The proposed JTP principle is suspended in midair, too. I can’t identify its ethical basis. Whatever perceptible judgement is supposed to mean, why should it always be transparent? Mechanical systems, such as a physical slot that mounts a sliding door, automatically cause alignment of the force that you use to open that sliding door. Is that “judgement” of the slot perceptible as it corrects my slightly misaligned push? Do I care? No. I would say that any tool we use responsibly requires that we have a reliable and rich model of the tool in our minds. If we do not have that then we cannot plan and predict what the tool will do. It has nothing to do with “judgements” that the tool makes. Tools don’t make judgements. Tools exhibit behavior.
- daikikadowaki 8mo ago
  [flagged]
  - satisfice 8mo ago
    1. It does not necessarily provide any feedback that you understand as such. You can THINK you are pushing of the door perfectly straight, while it is actually SILENTLY re-aligning the force along the plane of the slot. The only feedback you are getting is that the door is opening or not opening, you aren't getting feedback about the how the slot is "making judgements." 2. It seems to me you have arbitrarily applied the word "judgement" here rather than basing your usage on a rigorous operational definition. You don't define "raw intent", either, or what it means to "override" it. I suggest you read The Shape of Actions, if you haven't already. It is an example of how to make clean distinctions. Harry Collins distinguishes between mimeomorphic and polimorphic actions in a way that clarifies what can and cannot be automated. 3. You haven't defined "ontological deception" and the term does not make sense to me on its face. It's not a term of art that is in common use. Remaining sovereign simply has to do with having a operating mental model. I don't see what you mean by "whispering" or why I should care. That sounds more like poetry than technology. You should really try to explain more of your jargon if you care to be understood. Also, cite your sources.
e-dant 8mo ago
The issue this paper is grappling with (to what extent humans have a place in the middlespace between them asking an ai to do something, and the ai doing it) is interesting (although I disagree with how the paper tries to solve it). I’m empathetic to a non-native English speaker using ai to help communicate. I mean, seriously, I would do the same thing if the lingua franca was Japanese. The author is here saying, well yeah, there’s this weird thing that happens when I use ai by which the ideas that come out the other end are a bit different from what the author intended. I think the sentiment “well you shouldn’t have used ai” is incomplete. The paper is not great, but it’s an interesting question.

10 points by daikikadowaki 8mo ago

daikikadowaki 8mo ago

Hi HN, I recently submitted a white paper on State Discrepancy (D) to the EU AI Office (CNECT-AIOFFICE). This paper, "The Judgment Transparency Principle (JTP)," is my attempt to provide a mathematical foundation for the right to human autonomy in the age of black-box AI. Philosophy: Protecting the Future While Enabling Speed • Neutral Stance: I side with neither corporations nor regulators. I advocate for the healthy coexistence of technology and humanity. • Preventing Rupture: History shows that perceiving new tech as a “controllable threat” often triggers violent Luddite movements. If AI continues to erode human agency in a black box, society may eventually reject it entirely. This framework is meant to prevent that rupture. Logic of Speed: Brakes Are for Racing • A Formula 1 car reaches top speed because it has world-class brakes. Similarly, AI progress requires precise boundaries between “assistance” and “manipulation.” • State Discrepancy (D) provides a math-based Safe Harbor, letting developers push UX innovation confidently while building system integrity by design. The Call for Collective Intelligence: Why I Need Your Strength I have defined the formal logic of Algorithm V1. However, providing this theoretical foundation is where my current role concludes. The true battle lies in its realization. Translating this framework into high-dimensional, real-world systems is a monumental challenge—one that necessitates the specialized brilliance of the global engineering community. I am not stepping back out of uncertainty, but to open the floor. I have proposed V1 as a catalyst, but I am well aware that a single mind cannot anticipate every edge case of such a critical infrastructure. Now, I am calling for your expertise to stress-test it, tear it apart, and refine it right here. I want this thread to be the starting point for a living standard. If you see a flaw, point it out. If you see a better path, propose it. The practical brilliance that can translate this "what" into a robust, scalable "how" is essential to this mission. Whether it be refining the logic or engineering the reality, your strength is necessary to build a better future for AI. Let’s use this space to iterate on V1 until we build something that truly safeguards our collective future. Anticipating Pushback: • “Too complex?” If AI is safe, why hide its correction delta? • “Bad for UX?” A non-manipulative UX only benefits from exposing user intent. Calling it “too complex” admits a lack of control; calling it “bad for UX” admits reliance on hiding human-machine boundaries. If this framework serves as a mere stepping stone for you to create something superior—an algorithm that surpasses my own—it would be my greatest fulfillment. Beyond this point, the path necessitates the contribution of all of you. Let us define the path together.

daikikadowaki 8mo ago
For example, a critical engineering challenge lies in the high-dimensional mapping of 'Logical State'. While Algorithm 1 defines the logic, implementing CalculateDistance() for a modern LLM involves normalizing vectors from a massive latent space in real-time. Doing this without adding significant latency to the inference loop is a non-trivial optimization problem. I invite ideas on how to architect this 'Observer' layer efficiently.
kingkongjaffa 8mo ago
> If AI continues to erode human agency in a black box What do you mean by this? Is there evidence this has happened? > I advocate for the healthy coexistence of technology and humanity. This means whatever you want it to mean at any given time, I don't understand this point without further elaboration.
- daikikadowaki 8mo ago
  Thanks for the direct push. Let me ground those statements in the framework of the paper: 1. On "eroding human agency in a black box": I am referring to "Agency Misattribution". When Generative AI transitions from a passive tool to an active agent, it silently corrects and optimizes human input without explicit consent. The evidence is observable in the psychological shift where users internalize system-mediated successes as personal mastery. For example, when an LLM silently polishes a draft, the writer claims authorship over nuances they did not actually conceive. 2. On "healthy coexistence": In this paper, this is defined as "Seamful Agency". It is a state where the human can quantify the "D" (Discrepancy) between their raw intent and the system's output. Coexistence is "healthy" only when the locus of judgment remains visible at the moment of intervention. For a more rigorous definition of JTP and the underlying problem of "silent delegation," I highly recommend reading Chapter 1 of the white paper. Does this technical framing of "agency as a measurable gap" make more sense to you?

QuadmasterXLII 8mo ago

Hi, I think I saw you on slate star codex the other day!

daikikadowaki 8mo ago
Wow, good catch! I was just lurking in the shadows of that open thread. I didn't think anyone was actually reading my comments there. If you've been following my train of thought since then, this white paper is basically my attempt to formalize those chaotic ideas into a concrete metric. I’d love to know if you think this 'State Discrepancy' approach actually holds water compared to the usual high-level AI ethics talk.

amarcheschi 8mo ago

Is this paper written with heavy aid by ai? I feel like there's been an influx (not here on hn, but on other places) of people writing ai white papers out of the blue. /r/llmphysics has a lot of these

daikikadowaki 8mo ago
[flagged]
- a-dub 8mo ago
  did you use ai to write this as well?
  - daikikadowaki 8mo ago
    To be consistent with my own principle: Yes, I am using AI to help structure these responses and refine the phrasing. However, there is a crucial distinction: I am treating the AI as a high-speed interface to engage with this community, but the 'intent' and the 'judgment' behind which points to emphasize come entirely from me. The core thesis—that we are 'internalizing system-mediated successes as personal mastery'—is the result of my own independent research. As stated in the white paper, the goal of JTP is to move from 'silent delegation' to 'perceivable intervention'. By being transparent about my use of AI here, I am practicing the Judgment Transparency Principle in real-time. I am not hiding the 'seams' of this conversation. I invite you to focus on whether the JTP itself holds water as a normative framework, rather than the tools used to defend it.
    - durch 8mo ago
      I am 100% in agreement, AI is a tool and it does not rob us of our core facilities , if anything it enhances them 100x if used "correctly", ie intentionally and with judgement. I will borrow your argument for JTP since it deals with exactly the kind of superficial objections I'm used to seeing everywhere these days, and that don't move the discussion in any meaningful way.

stuartjohnson12 8mo ago

https://www.lesswrong.com/posts/rarcxjGp47dcHftCP/your-llm-assisted-scientific-breakthrough-probably-isn-t https://www.lesswrong.com/posts/rarcxjGp47dcHftCP/your-llm-a... Hi author, this isn't personal, but I think your AI may be deceiving you into thinking you've made a breakthrough.

daikikadowaki 8mo ago
[flagged]
- durch 8mo ago
  If you have a few minutes I invite you to check what we're doing over at Open Horizon Labs, its exactly the type of thinking we have around the current state of the world. Apologies I feel like I'm stalking you in the comments, but what you're saying absolutely resonates with what I've been thinking, and what I've been trying to build, and its refreshing to finally feel that I'm not insane. https://github.com/open-horizon-labs/superego https://github.com/open-horizon-labs/superego is probably the most useful tool we have, but I'm hoping that we can package it and bring it to the people, as it does make all these LLMs orders of magnitude more useful
  - daikikadowaki 8mo ago
    No apologies needed—I'm just glad to find I'm not the only 'insane' person here. It's easy to feel that way when obsessing over these problems, so knowing my ideas resonate with what you're building at superego is a huge relief. I’m diving into your repo now. Please keep me posted on your progress or any new thoughts—I'd love to hear them.
  - judahmeek 8mo ago
    > as it does make all these LLMs orders of magnitude more useful That seems like something that should be really easy to prove statistically.
    - 8mo ago

durch 8mo ago

This is exciting, I hope you manage to get traction for the idea! I currently have rely on a sort of supervisor LLM to check and detect if we're drifting, or overcomplicating or similar (https://github.com/open-horizon-labs/superego https://github.com/open-horizon-labs/superego). While I still to figure out who watches the watchers, they're are pretty reliable given the constrained mandate they have, and the base model actually (usually) pays attention to the feedback.

daikikadowaki 8mo ago
[flagged]
- durch 8mo ago
  Thank you! I really hope we can make some headway here :)
  - daikikadowaki 8mo ago
    Thanks! I'm glad you feel the same. Unfortunately, the thread was just flagged, so I've messaged the mods to appeal it. I hope it gets restored so we can continue the debate. Let’s see what happens!

frizlab 8mo ago

> the risk of being rejected entirely I would have phrased it the hope of being rejected entirely, but to each his own I guess.

daikikadowaki 8mo ago
'Hope' might be a more honest word in an era of infinite noise. If my logic is just another hallucination, then I agree—it deserves to be rejected entirely. I have no interest in contributing to the 'AI-generated debris' either. But that’s exactly why I’m here. I’m betting that the 'State Discrepancy' metric and the JTP hold up under actual scrutiny. If you find they don't, then by all means, fulfill your 'hope' and tear the paper down. I'd rather be rejected for a flawed idea than ignored for a fake one."

daikikadowaki 8mo ago

[flagged]

daikikadowaki 8mo ago
[flagged]
- deleted 8mo ago
  [deleted]

satisfice 8mo ago

“perceptibility of judgement” is not rigorously defined in these papers, as far as I can tell. The proposed JTP principle is suspended in midair, too. I can’t identify its ethical basis. Whatever perceptible judgement is supposed to mean, why should it always be transparent? Mechanical systems, such as a physical slot that mounts a sliding door, automatically cause alignment of the force that you use to open that sliding door. Is that “judgement” of the slot perceptible as it corrects my slightly misaligned push? Do I care? No. I would say that any tool we use responsibly requires that we have a reliable and rich model of the tool in our minds. If we do not have that then we cannot plan and predict what the tool will do. It has nothing to do with “judgements” that the tool makes. Tools don’t make judgements. Tools exhibit behavior.

daikikadowaki 8mo ago
[flagged]
- satisfice 8mo ago
  1. It does not necessarily provide any feedback that you understand as such. You can THINK you are pushing of the door perfectly straight, while it is actually SILENTLY re-aligning the force along the plane of the slot. The only feedback you are getting is that the door is opening or not opening, you aren't getting feedback about the how the slot is "making judgements." 2. It seems to me you have arbitrarily applied the word "judgement" here rather than basing your usage on a rigorous operational definition. You don't define "raw intent", either, or what it means to "override" it. I suggest you read The Shape of Actions, if you haven't already. It is an example of how to make clean distinctions. Harry Collins distinguishes between mimeomorphic and polimorphic actions in a way that clarifies what can and cannot be automated. 3. You haven't defined "ontological deception" and the term does not make sense to me on its face. It's not a term of art that is in common use. Remaining sovereign simply has to do with having a operating mental model. I don't see what you mean by "whispering" or why I should care. That sounds more like poetry than technology. You should really try to explain more of your jargon if you care to be understood. Also, cite your sources.

e-dant 8mo ago

The issue this paper is grappling with (to what extent humans have a place in the middlespace between them asking an ai to do something, and the ai doing it) is interesting (although I disagree with how the paper tries to solve it). I’m empathetic to a non-native English speaker using ai to help communicate. I mean, seriously, I would do the same thing if the lingua franca was Japanese. The author is here saying, well yeah, there’s this weird thing that happens when I use ai by which the ideas that come out the other end are a bit different from what the author intended. I think the sentiment “well you shouldn’t have used ai” is incomplete. The paper is not great, but it’s an interesting question.