6 ms·

So forgive my, "adversarial," questions here, but you seem to be encouraging adversarial thinking, which is laudable, but also invites adversarial thinking onto

So forgive my, "adversarial," questions here, but you seem to be encouraging adversarial thinking, which is laudable, but also invites adversarial thinking onto the construct you're building as well. Marqt, much like prediction markets, is not seeking truth, it's creating a map of a showcase of whoever bought into a particular market's emotions. It sounds like you're motivated by creating a repo of human feedback reinforcement data which could then be used to further train or fine-tune LLM's, perhaps even to sell, is that correct? > important for society to accurately define truth in an age when information travels (and changes) faster than ever, and when LLMs so confidently lie and have been heavily prompted to avoid controversial topics. Yeah, I don't think what you're building is getting at the goal of, "truth," it's getting at the goal of building something sellable per my question above. To the extent that someone will be willing to buy that human feedback database is a function of what they may be trying to accomplish. Much like in the world of so-called prediction markets, I see several of your markets as being heavily biased toward speculation and feelings, as opposed to a pragmatic approach to predicting a future outcome. What I mean by that is, you could create a really excellent scientific model and way of monitoring under what conditions a dam will break, or you could poll local people around a dam whether they feel like it will break, and you may get wildly different means and variances from the two methods. In all likelihood, there is no extra new synthesized knowledge about empirically what will physically happen with the dam that a group of average people off the street will have about that dam, in the context of a society with functioning infrastructure and tens of thousands of working dams. What you will have is a database on people's emotions about the dam, or perhaps dams in general, or the company behind the dam, etc. So what are you really doing to, "make the world a better place?" You're not, you're making it worse by selling a vision of, "truth," that is actually muddling certain applications of practical engineering or scientific predictions further. I think if you instead tweaked your pitch and just honestly stated that it's a repo of groups of people's feelings on various topics (and who those groups are, to the extent that you can figure that out), and not a way to, "beat LLM's with real truth," then it would be a lot more responsible. On the other hand, you could just be a troll trying to actively muddle the truth further, in which case you can just ignore my advice, that's fine too. > Inside the marqt maker is a fine-tuned BERT model that roughly ensures the statement is sensical, grammatical, and not a question. Can BERT conduct logical reasoning? So further to the above, it sounds like you're using an LLM to clean human feedback data, if that's the real crux of your plan, to make reinforcement data cheaper, and everything else is a sort of 4D chess, bravo, cool project!

arthurhur 3y ago
That was a very thoughtful comment. Let me attempt to unpack it. My first motivation is that I believe network effects can be used to represent empirical truths and track how they change over time. The more people vote on a statement, the more accurate the sentiment around that statement will be. You can check this comment out for more context: https://news.ycombinator.com/item?id=35819443 https://news.ycombinator.com/item?id=35819443 The presumed winners behind prediction and finance markets are the market makers that choose what markets they will make, and either charge a commission or collect the bid/ask spread. Anyone that is on the other side of that transaction opens themselves up to financial risk. In the marqt, anyone can be the marqt maker and there is no financial downside. It is more of a UGC model where the creator that builds engagement earns the points for all activity generated inside their marqt. However, there is no financial risk for participants in this system – you either vote true or false, and then try to write the best reason why you are correct (which you are both rewarded for also). You are only penalized if someone downvotes you as unremarqable (side note: you can only downvote a remarq that matches your marq, so if you marq a marqt false, then you can only downvote a remarq that is also marqed false, and those downvotes cancel if you change your marq to true). Keep in mind that "marqs earned" are just numbers that represent activity. I'm not sure how karma is calculated on HN, but it's a similar concept. What I am basically trying to do is think through how you might go about building and evolving something like Wikipedia using the pithiness of Twitter, as verified by Stack Overflow, while compensating creators like YouTube. Do I think this can be used to train LLMs? Of course you could train an LLM on such a dataset. But I believe information and facts change over time, the same way sentiments around companies and weather change over time, as represented by changing stock prices and temperature. So instead of a static database, it would be a living knowledge base that could be referenced at any point. For example, imagine a situation where you could ask, "What's the marqt say about who is responsible for the government shutdown right now?" and you could see, "Well, right now, the marqt says that 54% of people believe that Republicans are responsible." I'd rather open-source that for anyone to put out there instead of having to rely on some media outlet. Okay, so to directly answer your question about whether I'm trying to build something sellable, the answer is yes. But who gets paid in the marqt should be the actual people that create value – anyone that makes a marq, adds a remarq, makes remarqable/unremarqable votes, or makes a marqt – and I want to track the amount of value that users create in the system so that they can eventually monetize it. It's not controversial to imagine a world where AGI replaces human labor, and if that happens, I want to have built a system where humans can be compensated by AGI for maintaining its "intelligence." I have an idea for how it would work (which I can go into, if you want), but it would only be later down the line, if we can actually build up a resilient network of truth. I understand your reaction to the marqts I kicked things off with, but the idea is that anyone can create a marqt, and much the same way Wikipedia was limited and criticized when it first started, I believe an open-source knowledge base has exponential potential. There's a little more context in this response I posted: https://news.ycombinator.com/item?id=35815567 https://news.ycombinator.com/item?id=35815567 In response to your anecdote about the dam, I think about it the way efficient markets work. If all the information is out there, then trusted sources would asymmetrically influence everyone else's votes, unless they lose that trust. But, generally speaking, the more votes, the better. And regarding your thoughts about sentiment, I think that it plays a stronger role in the grand scheme of things. There was a world where people were paying astronomical sums of money for cheaply manufactured masks and alcohol. There was a world where a crypto shell company was worth billions of dollars. There was a world where under-water mortgage backed securities were rated AAA. What's most interesting to me is when that moment changes, because no one can predict the future. And eventually the future becomes the present. I could keep going, but maybe I'll stop here for now. Hopefully this answers most of your questions. Feel free to follow up with any other thoughts or get clarifications. But I really hope you try out the app and let me know what you think.

arthurhur 3y ago

That was a very thoughtful comment. Let me attempt to unpack it. My first motivation is that I believe network effects can be used to represent empirical truths and track how they change over time. The more people vote on a statement, the more accurate the sentiment around that statement will be. You can check this comment out for more context: https://news.ycombinator.com/item?id=35819443 https://news.ycombinator.com/item?id=35819443 The presumed winners behind prediction and finance markets are the market makers that choose what markets they will make, and either charge a commission or collect the bid/ask spread. Anyone that is on the other side of that transaction opens themselves up to financial risk. In the marqt, anyone can be the marqt maker and there is no financial downside. It is more of a UGC model where the creator that builds engagement earns the points for all activity generated inside their marqt. However, there is no financial risk for participants in this system – you either vote true or false, and then try to write the best reason why you are correct (which you are both rewarded for also). You are only penalized if someone downvotes you as unremarqable (side note: you can only downvote a remarq that matches your marq, so if you marq a marqt false, then you can only downvote a remarq that is also marqed false, and those downvotes cancel if you change your marq to true). Keep in mind that "marqs earned" are just numbers that represent activity. I'm not sure how karma is calculated on HN, but it's a similar concept. What I am basically trying to do is think through how you might go about building and evolving something like Wikipedia using the pithiness of Twitter, as verified by Stack Overflow, while compensating creators like YouTube. Do I think this can be used to train LLMs? Of course you could train an LLM on such a dataset. But I believe information and facts change over time, the same way sentiments around companies and weather change over time, as represented by changing stock prices and temperature. So instead of a static database, it would be a living knowledge base that could be referenced at any point. For example, imagine a situation where you could ask, "What's the marqt say about who is responsible for the government shutdown right now?" and you could see, "Well, right now, the marqt says that 54% of people believe that Republicans are responsible." I'd rather open-source that for anyone to put out there instead of having to rely on some media outlet. Okay, so to directly answer your question about whether I'm trying to build something sellable, the answer is yes. But who gets paid in the marqt should be the actual people that create value – anyone that makes a marq, adds a remarq, makes remarqable/unremarqable votes, or makes a marqt – and I want to track the amount of value that users create in the system so that they can eventually monetize it. It's not controversial to imagine a world where AGI replaces human labor, and if that happens, I want to have built a system where humans can be compensated by AGI for maintaining its "intelligence." I have an idea for how it would work (which I can go into, if you want), but it would only be later down the line, if we can actually build up a resilient network of truth. I understand your reaction to the marqts I kicked things off with, but the idea is that anyone can create a marqt, and much the same way Wikipedia was limited and criticized when it first started, I believe an open-source knowledge base has exponential potential. There's a little more context in this response I posted: https://news.ycombinator.com/item?id=35815567 https://news.ycombinator.com/item?id=35815567 In response to your anecdote about the dam, I think about it the way efficient markets work. If all the information is out there, then trusted sources would asymmetrically influence everyone else's votes, unless they lose that trust. But, generally speaking, the more votes, the better. And regarding your thoughts about sentiment, I think that it plays a stronger role in the grand scheme of things. There was a world where people were paying astronomical sums of money for cheaply manufactured masks and alcohol. There was a world where a crypto shell company was worth billions of dollars. There was a world where under-water mortgage backed securities were rated AAA. What's most interesting to me is when that moment changes, because no one can predict the future. And eventually the future becomes the present. I could keep going, but maybe I'll stop here for now. Hopefully this answers most of your questions. Feel free to follow up with any other thoughts or get clarifications. But I really hope you try out the app and let me know what you think.