8 ms·
Show HN: AskHN
- MikeTheRocker 4y agoI love this! I used to append "reddit" to my Google search queries to get best results, but the quality of dialog over there has really dropped in recent years. These days I've switched to appending "hackernews", but this is even better.
- SilverBirch 4y agoThis might be a dumb question, but is this based on the collective wisdom of HN. Because I would say that the collective wisdom is just as much in the interaction of the comments and the ranking of those comments as it is in the comments themselves. If you just injest all the comments wholesale, aren't you rather getting the average wisdom of HN?
- inportb 4y agoI believe it's always going to be an average. The more interesting question is how is the average weighted?
- clark-kent 4y ago> 4. Index the embeddings in a database If Op is reading. I'm curious about the database you are using to store the embeddings. Pinecone, Weaviate ...?
- gk1 4y agoFrom the article: > The embeddings were then indexed with Pinecone.
- jaequery 4y agoSeeing a ton of projects utilizing chatgpt nowadays. Are the project owners basically paying the API costs out of pocket ? Think it would add up pretty quick especially if front page on HN.
- egypturnash 4y agoIs there a way to opt out of one's comments being used for this?
- andai 4y agoIf you're willing to pay for the retraining? ;)
- hombre_fatal 4y agoYes, don’t post on online forums.
- samstave 4y agoThats how I decided to opt-out of reddit after 16 years.
- jdthedisciple 4y agoWhy would you want to? Genuinely wondering. I for one am oh so proud that my valuable ramblings contributed to this majestic machinery.
- anaganisk 4y agoNah, it's no big a deal, its not like cambridge analytica will happen again. They're just using your data to train AI. Who knows may be based on the way you comment, you may get suggestions on which medication you need, or if it's time for the Redbull/starbucks coffee. Nah, all is good. Nothing bad will happen in allowing companies to scrape comments and build models. They're very ethical. In fact, people here are suddenly not so concerned that the model is not open. There is no oversight on how data is being used They are just proud to get answers from a text generator.
- olivierduval 4y agoThe BIG DEAL is not THAT specific instance but the fact that the ML crowd think it's OK to take everything without even asking permission
- LeoPanthera 4y agoI'm a little surprised that Hacker News comments weren't already in the GPT-3 training set. I just assumed that OpenAI had vacuumed up most of the web already.
- retube 4y agoI am guessing they already were? But this is 100% pure, concentrated HN not contaminated with nonsense from the rest of the web :)
- MuffinFlavored 4y agoIs it exclusively HN comments and nothing else? How does a model like that know how to speak English (noun/verb and all that) if you are starting from scratch and feeding it nothing but HN comments?
- neoromantique 4y agoI'm sorry to be THAT GUY, but it is addressed in the article :) >GPT embeddings To index these stories, I loaded up to 2000 tokens worth of comment text (ordered by score, max 2000 characters per comment) and the title of the article for each story and sent them to OpenAI's embedding endpoint, using the standard text-embedding-ada-002 model, this endpoint accepts bulk uploads and is fast but all 160k+ documents still took over two hours to create embeddings. Total cost for this part was around $70.
- nkozyra 4y ago> How does a model like that know how to speak English Mimicry.
- gorbypark 4y agoIn a nut shell, this is using openai’s api to generate embeddings for top comments on hn, then also generating an embedding for the search term. It then can find the closest related comments for the given question by comparing the embeddings and then send the actual text to GPT3 to summarize. It’s a pretty clever way to do it.
- MuffinFlavored 4y ago> I trained on a corpus of over 6.5 million Hacker News comments How long did it take to scrape them and train the "corpus" on this content?
- have_faith 4y agoIt seems to write in the generic "style" of GPT, instead of in the style I would recognise as a HN poster. Is that because of something baked into how the training process works? It lacks a sort of casualness or air of superiority ;)
- sebzim4500 4y agoThere was no training process, this is just running GPT with relevant HN comments as part of the prompt. If he wanted it to replicate that classic HN feel he would either have to extend the prompt with additional examples or, better yet, use finetuning. I guess he could also just randomly sprinkle in some terms like 'stochastic parrot' and find a way to shoehorn Tesla FSD into every conversation about AI.
- btbuildem 4y ago> “AskHN” is a GPT-3 bot I trained on a corpus of over 6.5 million Hacker News comments to represent the collective wisdom of the HN community in a single bot. First sentence of the first paragraph on OP's page EDIT: it's a bit misleading, further down they describe what looks like a semantic-search approach
- agolio 4y agoScroll a bit further down and you will see > 7. Put top matching content into a prompt and ask GPT-3 to summarize > 8. Return summary along with direct links to comments back to Discord user
- btbuildem 4y agoAh got it. Perhaps they should edit the intro then, it's misleading.
- stnmtn 4y agoI agree, that language could be very improved. This is not a GPT-like LLM whose training corpus is HN comments, which I found to be an extremely interesting idea. Instead, it looks like it's finds relevant HN threads and tells GPT-3 (the existing model) to summarize them. To be clear, I think this is still very cool, just misleading.
- pknerd 4y agoCan anyone help me to guide some tutorials using GPT-3 model on a certain dataset. I am a Python programmer.
- bilekas 4y ago"He only went and did it... " !
- georgelyon 4y agoAm I correct in understanding that this doesn't actually generate answers based on HN, but instead finds semantically-near comments and sends them verbatim to GPT to summarize? Seems like a good enough hack, though I'd love a detailed writeup of how to actually specialize an existing LLM with additional training data (like HN).
- serjester 4y agoAgreed, I think the better approach is to do some custom tuning but that becomes cost prohibitive very quickly. Not really much different than Algolia with a minor GPT-3 integration but neat project regardless.
- jerpint 4y agoThe summary itself is still generated, but has all the context to do summarization in the prompt. It's very difficult to otherwise finetune existing LLMs. GPT itself is closed-sourced, and doesn't allow for finetuning (except via an opaque API and with limited amounts of data). Other open models are either very difficult to load in memory and/or simply not as expressive as GPT
- ilaksh 4y agoTechnically it does give a specific answer to the question, but it is based on the semantically similar comments (and the question). The thing people don't realize is that right now there is a very large gap between the capabilities of a few models including OpenAI's most recent ones, and most of the other LLMs. So there are several options for actually training or fine-tuning with open models, but actually none of them have the language understanding and generation capabilities at the level of those new OpenAI models. As far as I know.
- redox99 4y agoYou can literally finetune these OpenAI models using their API. In this case it probably wasn't done because the author found it too much work and/or too expensive.
- moremetadata 4y ago
- monkeydust 4y agoNice work! Been playing with Langchain and was not aware of patterns.app. This whole space is moving so fast its hard to keep up for someone whos immediate day job doesn't revolve around this space. Congrats.
- jawadch93 4y ago[dead]
- helsontaveras18 4y agoNow that we have this bot to answer questions for us, I think we can all go home!
- la64710 4y agoIs there any LLM model that can be self hosted and fed a corpus of data to ingest for question answering? The part I find difficult is how to feed (not train) the open LLM models with entire dataset not available to public?
- bayan1234 4y agoThe hack to solve this is to embed each paragraph in your large corpus. Find paragraphs most similar to the user query using embeddings. Put the paragraphs and the raw user query into a prompt template. Send the final generated prompt to gpt3. This actually works surprisingly well. Check out the OpenAI cookbook for examples.
- motohagiography 4y agoNice. I just sort of assumed early on my comments were training some future AI, and I hope that in some small way I have been able to moderate some of its stupider urges. A version where you can turn knobs of flavored contributors would be pretty funny. I know my comment style is easily identifiable and reproducable, and it encodes a certain type of logical conjugation, albeit biased with some principles and trigger topics, and I think there is enough material on HN that there may be such a thing as a distinct, motohagiographic lens. :)
- PetrBrzyBrzek 4y agoHi, thanks for the interesting article. I have a question about Pinecone. What is the cost of storing all these vectors?
- btbuildem 4y agoMy own experiments made me think that the impact of finetuning is comparable to that of a molecule in a drop in a bucket. > “AskHN” is a GPT-3 bot I trained on a corpus of over 6.5 million Hacker News comments to represent the collective wisdom of the HN community in a single bot. I'm assuming you used the openai fine-tuning pathway to make a custom model? Have you tested the responses on vanilla GPT3 vs your custom model? I'd be curious to see the comparison.
- danuker 4y agoYeah, to me it looks like the learning rate was way too low to make a difference. I don't see any of the sublime and succinct snark.
- notahacker 4y agoYeah. Also full of GPT-3isms like "ultimately the choice ... comes down to the specific project and its ... requirements" and not nearly contrarian enough A bot focused on the output of HNers would insist on providing arguments against going through Google's interview process in the first place and suggestions that the correct answer to "Python or R" should be Haskell or Julia and would never suggest prioritising emotional vulnerability or being a happy person!
- danuker 4y agoThank you for the laffs =)
- clark-kent 4y agoFrom the article, they did not use fine-tuning. This is semantic search + GPT-3 to provide human-like answers.
- btbuildem 4y agoThanks! I missed that part. The semantic search approach seems to focus the answers better than fine-tuning; at the cost of preloading the prompt with a lot of tokens, but with the benefit of a more constrained response.
- tosemlep 4y agoDid you also ingest dead comments to the corpus? I would very much like to see the ghost of Terry pop up from time to time, to offer his wisdom and unique style of response.
- dalmo3 4y agoThis is nice! The official algolia search is useless. Otoh, did I miss something or is it only on discord?
- cactusplant7374 4y agoI really like Algolia. I usually use it to see if a particular link has been submitted. Other times I use it to find relevant comments or posts.
- AndrewKemendo 4y agoFirst thing I saw is my answers to someones question. Can you cut me a distro of myself?
- hbn 4y agoAmazing, an AI that is incapable of picking up on jokes or sarcasm!
- osigurdson 4y agoI have an experiment that uses the embeddings to visualize clusterings of HN comments (using tsne). Not super useful but interesting to view the comments in 3D and seeing how similar ones cluster together into mostly relevant themes.
- tomr75 4y agocould you do this for medical journal articles?
- all2 4y agoYou'd probably need to prepend a prompt that told the bot how to analyze experiment design. Maybe have it read a book or 10 on experiment design. Also a few books on social networks, financial motivations and other human factors in science. Then let it take a look at journal articles and their metadata. In short, you need a way to vet for quality.
- jerpint 4y agoNice! We built something very similar recently, it is more like "Ask your documentation" but very similar implementations otherwise. See a demo on the huggingface transformers documentation: https://huggingface.co/spaces/jerpint/buster https://huggingface.co/spaces/jerpint/buster code: https://github.com/jerpint/buster https://github.com/jerpint/buster
- freeqaz 4y agoStarred! We've been looking to build something similar so I appreciate you sharing this here. The only other project that I've seen that's doing something close to this is this one: https://github.com/getbuff/Buff https://github.com/getbuff/Buff It's a bit more similar to the OPs bot (it's a Discord bit). Cool to see momentum in this space!
- jn2clark 4y agoAs did we! It works very well. Article: https://github.com/marqo-ai/marqo/blob/mainline/examples/GPT-examples/article/article.md https://github.com/marqo-ai/marqo/blob/mainline/examples/GPT... and code:https://github.com/marqo-ai/marqo/tree/mainline/examples/GPT-examples https://github.com/marqo-ai/marqo/tree/mainline/examples/GPT...
- gnicholas 4y agoLove that it includes sources — this makes it much more valuable because you can tell if it's giving useful information or just blowing smoke.
- ada1981 4y agoHmm. I thought perhaps he was going to take the questions from askHN and the top upvoted comments and fine tuning a model with that as the prompt / reply pair. Curious how that would differ; but would be an expensive endevour.
- smsm42 4y agoHow to get a job at Google? Oh, that's easy, just get a PhD. Thanks bottie, very use, much helpful.
- boywitharupee 4y agoFor those who are wondering, HN data is indexed with embeddings for semantic search. When queried, it finds closest article, top comments and summarizes with GPT-3. GPT-3 serves as a rendering tool for compressed comments.
- DanielBMarkham 4y agoRelated question: I've written probably a million words over my lifetime. Is there an easy way to load up GPT with my thoughts to have it be a fake me?
- mittermayr 4y agoThis I think would be a great little SaaS idea to make some money. I keep seeing more and more people askingn how they can transform their data into an interactive archive that responds as chat, or with voice.
- ed_mercer 4y agoSort of. Look into gpt-index/Langchain
- einpoklum 4y ago> The methodology I used here is a generic, scalable solution for distilling a knowledge corpus into an embodied intelligence The methodology used here is a generic solution for distilling a non-generic corpus of utterances of , into a generic platitude machine.
- renewiltord 4y agoChatGPT and friends always talk like those Microsoft and Apple forum responders with 100k reputation. I see that you are asking about "How to get a job at Google". I will help you with "How to get a job at Google". In order to solve the problem of "How to get a job at Google" please follow the following steps first: - rewrite your resume in Google Docs - reinstall Chrome - apply to the job Let me know if I can help further with "How to get a job at Google". I like using it, but I have to tune my prompts to make sure that they don't bullshit me before getting to the point.
- blueicelake121 4y ago[flagged]
- gus_massa 4y agoIt looks interesting, but posting it on random threads of HN will make users flag your post and mods ban your account. The post definitively needs more info! Who are you? How do you pick the kids? Are you the "teacher", an "organizer" or just someone enthusiastic that is related to the project? Programing language? Age of the kids? Have you done something similar before? Length of the course? Why do you need money? Try to write a long post answering all that questions and perhaps a few more, but not too long. Make a new post, and then make a comment explaining you are the [teacher or whatever], and be ready reply to the comments in the thread. Some official suggestions in https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html
- deleted 4y ago[deleted]
- setgree 4y agoAs is often true of GPT responses, there's some nonsense interspersed here, e.g. the claim that R has "a more mature package universe" than Python. I think this is false, but if you're reading quickly, it sounds cogent enough. As Sarah Constantin observed about GPT2 [0]: > if you skim text, you miss obvious absurdities. The point is OpenAI HAS achieved the ability to pass the Turing test against humans on autopilot...The mental motion of “I didn’t really parse that paragraph, but sure, whatever, I’ll take the author’s word for it” is, in my introspective experience, absolutely identical to “I didn’t really parse that paragraph because it was bot-generated and didn’t make any sense so I couldn’t possibly have parsed it”, except that in the first case, I assume that the error lies with me rather than the text. This is not a safe assumption in a post-GPT2 world. Instead of “default to humility” (assume that when you don’t understand a passage, the passage is true and you’re just missing something) the ideal mental action in a world full of bots is “default to null” (if you don’t understand a passage, assume you’re in the same epistemic state as if you’d never read it at all.) [0] https://www.skynettoday.com/editorials/humans-not-concentrating https://www.skynettoday.com/editorials/humans-not-concentrat...
- leereeves 4y ago> there's some nonsense interspersed here, e.g. the claim that R has "a more mature package universe" than Python. As a programmer, I find R hard to use and not very well designed, so I can see why you'd call that nonsense. But when I was a math student, I found that in some ways R does have "a more mature package universe". There were many math algorithms that I could find packages for in R and not in Python, even as a mere grad student.
- setgree 4y agoAbsolutely, for statistics and visualization I think R and its packages are (sometimes) superior. But GPT responses don't generally offer those kinds of nuances; the claim is that the packages are "more mature," period. And it's for good reason that the _most_ mature Python packages, e.g. numpy and pandas, are used by data scientists in production pretty much everywhere.
- deleted 4y ago[deleted]
- flappysolutions 4y agoI'm quite pleasantly surprised by this response: Question: what are the biggest fallacies of people that post on hackernews? Answer: One of the biggest fallacies of people that post on Hacker News is the nirvana fallacy, which is when someone jumps from "thing x is imperfect" to "thing x is worthless" without considering any of the benefits. People also tend to be familiar with certain technologies and will discount the flaws of existing technology vs. the flaws of new technology. Another fallacy is the assumption that there is one best tool for doing things - the one that supports the largest amount of use cases. Additionally, people may use appeals to authority, invent their own solutions rather than use existing ones, and assume that the approaches they first learned are most correct. Finally, people may assume open source software is a gift and that dependencies don't need to be tested.
- ducktective 4y agoIt kinda enumerated all possible characteristics of people and their approaches to technical issues, not something unique to HN.
- autoexec 4y ago> People also tend to be familiar with certain technologies and will discount the flaws of existing technology vs. the flaws of new technology. Is that really a problem? It's only natural to get accustomed to old problems caused by old tech (where workarounds probably already exist or limitations understood and accepted) while it makes sense to hold new tech to a higher standard since it has the benefit of learning from past mistakes and should avoid introducing obvious problems for people in the future to deal with. Our future selves will thank us for being more critical of flawed new tech than we are of old tech.
- m3kw9 4y agoI thought chatgpt may already used hacker news (Reddit) to train?
- adversaryIdiot 4y agoI Didn't know the api supported downloading all of its database. Are you the reason HN has sporadic downtime lately? ;)
- leobg 4y agoI like the project. Had been wanting to do this myself for a long time, because HN has become the first place I go to nowadays for answers, and I value the intelligence and experience distilled in the comments here. I do not like that it seems to be effectively an ad. > Embedding every single one of the 6.5 eligible comments was prohibitively time-consuming and expensive (12 hours and ~$2,000). Does anybody understand what he’s talking about here? Assuming 6.5 million comments and an average token length of 70 we'd be looking at $180 ($0.0004 / 1K tokens).
- fleddr 4y agoLet's admit that HN's culture is that many of us are confidentially wrong, which we cover up with impressive technical jargon. As such, any wrong answer in this AI is in fact correct.
- kerogerokero 4y agoAnyone here know how to generate subtitles automatically using AI when a video is playing on the web? Was planning to see how I can build something like these sites but without the need to regularly update the subtitles catalog: https://subscene.be https://subscene.be https://subtitlecat.com https://subtitlecat.com https://subtitletrans.com https://subtitletrans.com https://my-subs.co https://my-subs.co
- hummus_bae 4y ago[dead]
- underlines 4y agoJust to be sure: This is NOT a finetuned GTP model, but rather standard GPT-3 API, used to summarize search results of a HN Comments DB, based on user input. Right?
- _8j50 4y agoSome day I will sue people like OP (if they're monetizing it) and OpenAI for monetizing my public posts. You can use, reuse and alter public speech but when you earn ad dollars...yeah part of that is mine if your model used my public content. I probably won't actually sue but someone will. I am not a lawyer but there has to be a jurisdiction where I can establish standing at least.
- sorokod 4y agothe collective wisdom of the HN community Made me smile
- Existenceblinks 4y agoAsk HN: here is my idea, can I build this in a weekend AI: of course .. here is your bash script (220 lines long)
- wastu 4y agoInstead of being summarize tool, this bot is really useful if you want to search related hn post based on abstract thinking imo.