simonhughes22

11 ms·

1.
Generating Rules and Systems for Roguelikes (w/o AI) [video](youtube.com)
1 points by simonhughes22 3y ago|0 comments
2.
by simonhughes22 3y ago
This is like calling the nuclear anti-proliferationists useless when they had just gotten started. AGI has only been in the general public's consciousness for about a year since ChatGPT released (before then far fewer people were worri
3.
by simonhughes22 3y ago
Llama 2 (and variants). Has the lowest hallucination rate ( https://github.com/vectara/hallucination-leaderboard ), and its open source and so we know what went into it, and the community can improve it
4.
by simonhughes22 3y ago
Yeah it's odd they chose Palm 2 to compare against. Not a very strong model by most measurements.
5.
by simonhughes22 3y ago
Like a number of other LLMs we tested, including the Palm 2 chat model (chat-bison-001), it adds in the street value, and assumes the plants are cannabis (which is reasonable but is an assumption not mentioned in the article).
6.
by simonhughes22 3y ago
Prompt: You are a chat bot answering questions using data. You must stick to the answers provided solely by the text in the passage provided. You are asked the question 'Provide a concise summary of the following passage, covering the
7.
by simonhughes22 3y ago
The model is bad at hallucinating despite their claims. See the first prompt i tried here: https://twitter.com/hughes_meister/status/172740068973816258...
8.
by simonhughes22 3y ago
This is just typical of so much work in the field. They pick and choose which models to compare against and on which benchmarks. If this model was truly great, they would be comparing against Claude 2 and GPT4 across a bunch of different be
9.
by simonhughes22 3y ago
You can view the responses here in the linked csv file: https://github.com/vectara/hallucination-leaderboard
10.
by simonhughes22 3y ago
The original data we used was not annotated with sources, only where the overall data came from. Most was news articles. The length doesn't seem to matter too much as we see a lot of errors even when summarizing a single sentence (some
11.
by simonhughes22 3y ago
We may write a research paper at some point. For now, see here: https://vectara.com/cut-the-bull-detecting-hallucinations-in... Given the number of models involved, we have over 9k rows currently. Judging for this task is q
12.
by simonhughes22 3y ago
Yes. Just because the model is smaller doesn't always mean by default it's worse, as they may be trained for less time or on less data, which in some cases could be beneficial. The differences are small so may not be statistically
13.
by simonhughes22 3y ago
Yes thanks for fixing that.
14.
by simonhughes22 3y ago
I worked on the model with our research team. Recently featured in this NYT ( https://www.nytimes.com/2023/11/06/technology/chatbots-hallu... . Post here to AMA. We are also looking for collaborators to he
15.
by simonhughes22 3y ago
That's the term used by the academic literature also, so Hallucinate is an industry standard term.
16.
Show HN: How to Evaluate Different AI Search Models(opensourceconnections.com)
15 points by simonhughes22 3y ago|0 comments
17.
by simonhughes22 3y ago
It's not that simple. Originally OpenAI released a model to try and detect whether some content was generated by an LLM or not. They later dropped the service as it wasn't accurate. Today's models are so good at text generati
18.
by simonhughes22 3y ago
Wondering how many people are now downloading this and other libs like Dart and trying to do stock market prediction or crypto price forecasting. Most of the devs i know, myself included, have dabbled in coding up trading algorithms at some
19.
by simonhughes22 4y ago
This is why i moved to data science so i can focus more on solving problems than picking frameworks and libraries. We are not completely immune to this problem, but by and large the tooling ecosystem is much smaller and the focus is on prob
20.
by simonhughes22 5y ago
Thanks that is definitely broken. Logging the issue right now.
21.
by simonhughes22 5y ago
I'd suggest using the app over the mobile website. If you are in store, it will tell you where in the store the items are (Bay and Aisle) which is super useful.
22.
by simonhughes22 5y ago
It does semantic matching. We don't have a lot of exact matches for that search for birch wood (given the exact dimensions) so the engine broadens the search criteria automatically. You never want to just show exact matches, so long as
23.
by simonhughes22 5y ago
I worked on the system. It's a similar idea but it's on e-commerce products and not websites. So you can't use things like page rank when doing product search.
24.
by simonhughes22 5y ago
The results here look fine to me in positions 3 and 4 - https://www.homedepot.com/s/2'x4'%2520piece%2520of%2520birch... What are you expecting to see? I worked on this search engine so interested to get feedb
25.
by simonhughes22 5y ago
I agree.
26.
by simonhughes22 6y ago
It would be great if (in the repo) you could briefly explain what fuzzing in and why you'd need it. I assume it's some sort of obfuscation tool?
27.
by simonhughes22 6y ago
Similar - my grandfather is my only grand parent with high cholesterol, he's now 93 and has out lived all my other grand parents and his sister. It's clear to me that there's a lot here that we still don't understand.
28.
by simonhughes22 6y ago
My thoughts exactly. A DL for 100 row dataset seems like overkill (and likely overfits).
29.
by simonhughes22 6y ago
This is really good. It states a lot of things that it has taken me time and experience to learn, which I would have benefited from knowing early in my career. Authority is not always right, people in power have ulterior motives other than
30.
by simonhughes22 6y ago
Short answer - no. Where is the explanation for how the predictive queries work. Is it some sort of bayesian model? It's not too hard to quickly fit some NB or regression model on some dataset on the fly given the simplicity of those m

More ›