7 ms·
> I myself shall certainly continue to leave such research to others, and to devote my time to developing concepts that are authentic and trustworthy. And I hop
by never_inline 3y ago
> I myself shall certainly continue to leave such research to others,
and to devote my time to developing concepts that are authentic
and trustworthy. And I hope you do the same.
Haha.
Maybe someone with GPT-4 access can check if it has improved. But I find it essentially regurgitating its sources.
I am by no way an expert in questions I ask it. But I find the answers generally speculative and not helpful.
I have been asking bard and chat gpt once in a while "What's the internal representation of strings in Java"?
This is a good question to test its information retrieval capabilities. because your average blogspam site or tutorial doesn't cover the this.
They both somehow fail to convey me that it's a combination of char array and a cached hashcode.
At best, chat GPT told me a 3 paragraph sentence that java String was an array of UTF-16 characters, which is not quite the case since JDK 9 as I know. There's no mention of compact strings.
Other day I asked what's the GCC pass which deduplicates similar function definitions. Both chat gpt and bard made up their own pass name.
I am no expert in these topics, nor in ML or IR. But I don't believe LLM is the way towards information retrieval.
If it's ingesting everything it reads, it would be worse than pagerank, right?
Granted, it does normie stuff pretty well, like writing a data class or making HTTP request. But as soon as you need something deep, it is worse than useless, because it confidently claims incorrect stuff.
- neom 3y agoI asked GPT4 your question, it answered: In Java, strings are internally represented as objects of the String class, stored in a character array (char[]). Each character in the string is represented as a Unicode character, using UTF-16 encoding. This allows Java strings to handle a wide range of international characters.
- penneyd 3y agoPerhaps we shouldn't expect these models to know everything about everything. What sources did you yourself use to learn this knowledge and did the training data incorporate them? It's a bit like asking a software engineer law questions, you can only draw from what you've studied. I feel as though what's missing is the ability for the model to understand what it doesn't know or cite sources. It's not like humans know everything either.
- luma 3y agoIt’s interesting to me how people approach an AI with simple knowledge retrieval requests. We’ve had search engines for a while and being able to search for facts isn’t a particularly interesting use case. It doesn’t take anything like intelligence to regurgitate existing facts.
- kaba0 3y agoBut that's the only thing they are good at, being smarter search engines (and that's why they should be backed by real search results, like Bing does it)
- MacsHeadroom 3y agoKnowledge retrieval (being a better search engine) is just about the worst thing LLMs are any good at, and by far the least useful or interesting.
- 0xBA5ED 3y agoSo what, by your estimation, are LLMs best for? Because they seem good for serving up relevant bits of information from vast amounts of information. Why do you think it's the worst thing they are good at?
- MacsHeadroom 3y agoBecause it's the most basic use. In a single prompt you can have the LLM serve up relevant bits covering multiple perspectives, contrast and compare the perspectives, analyze their effectiveness in a given problem domain, and then produce meaningful output towards a solution. Information retrieval is just step 1. Consider a prompt like the following: "Given the task: 'TASK GOES HERE', break it down into intermediate steps or 'thoughts'. Consider multiple different reasoning paths that could be taken to solve the task. Explore these paths individually, reflecting on the possible outcomes of each. Then, consider how you might backtrack or look ahead in each path to make global decisions. Based on this analysis, develop a final to do list and complete the first course of action."
- cloudking 3y agoThe difference between quality in GPT-3.5 to GPT-4 is game changing. It's hard to explain without you trying it for yourself.
- pulvinar 3y agoIndeed. 3.5 didn't recognize my reference to Bomb #20. But 4 did, and could describe the scene. And it even appreciated its relation to our discussion on epistemology when I said that I sometimes "hear" its voice when reading its replies!
- SamPatt 3y agoMy first question about any criticism of an AI response is "ChatGPT or GPT-4?" If it's ChatGPT I'll offer to run the prompt through GPT-4 for them. It's better 100% of the time, and it negates their complaint probably 90% of the time. I know it's not perfect, but GPT-4 is categorically different from ChatGPT.
- CSMastermind 3y agoI asked GPT-4 the sun over Japan question which GPT-3 got wrong, this was 4's response: The sun is directly overhead (at the zenith) when it's at a 90-degree angle relative to the observer. This phenomenon typically occurs in the tropics, between the Tropic of Cancer (23.5 degrees North) and the Tropic of Capricorn (23.5 degrees South). These are the maximum latitudes at which the sun can appear directly overhead. Japan is located between about 24 and 46 degrees north latitude. This means it is generally too far north for the sun to ever be directly overhead. On July 4th, the sun is still moving northward from the Tropic of Cancer (which it reaches at the summer solstice, around June 21st). As a result, even in the southernmost parts of Japan, the sun will not be directly overhead, though it will be relatively high in the sky. In more northern areas, the angle will be even more pronounced. If you want to know more precise details, such as the exact angle and time for a specific location in Japan on July 4, you'd need to consult a detailed almanac or a sun position calculator that takes into account the exact coordinates and date. --- It no longer hallucinates an incorrect answer, though I don't have the knowledge to know if it's correct in saying that there isn't one.
- ta1243 3y ago> On July 4th, the sun is still moving northward from the Tropic of Cancer Shouldn't that be "southward"
- Sharlin 3y agoIt seems to mash up the concepts of moving northward from the equator (which the sun does before June 21st) and moving southward from the Tropic of Cancer (which it would be doing on July 4th).
- NameError 3y agoIt's right that only places between the tropics will ever see the sun directly overhead. I asked it the same question and it was kinda subtly misleading: "The Sun is directly overhead, or at the zenith, at a given location only when that location is on the Tropic of Cancer (23.5 degrees north latitude) during the Summer Solstice (around June 21st each year) or on the Tropic of Capricorn (23.5 degrees south latitude) during the Winter Solstice (around December 21st each year)." There are other situations besides these - you don't have to be -on- one of the tropics for the sun to be overhead, you have to be -on or between- them.
- Kiro 3y agoPeople need to stop drawing conclusions based on GPT-3. If you're seriously interested in evaluating LLMs you should definitely try GPT-4. Most of my issues in 3 are non-existent in 4.
- never_inline 3y agoOnce I had tried Bing Chat, it had prompted to open in edge and I ignored. After your comment I went back and tried it again. It certainly seems to find some more details after I specify the prompt well. I admit it's a clear improvement. (Thank you for pointing out.) But it still gave some unsatisfactory answers in short interaction. I asked to write a comparator to sort in decreasing order, for which it returned verbose (anon. class) form of `b - a`, which can go wrong with large negative integers. Surprisingly, it did fairly well when asked to write a custom `JavaFileObject` class for `JavaDoc` toolprovider API, which is fairly obscure and I knew only because I worked with it. (There were errors in code, but they could be spotted by someone who can figure out the code's meaning). So my conclusion from short interaction, bing chat finds stack overflow pages more reliably than Google does (LoL), but still gets affected by crap of society like geeksforgeeks.
- nomel 3y agoBest results come with a conversation style prompt chain. When I have it write code, I always say "identify any corner cases that are not properly handled". If you see an obvious problem, you could tell it to correct it "b - a will cause overflow for large negative numbers. Use an alternative." I see it as an extremely broad, but maybe not incredibly deep, tool, that's most useful when you're not at the extreme specifics/esoteric knowledge. And, bing chat is quite a bit different than ChatGPT 4 (no search), which is quite a bit different than raw GPT 4 completions.
- dns_snek 3y agoAs a counter-anecdote, most of my issues with GPT 3/3.5 are just as present in GPT-4 in slightly milder form. Any code GPT4 produces needs to be checked with a fine tooth comb, any stated "facts" need to be double checked, any solutions to a given problem need to be examined by someone who's already familiar with the subject matter. I can't deny its usefulness as a rubber duck, though.
- ta1243 3y ago> Maybe someone with GPT-4 access can check if it has improved. But I find it essentially regurgitating its sources. My wife's a librarian in a teaching hospital. A recent senimar reported the same problem all over the country -- people coming in asking for papers which don't exist. Chat GPT is making up sources and quoting them. That's whatever version the free option at chat.openai.com, or whatever app someone downloads, uses. > Granted, it does normie stuff pretty well, like writing a data class or making HTTP request. I find it really handy to find useful libraries in an unfamiliar language. I needed to deal with some network addresses in python, it introduced me to the ipaddress module. I'm not a software developer, I certainly don't keep up with the latest developments (which I'd define as anything post matts-script-archive), so things like that are valuable.
- never_inline 3y ago> I find it really handy to find useful libraries in an unfamiliar language. I needed to deal with some network addresses in python, it introduced me to the ipaddress module. I'm not a software developer, I certainly don't keep up with the latest developments (which I'd define as anything post matts-script-archive), so things like that are valuable. That's correct. It has been many times helpful to me as well. But that's fundamentally because a google search for the same thing will be SEOd to death by blogspam sites listing top 100 python libraries.
- robryan 3y agoPython seems to suffer from this more than other languages. I guess because it is a popular beginner/ teaching language.
- pseudalopex 3y agoLibraries ChatGPT told me about were libraries I could have found faster with a search engine and libraries it made up. It can be handy if a library doesn't have examples though.
- sanderjd 3y ago
- zappchance 3y agoI asked your question to Claude but added a line specifying that I wanted more technical details. It actually included a line mentioning a cache but that's about it. Here is Claude's response: --- Strings in Java are implemented as objects of the String class. Under the hood, Java strings use the following internal representation: - A char array to store the characters of the string - An integer storing the length of the string - Some other internal fields for synchronization and cache (since Java 5) When a string is created in Java, a char array is allocated to store the characters of that string. The length integer is set to the actual length of the string. For example, if you create a string like this: ```java String s = "hello"; ``` Java will do the following: - Allocate a char array of size 5 to store the 5 characters 'h', 'e', 'l', 'l', 'o' - Set the length field to 5 - Initialize the char array with the string characters So the internal representation in this case would be: - A char[5] array initialized to {'h', 'e', 'l', 'l', 'o'} - length = 5 When string operations like concatenation are performed, a new char array is allocated to store the resultant string and the length is updated accordingly. The String class then manages this char array and length field. So in summary, Java strings use a char array and length field as their internal representation, and the String class encapsulates and manages this representation. The char array approach provides an efficient implementation for string operations in Java.
- fweimer 3y agoThat's fairly underwhelming? I don't think it's correct for any version of the JVM. Really old versions had offset/length fields, but never a length field on its own. (Back then, substring() didn't make full copies.) For mainline Java, since version 9, strings can use a byte[] to store characters if the string consists entirely of characters in the Latin-1 subset. String literal assignment does not allocate, and concatenation allocates a new String object and internal array. What is being described is closer to a typical implementation of C++ std::string, without the small-strings optimization. And it's written rather convincingly.
- never_inline 3y agoYeah right nothing about hashcode either.
- zappchance 3y ago> At best, chat GPT told me a 3 paragraph sentence that java String was an array of UTF-16 characters, which is not quite the case since JDK 9 as I know. There's no mention of compact strings. For what it's worth, I got both ChatGPT and Bing chat to include compact strings in their replies to my query. I think people need to be much more specific in their prompts instead of expecting the LLM to read their minds exactly as to how much detail should be provided.
- deleted 3y ago[deleted]
- alain94040 3y agoI'm quite disappointed in Knuth's dismissal. As the "king of algorithms", he of all people, should appreciate the leap. To put it another way, imagine if one of his students came to him with this question: I want to write an algorithm that given any of those 20 sample questions and others similar to them, gives this kind of answer. I couldn't find such algorithm in your books. What am I missing?
- minusf 3y agomr knuth has dedicated his life's work to mathematical correctness down to the typographic level of the books he wrote. last i heard he was not much into hallucinations... what he wrote i read as a huge praise for the field.
- kangalioo 3y agoPeople keep trying to use those LLMs as if it were a search engine but it's not. The value in ChatGPT is its faithful recreation of human common sense. Use it to hold a conversation, to ask feedback on a text you wrote, to come up with ideas for something. Don't use it as Google and be dismissive when it's not the universal information retrieval tool it's not meant to be
- bigbacaloa 3y ago"faithful recreation of human bullshitting" would be more accurate.
- Sharlin 3y agoRight. It's almost as if your average human has little creativity, indeed less than what ChatGPT can demonstrate, even though that magical "creativity" is supposed to be the current Big Thing that separates us from the machines. (In a year, once everybody is forced to agree that LLMs exhibit real creativity, not just regurgitation of their training data, the next Big Thing is going to be something else.)
- sanderjd 3y agoYeah it's really the back-and-forth nature of it that I find to be a big improvement over searching-and-clicking.
- teleforce 3y agoSooner or later most of the people will use it as a better replacement for Google search or Google on steroids. Before the advent of ChatGPT researchers especially, have been clamoring for better Google search with more contexts, intuitive and relevant feedbacks. With the new ChatGPT (Plus) features introduction for examples web online search and plug-ins, ChatGPT has becoming a very powerful and viable better alternative to Google search.
- nearbuy 3y agoChatGPT with GPT-4 seems to get this right: In Java, strings are internally represented by an instance of the `String` class, which is an immutable class that means once created, a `String` object cannot be changed. The `String` class stores character data as an array of `char` data type. Prior to Java 9, this array was encoded as UTF-16. This means that each character in the string is typically stored in 2 bytes (16 bits), which can represent a range of Unicode characters. Starting with Java 9, the `String` class uses a `byte` array, plus an encoding-flag field, to store string data. This change was part of the "compact strings" optimization, which is aimed at reducing the memory footprint of `String` objects. The flag tells the JVM whether the string is encoded as UTF-16 or Latin-1 (ISO-8859-1), depending on the string's content. If a string can be represented using Latin-1, it will be, because Latin-1 uses only 1 byte per character, which can save memory compared to the 2-byte UTF-16 encoding. It's worth noting that these implementation details are internal to the `String` class and aren't usually something you need to worry about when writing Java code. They could also potentially change in future versions of Java.