michaelhartm

7 ms·

1. 1.▲
   by michaelhartm 3y ago

   Btw, semantics and syntax is separated in the LLMs (the author is wrong). The embedding function (matmul) can map syntax and the proximity in the embedding (e.g. cosine similarity) is the semantics (that's attention). So not convinced.
2. 2.▲
   by michaelhartm 3y ago

   Nobody knows how the LLMs work under the hood. It's just lots of stacked transformers that encode various concepts. Nothing in this book refutes whether Chomsky's concepts are actually being encoded in LLMs or not. For all we know
3. 3.▲
   by michaelhartm 3y ago

   I guess Databricks is now going after OpenAI?
4. 4.▲
   by michaelhartm 3y ago

   Btw, it's kinda crazy how bad the GPT4-J results in the blog are compared to the Dolly one, which seem pretty good. Do we know why it works so well to use this 50k dataset?
5. 5.▲
   by michaelhartm 3y ago

   They used the 6b GPT4-J, not 20B. That's what's interesting, it's a smallish large language model :).
6. 6.▲
   by michaelhartm 5y ago

   Data Wars: Snowflake vs Databricks (0 - 2)?
7. 7.▲
   by michaelhartm 5y ago

   * Databricks is unethical * Nobody should benchmark anymore, just focus on customers instead * But hey, we just did some benchmarks and we look better than what Databricks claims * Btw, please sign up and do some benchmarks on Snowflake, we