madisonmay

5 ms·

1.
by madisonmay 5mo ago
LLMs are not inherently non-deterministic during inference. I don't believe non-determinism implies lack of abstraction. Abstraction is simply hiding detail to manage complexity.
2.
by madisonmay 2y ago
pypdfium2 is a great choice and a solid piece of software! You might want to look into https://github.com/VikParuchuri/surya as an alternative to tesseract. Yes, it's associated with a commercial company, but as y
3.
by madisonmay 3y ago
This is an excellent use case for LLM fine-tuning, purely because of the ease of generating a massive dataset of input / output pairs from public C code
4.
by madisonmay 3y ago
It's more like saying "I've upgraded to 128GB of RAM, I'll never use my disk again".
5.
by madisonmay 3y ago
See figure-2
6.
by madisonmay 3y ago
Why the decision to license as GPL?
7.
by madisonmay 3y ago
Thanks, I hate it.
8.
by madisonmay 3y ago
Coding aid for unittests. Debugging aid for languages / frameworks I'm not particularly familiar with. Work that requires reformatting. Translating from rough drafts to more polished / professional language. Learning more a
9.
RLHF: Reinforcement Learning from Human Feedback(huyenchip.com)
4 points by madisonmay 3y ago|1 comments
10.
by madisonmay 3y ago
Whether or not to split is more a measure of whether or not these two concepts are likely to split down the road than whether or not share similarity today.
11.
by madisonmay 3y ago
Imperfect systems are still useful, and any sufficiently complex system is imperfect.
12.
by madisonmay 4y ago
Interestingly it sounds like offloading could be made quite efficient in a batch setting if you primarily care about throughput rather than latency. Though I guess for most current LLM applications latency is quite important.
13.
by madisonmay 4y ago
Often it might be viable to implement prediction w/o necessarily implementing training (especially if there are published weights or a reference implementation). Not viable for papers where the key contribution is a change to the pre-
14.
by madisonmay 4y ago
Guessing data security constraints -- I'm likely in a similar boat.
15.
by madisonmay 4y ago
I'm getting a variety of CORS errors in console. Maybe this helps: ``` Access to XMLHttpRequest at ' https://api.doqs.dev/v1/organization ' from origin ' https://app.doqs.dev ' has bee
16.
by madisonmay 4y ago
Awesome idea, but website seems unstable. Wasn't able to login after sign-up :/
17.
by madisonmay 4y ago
For more efficient einsum, see projects like https://optimized-einsum.readthedocs.io/en/stable/path_findi... .
18.
by madisonmay 5y ago
I suppose so, but perhaps trying to prevent the spaghettification has some positive benefits in terms of DRY + code structure.
19.
by madisonmay 5y ago
Wojciech stated this pretty explicitly on his Gradient Dissent podcast a few months back.
20.
by madisonmay 7y ago
Sweet! Looking forward to it.
21.
by madisonmay 7y ago
@nloui any chance you're willing to share your dataset? Would be fun to replicate this with GPT-2 fine-tuning instead of a Markov chain.
22.
Albert: A Lite Bert for Self-Supervised Learning of Language Representations(openreview.net)
1 points by madisonmay 7y ago|0 comments
23.
by madisonmay 7y ago
It's also possible it's part of a docker build step or similar. Even if they're aren't downloading models at run time they may be loading s3 if their pytorch-transformers lib docker cache gets invalidated frequently.
24.
Model Finetuning for Fun and Profit(indico.io)
1 points by madisonmay 8y ago|0 comments
25.
by madisonmay 8y ago
This is a rather painful read and a relatively shallow article on CNN's vs. RNNs. Potentially useful for someone new to the field but it's a very grab bag take. The reality is much less well defined than CNNs = eyes, RNNs = ears
26.
by madisonmay 8y ago
I also very much like https://www.figure-eight.com/data-for-everyone/ . It's not optimized for search but it's an excellent repository of high quality datasets.
27.
More Effective Transfer Learning for NLP(indico.io)
2 points by madisonmay 8y ago|0 comments
28.
by madisonmay 8y ago
Or as an alternative -- give them something legitimately beneficial to your company to work on, pay them standard consulting rates. This puts the impetus on the company to have fairly high certainty the candidate is qualified before the ta
29.
by madisonmay 8y ago
Well put. The NLP community seems to be slowly standardizing on language models trained on large unlabeled corpora as an analogue to pre-trained ImageNet models (ELMo, ULMFit, "Improving Language Understanding by Generative Pre-traini
30.
Finetune – Scikit-learn style model finetuning for NLP(github.com)
52 points by madisonmay 8y ago|8 comments

More ›