7 ms·

It's amazing how much misinformation and vague information there is on this topic. I tried getting to the bottom of this in the following post in the OpenAI fo

It's amazing how much misinformation and vague information there is on this topic. I tried getting to the bottom of this in the following post in the OpenAI forum: https://community.openai.com/t/fine-tuning-myths-openai-documentation/ https://community.openai.com/t/fine-tuning-myths-openai-docu... Bottom line is that fine-tuning does not seem to be a feasible option for adding new knowledge to a model for question answering.

bitL 3y ago
My research shows otherwise. Tuning via transformer adapters pretty much added new knowledge to QA models or could be used for adversarial QA training. You can throw away learned adapters anytime and retrain from scratch with new information if your adapters become stale. Fine-tuning is cheap and small (e.g. 60kB data in an adapter). You can customize it in production for each individual customer as well by swapping adapters at the time of inference. Embeddings for very short-term facts and adapters for medium-long-term info seems like the best combination.
- pumanoir 3y ago
  You mean like what's described in this blog post, correct? https://adapterhub.ml/blog/2022/03/adapter-transformers-v3-unifying-efficient-fine-tuning/ https://adapterhub.ml/blog/2022/03/adapter-transformers-v3-u...
  - bitL 3y ago
    Yes, those adapters. There are many types now, most recently LLaMA adapter: https://arxiv.org/abs/2303.16199 https://arxiv.org/abs/2303.16199
- crosen99 3y ago
  Could you link to your research and/or describe the models, libraries, data and tests you used for this?
- tmaly 3y ago
  Have you tried fine-tuning via adapter, if so what has been your experience and was the the total cost?
saurik 3y ago
I liked this video: https://youtu.be/9qq6HTr7Ocw https://youtu.be/9qq6HTr7Ocw "OpenAI Q&A: Finetuning GPT-3 vs. Semantic Search - which to use, when, and why?"
- osigurdson 3y ago
  I can't determine if this person is knows what they are talking about or is an extreme amateur just making things up and speaking confidently. He has other videos that are complete nonsense.
  - saurik 3y ago
    I mean... I you aren't OK with that, how are you going to ever use a large language model? ;P

bitL 3y ago

My research shows otherwise. Tuning via transformer adapters pretty much added new knowledge to QA models or could be used for adversarial QA training. You can throw away learned adapters anytime and retrain from scratch with new information if your adapters become stale. Fine-tuning is cheap and small (e.g. 60kB data in an adapter). You can customize it in production for each individual customer as well by swapping adapters at the time of inference. Embeddings for very short-term facts and adapters for medium-long-term info seems like the best combination.

pumanoir 3y ago
You mean like what's described in this blog post, correct? https://adapterhub.ml/blog/2022/03/adapter-transformers-v3-unifying-efficient-fine-tuning/ https://adapterhub.ml/blog/2022/03/adapter-transformers-v3-u...
- bitL 3y ago
  Yes, those adapters. There are many types now, most recently LLaMA adapter: https://arxiv.org/abs/2303.16199 https://arxiv.org/abs/2303.16199
crosen99 3y ago
Could you link to your research and/or describe the models, libraries, data and tests you used for this?
tmaly 3y ago
Have you tried fine-tuning via adapter, if so what has been your experience and was the the total cost?

saurik 3y ago

I liked this video: https://youtu.be/9qq6HTr7Ocw https://youtu.be/9qq6HTr7Ocw "OpenAI Q&A: Finetuning GPT-3 vs. Semantic Search - which to use, when, and why?"

osigurdson 3y ago
I can't determine if this person is knows what they are talking about or is an extreme amateur just making things up and speaking confidently. He has other videos that are complete nonsense.
- saurik 3y ago
  I mean... I you aren't OK with that, how are you going to ever use a large language model? ;P