Having done this for domain specific engineering paperwork that looks similar to cause analysis, it does work well at param sizes << 70B. lora does not work th

Having done this for domain specific engineering paperwork that looks similar to cause analysis, it does work well at param sizes << 70B. lora does not work though, you need full parameter training if the knowledge isnt already present in pre training set.