7 ms·JM2C lood_in_4bit=True will let you run Llama2-7B variants at 6.3GB VRAM.by DarthNebo 3y agoJM2C lood_in_4bit=True will let you run Llama2-7B variants at 6.3GB VRAM.