I did this with mlc @ https://wiz.chat https://wiz.chat some time ago. Warning: it has a llama 3.1 7b model and is around 4 gb. It needs either a GPU or a Maca

I did this with mlc @ https://wiz.chat https://wiz.chat some time ago. Warning: it has a llama 3.1 7b model and is around 4 gb. It needs either a GPU or a Macand works only on chrome