5 ms·

LLama.cpp now has a web interface

328 points by xal 3y ago

mliker 3y ago
very cool to see the CEO of Shopify putting together this interface.
steren 3y ago
And the PR is from Shopify's CEO!
- vdfs 3y ago
  OP is Tobi
wiihack 3y ago
Nice simple interface! And great to see a CEO keeping up with recent technological developments :)
vdfs 3y ago
Good to see a CEO is still hacking around! Shopify did change my life in a way i could never imaging, it will always hold a special place in my heart
- samstave 3y ago
  Can you give a detailed account on specifics?
  - deleted 3y ago
    [deleted]
- Xenoamorphous 3y ago
  A billionare co-founder at that!
  - swyx 3y ago
    actually did tobi even have any other cofounders? i just realized i only ever hear about him and harvey finkelstein edit: google says scott lake was actually the founding CEO. TIL! https://www.linkedin.com/mwlite/profile/in/scottlake?originalSubdomain=ca https://www.linkedin.com/mwlite/profile/in/scottlake?origina...
    - Xenoamorphous 3y ago
      Just going by the Google snippet I got when searching for “shopify ceo”. Not sure if the downvotes I got are because somone thinks that the co-founding aspect is wrong or because they think that him being a billionaire is irrelevant, but IMO it makes the fact he’s opening PRs even cooler.
    - jonny_eh 3y ago
      > harvey finkelstein Harley Finkelstein, he's a nice guy. Not a co-founder but very important to the history and running of Shopify.
    - bhouston 3y ago
      There was 3 cofounders. Scott Lake left relatively early but Daniel Weinand (designer) stayed on for a while.
mteam88 3y ago
Are these comments about the CEO bots?
- isanjay 3y ago
  Dedo something fishy. But accounts are created ages ago
  - Kiro 3y ago
    Not really. I also think it's remarkable and would have posted something similar if it hadn't already been pointed out.
- Cyph0n 3y ago
  People are just trying to point out who the author is for those not familiar with him.
- mrtranscendence 3y ago
  Were you aware that the author is Shopify's CEO? How deliciously absurd!
- generalizations 3y ago
  I think this has been seen before, where it turned out to be a bunch of employees. But that time it was a product launch, not a CEO's pr.
- mliker 3y ago
  I think it's just impressive to folks that the CEO of a public company is still coding and putting out useful PRs.
creekcreek 3y ago
If only all tech CEOs were as code-savvy as the magnificient CEO of Shopify! What a amazing accomplishment from a CEO who cares about development.
zoklet-enjoyer 3y ago
Wow. Every comment pointing out the author but not the content
mk_stjames 3y ago
I'm always wondering about the, I don't even know what to call this, etiquette? of proposing PR's to projects like these that add a feature or a demo or whatnot to the main branch of a very focused project by adding something that is very different in interface, language, set and setting etc. So in this case, Tobi made this awesome little web interface that uses minimal HTML and JS as to stay in line with llama.cpp's stripped-down-ness. But it is still a completely different mode of operation, it's a 'new venue' essentially. What if GG didn't want such a thing? When is something like this better for a separately maintained repo and not a main merge? How do you know when it is OK to submit a PR to add something like this without overstepping (or is it always?) I see this with a few projects on github that really 'blow up' and everyone starts working on. They get a million PR's from people hacking things on it in their domain of knowledge, expanding the complexity (and potentially difficulty to maintain quality). Sometimes it gets weird feeling watching from the outside at least (I'm not a maintainer on any public FOSS). Just curious what others think because those are my thoughts that came to mind when I saw this.
- xal 3y ago
  GG would just say no and that's that. No hard feelings, that's what makes open source so great.
- bfuller 3y ago
  My view is you get so many people blatantly ripping off your code to try and pass it off as their own that you actually appreciate when you come across someone doing something novel or interesting that users like. That was my view anyway.
- LawnGnome 3y ago
  I generally think it's fine to do this sort of thing for your own benefit and open a PR as long as you're really 100% fine with "no, I'm not interested in merging this" being the answer. Where the problems tend to arise (in my experience, at least) is when people hack on something expecting that it will be merged, get invested in it, and then get upset when the maintainer(s) aren't interested. Checking in before starting to work on something is important if your goal is to have it merged, not just to do the work. The problem is that a lot of people start in the first category, but then move into the second category as they get invested in their project.
isoprophlex 3y ago
> I tried to match the spirit of llama.cpp and used minimalistic js dependencies and went with the ozempic css style of ggml.ai. Ozempic? The anti-diabetes drug? That's either a glorious typo or an interesting new adjective...
- jahewson 3y ago
  I did a double-take when I saw that too. It’s now prescribed as a weight-loss drug and is very much in the zeitgeist so yeah I think it’s a new adjective. Personally I’m going to stick with “light weight”.
  - isoprophlex 3y ago
    The more I think about it, the more I'm loving it
- xal 3y ago
  Nat coined this usage https://twitter.com/natfriedman/status/1668656170645749761 https://twitter.com/natfriedman/status/1668656170645749761
  - slaymaker1907 3y ago
    Lol, I’m totally using this to describe memory optimization from now on. There’s a lot of software out there that needs to be more ozempic.
ynniv 3y ago
i'm importing from js cdns instead of adding them here FWIW this seems counter to llama.cpp's philosophy.
- mhh__ 3y ago
  The philosophy in practice seems to be dirty deeds done dirt cheap rather than genuine simplicity (it only looks simple compared to python ecosystem)
- xal 3y ago
  I agree, I ended up getting rid of it. Only one dependency is downloaded (via bash script) and everything is baked into the binary now.
eclectic29 3y ago
Can someone shed light on how does a CEO with 3 kids get time to hack on something like this? Some might even argue that all the time spent doing this would’ve been better spent on CEO activities, but thankfully this is HN and people have hobbies, so that’s that, but this can be a very time consuming hobby.
- xal 3y ago
  I've tried to eliminate language around being "too busy" from my vocabulary and attempt to replace it with "can't prioritize". That's sometimes a bit awkward, but really trains better habits. Sometimes I prioritize hobbies if I feel I need it, and doing this seemed fun and useful besides!
  - eclectic29 3y ago
    Thanks for your helpful response.
    - vdfs 3y ago
      He have this in his Twitter bio: "CEO by day, Dad in evening, hacker at night"
deleted 3y ago
[deleted]
seydor 3y ago
didnt people run llamacpp with oobabooga et al?
franchiser 3y ago
[dead]
sebastiennight 3y ago
I'm wondering whether the rendering of markdown through a regex that only deals with H3, strong, em and code is enough, since this implementation specifically seems to ask the bot to reply with markdown?[0] Other than that, it's cool to see this web interface happen. It's little things like this that make new tools easier to grasp for less technical users. I believe that people are more likely to go through the hurdles of learning difficult setup procedures (knowing the UX/UI will be easy to use) versus making the setup easier but having the daily use be too hard. [0]: https://github.com/ggerganov/llama.cpp/pull/1998/commits/c19daa4eb58cca85c0263d77958370e201bf9551 https://github.com/ggerganov/llama.cpp/pull/1998/commits/c19...
- xal 3y ago
  I had an entire implementation that used a proper state machine but I couldn’t get it simple and small enough for this single file thing. The regexp chain is dumb but clear.

328 points by xal 3y ago

mliker 3y ago

very cool to see the CEO of Shopify putting together this interface.

steren 3y ago

And the PR is from Shopify's CEO!

vdfs 3y ago
OP is Tobi

wiihack 3y ago

Nice simple interface! And great to see a CEO keeping up with recent technological developments :)

vdfs 3y ago

Good to see a CEO is still hacking around! Shopify did change my life in a way i could never imaging, it will always hold a special place in my heart

samstave 3y ago
Can you give a detailed account on specifics?
- deleted 3y ago
  [deleted]
Xenoamorphous 3y ago
A billionare co-founder at that!
- swyx 3y ago
  actually did tobi even have any other cofounders? i just realized i only ever hear about him and harvey finkelstein edit: google says scott lake was actually the founding CEO. TIL! https://www.linkedin.com/mwlite/profile/in/scottlake?originalSubdomain=ca https://www.linkedin.com/mwlite/profile/in/scottlake?origina...
  - Xenoamorphous 3y ago
    Just going by the Google snippet I got when searching for “shopify ceo”. Not sure if the downvotes I got are because somone thinks that the co-founding aspect is wrong or because they think that him being a billionaire is irrelevant, but IMO it makes the fact he’s opening PRs even cooler.
  - jonny_eh 3y ago
    > harvey finkelstein Harley Finkelstein, he's a nice guy. Not a co-founder but very important to the history and running of Shopify.
  - bhouston 3y ago
    There was 3 cofounders. Scott Lake left relatively early but Daniel Weinand (designer) stayed on for a while.

mteam88 3y ago

Are these comments about the CEO bots?

isanjay 3y ago
Dedo something fishy. But accounts are created ages ago
- Kiro 3y ago
  Not really. I also think it's remarkable and would have posted something similar if it hadn't already been pointed out.
Cyph0n 3y ago
People are just trying to point out who the author is for those not familiar with him.
mrtranscendence 3y ago
Were you aware that the author is Shopify's CEO? How deliciously absurd!
generalizations 3y ago
I think this has been seen before, where it turned out to be a bunch of employees. But that time it was a product launch, not a CEO's pr.
mliker 3y ago
I think it's just impressive to folks that the CEO of a public company is still coding and putting out useful PRs.

creekcreek 3y ago

If only all tech CEOs were as code-savvy as the magnificient CEO of Shopify! What a amazing accomplishment from a CEO who cares about development.

zoklet-enjoyer 3y ago

Wow. Every comment pointing out the author but not the content

mk_stjames 3y ago

I'm always wondering about the, I don't even know what to call this, etiquette? of proposing PR's to projects like these that add a feature or a demo or whatnot to the main branch of a very focused project by adding something that is very different in interface, language, set and setting etc. So in this case, Tobi made this awesome little web interface that uses minimal HTML and JS as to stay in line with llama.cpp's stripped-down-ness. But it is still a completely different mode of operation, it's a 'new venue' essentially. What if GG didn't want such a thing? When is something like this better for a separately maintained repo and not a main merge? How do you know when it is OK to submit a PR to add something like this without overstepping (or is it always?) I see this with a few projects on github that really 'blow up' and everyone starts working on. They get a million PR's from people hacking things on it in their domain of knowledge, expanding the complexity (and potentially difficulty to maintain quality). Sometimes it gets weird feeling watching from the outside at least (I'm not a maintainer on any public FOSS). Just curious what others think because those are my thoughts that came to mind when I saw this.

xal 3y ago
GG would just say no and that's that. No hard feelings, that's what makes open source so great.
bfuller 3y ago
My view is you get so many people blatantly ripping off your code to try and pass it off as their own that you actually appreciate when you come across someone doing something novel or interesting that users like. That was my view anyway.
LawnGnome 3y ago
I generally think it's fine to do this sort of thing for your own benefit and open a PR as long as you're really 100% fine with "no, I'm not interested in merging this" being the answer. Where the problems tend to arise (in my experience, at least) is when people hack on something expecting that it will be merged, get invested in it, and then get upset when the maintainer(s) aren't interested. Checking in before starting to work on something is important if your goal is to have it merged, not just to do the work. The problem is that a lot of people start in the first category, but then move into the second category as they get invested in their project.

isoprophlex 3y ago

> I tried to match the spirit of llama.cpp and used minimalistic js dependencies and went with the ozempic css style of ggml.ai. Ozempic? The anti-diabetes drug? That's either a glorious typo or an interesting new adjective...

jahewson 3y ago
I did a double-take when I saw that too. It’s now prescribed as a weight-loss drug and is very much in the zeitgeist so yeah I think it’s a new adjective. Personally I’m going to stick with “light weight”.
- isoprophlex 3y ago
  The more I think about it, the more I'm loving it
xal 3y ago
Nat coined this usage https://twitter.com/natfriedman/status/1668656170645749761 https://twitter.com/natfriedman/status/1668656170645749761
- slaymaker1907 3y ago
  Lol, I’m totally using this to describe memory optimization from now on. There’s a lot of software out there that needs to be more ozempic.

ynniv 3y ago

i'm importing from js cdns instead of adding them here FWIW this seems counter to llama.cpp's philosophy.

mhh__ 3y ago
The philosophy in practice seems to be dirty deeds done dirt cheap rather than genuine simplicity (it only looks simple compared to python ecosystem)
xal 3y ago
I agree, I ended up getting rid of it. Only one dependency is downloaded (via bash script) and everything is baked into the binary now.

eclectic29 3y ago

Can someone shed light on how does a CEO with 3 kids get time to hack on something like this? Some might even argue that all the time spent doing this would’ve been better spent on CEO activities, but thankfully this is HN and people have hobbies, so that’s that, but this can be a very time consuming hobby.

xal 3y ago
I've tried to eliminate language around being "too busy" from my vocabulary and attempt to replace it with "can't prioritize". That's sometimes a bit awkward, but really trains better habits. Sometimes I prioritize hobbies if I feel I need it, and doing this seemed fun and useful besides!
- eclectic29 3y ago
  Thanks for your helpful response.
  - vdfs 3y ago
    He have this in his Twitter bio: "CEO by day, Dad in evening, hacker at night"

deleted 3y ago

[deleted]

seydor 3y ago

didnt people run llamacpp with oobabooga et al?

franchiser 3y ago

[dead]

sebastiennight 3y ago

I'm wondering whether the rendering of markdown through a regex that only deals with H3, strong, em and code is enough, since this implementation specifically seems to ask the bot to reply with markdown?[0] Other than that, it's cool to see this web interface happen. It's little things like this that make new tools easier to grasp for less technical users. I believe that people are more likely to go through the hurdles of learning difficult setup procedures (knowing the UX/UI will be easy to use) versus making the setup easier but having the daily use be too hard. [0]: https://github.com/ggerganov/llama.cpp/pull/1998/commits/c19daa4eb58cca85c0263d77958370e201bf9551 https://github.com/ggerganov/llama.cpp/pull/1998/commits/c19...

xal 3y ago
I had an entire implementation that used a proper state machine but I couldn’t get it simple and small enough for this single file thing. The regexp chain is dumb but clear.