12 ms·

On Rendering Diffs

210 points by amadeus 4mo ago

amadeus 4mo ago
A bit of a technical deep dive into how we built CodeView, a review surface that can handle rendering diffs of immense size, all in a browser.
- OutThisLife 4mo ago
  big fan :)
  - amadeus 4mo ago
    thank you!
- AlexErrant 4mo ago
  Feature request; `git diff --color-moved` uses colors to display moved chunks of code. Scanning https://diffs.com/docs https://diffs.com/docs it isn't obvious that yall support that; please add it :)
cipherself 4mo ago
For anyone else who's suffering, paste this in the console in devtools: document.getElementsByTagName('main')[0].style.margin = '0 auto';
- fat 4mo ago
  lmao - thank you!
  - AdamTReineke 4mo ago
    Aw, Pierre shut down? Is there a write-up on that? (The code review startup idea.)
- mi_lk 4mo ago
  There's always reader mode when an author tries to be cute about their layout
- quadrature 4mo ago
  thank you, how do they live like this.
- chrismorgan 4mo ago
  I’m surprised to see such a reaction to a left-aligned content column. I’d have said it was pretty common, though probably not as common as it used to be. If I were to pick one problem with the presentation, it would rather be the font-size (12px is unequivocally too small), or the use of monospace (simply unsuitable for body text). And incidentally, HN suffers from two of these three problems.
  - nextaccountic 4mo ago
    > I’m surprised to see such a reaction to a left-aligned content column. I’d have said it was pretty common, though probably not as common as it used to be. When it was common, columns weren't so thin I think
    - adrian_b 4mo ago
      Left aligned content was much less of a problem on the old displays with a 4:3 aspect. Nowadays, with wide displays, the browser windows may also be wide, especially when using a lot of open tabs, and left aligned content becomes much more annoying when it is thrown far away from your center of vision.
IshKebab 4mo ago
Very impressive! I doubt Github or Gitlab would ever do something as good as this but maybe there's a chance we could get it in Forgejo?
joosters 4mo ago
I was hoping that this would talk more about the logic behind generating a diff, rather than the optimisations involved in rendering the text. IMO (as someone who doesn't have to deal with the actual rendering) it would go a bit deeper into talking about deciding how to show what has changed. There's a lot of improvements that could be made there. e.g. "whitespace has changed here" so there's no real code changes involved. Or "this big list of imports has changed, and code formatting has line-wrapped the list into different lines" - gitlab for example copes poorly with this. I'd love to just see a clean diff that highlights the additional import, and not just ten lines of changes caused by adding one line to a big list of imported symbols/functions.
- amadeus 4mo ago
  One of our next big projects is actually to support semantic diffs, which I think will be a lot more applicable to what you're asking for here. Currently diffs just takes a normal git patch file, or generates one from 2 versions of a file.
- manquer 4mo ago
  > rather than the optimizations involved in rendering the text. Any views they have on this topic is going to come across as quite opinionated given their choices for text rendering for this post and general aesthetics of website.
  - amadeus 4mo ago
    Naw, the truth is I'm not really smart or intelligent enough to build a semantic diff system. For that you'll need to wait on a post from one of our smarter devs, this was a post about rendering diffs in a browser.
- stephbook 4mo ago
  Most projects start with tree-sitter and then switch to language-native parsers. Either way, it's not something you solve yourself – you just find the language-specific implementation load megabytes of WASM on the frontend or generate it on the backend. difftastic, semanticdiff.. lots of projects like that. Obviously they can offer stuff like "function name changed" instead of showing you 30 lines of +newName -oldName
logdahl 4mo ago
Maybe an intended effect, but the header ascii art is suspiciously misaligned... :^)
shaokind 4mo ago
Semi-related: have you considered making DiffsHub a browser extension, so you can serve private diffs as well? (I say this, having done a vibe-port of the code to a browser extension, so the underlying concept works.)
nerdypepper 4mo ago
rendering massive diffs is cool but ultimately a gimmick. in what scenario are you actually reading a 500k line diff? something i'd really want to see from forges is alternate diff techniques: like AST diffing.
- amadeus 4mo ago
  Performance and optimization is one of many pieces, but yes, it's a meme to render 500k lines. That said though, and maybe I didn't say it well in the post, the more performant and optimized your tool is, the less burden you put on developers and users. Sure you won't review 100k lines, but maybe the diff includes a ton of testing snapshots, or maybe it's a long running feature branch and you need to just quickly jump in and look at a specific change from a specific file. The less the developer or the user needs to think about `how` to render the diff or `how to navigate the diff`, the better we did our job.
- mrkcsc 4mo ago
  Optimizing for the P95/99 case of performance typically makes everything better as a whole.
deleted 4mo ago
[deleted]
cvince 4mo ago
Whatever happened to all the pretext hype? I feel like that would be perfect for rendering huge diffs.
- taejavu 4mo ago
  Can you select and copy text with pretext?
  - cvince 4mo ago
    Of course you can. It doesn't replace the dom, just changes how it's calculated. https://chenglou.me/pretext/dynamic-layout/ https://chenglou.me/pretext/dynamic-layout/
- amadeus 4mo ago
  Yes and no. It would help to improve things a bit when it comes the measure/reconciliation phase (unclear to say how much). However we've already done a pretty good job around batching writes vs reads. However passing a million lines of code through pretext is unlikely to be very efficient, so a lot of the work around estimation is still very important. That said, while I don't want to make pretext a direct dependency of the library, there's a good chance I'll explore the possibility of allowing devs to pass it in as an additional argument perhaps improve performance a bit. It should also be noted that we have a full API to support things like line annotations (comments, etc) that are entirely controlled by the user, so there's always a bit of a dynamic aspect there that would come into play
  - cvince 4mo ago
    I think it's amazing how deep you can go in something as simple as rendering diffs, and I'm really grateful you shared that writeup (in fact I've been following you loosely since the Kiip days as a then aspiring designer+developer type). I've always been curious how products like ag-grid are able to allow you to lazy-load 10,000,000+ rows in a table without the blanking you described, and I imagine this type of method describes how it can be done.
akdor1154 4mo ago
I disagree with the theory that scrolling frame rate doesn't need to be smooth for scrolling to feel smooth. On mobile it kinda does. Scrolling diffs on mobile just kinda feels crap. I have been spoiled by years of engineer hours spent getting scrolling to be 60- or even 120Hz smooth to match my finger, and diffs just.. isn't. I know this is frustrating to hear, and that this is technically compounded by mobile probably having the lowest device performance to be playing with too, but.. There you go.
- embedding-shape 4mo ago
  Matters a great deal on desktop too, and laptops for that matter. Even more on platforms like macOS that smooths scrolling by default too, but very noticeable on Windows and various Linux distributions too when native scrolling is janky/choppy, and it frustrates even casual users.
- amadeus 4mo ago
  > disagree with the theory that scrolling frame rate doesn't need to be smooth for scrolling to feel smooth It's possible you might be misunderstanding what I was trying to say here because 120hz scrolling on a 120hz device was the goal and why one of those virtualization techniques was not acceptable to me which lead me to coming up with a novel workaround to this problem (Inverse Sticky Technique). CodeView uses a system that allows scrolling to update at your native framerate (120hz) WITHOUT needing Javascript needing to keep up at 120hz. If you're seeing stuttering while scrolling on https://diffshub.com https://diffshub.com would love to know more context (device/diff link/etc) because that is very much NOT our experience.
  - akdor1154 4mo ago
    Even the linked ghostty PR on your home shows this - this is Firefox Android on a Nokia XR21 / TA-1486. It's not unuseable, but it definitely feels like 'js hacking my scrolling' and not a native surface flinging around. The experience is actually worse with smaller movements, i guess because my brain is more conscious when breaking the 'finger physically moving the text' illusion. I don't mean to be dismissive - you're working on a really hard problem, and you're clearly approaching it with a mindset of perfection. I'm posting because I know you're probably able to solve this too :) Edit: as a point of (unfair) comparison, the codemirror Huge File demo works fine: https://codemirror.net/examples/million/ https://codemirror.net/examples/million/ It does suffer from the occasional partial paint when quickly coasting, but I'm not bothered by this at all, it's far less intrusive than dropping frames / stuttering / etc.
brap 4mo ago
I don’t have much to say except I appreciate the ASCII art.
tomtom1337 4mo ago
It is SO NICE to see people working on making fast, nice-to-use tools. It's a lovely experience to use diffshub. Thank you for creating it, and than you for the great write-up! (I made it "that far" )
- amadeus 4mo ago
  Appreciate it, thank you!
darkamaul 4mo ago
What an interesting article. I did not assume I would read it until the end when I opened it, but the writing was super clear and easy to follow. At the end, I admire the craft and patience to try to solve code diff rendering, and wish the folks at GitHub could put the same effort to improve their platform. On a side note, I feel that we’re going to see more and more of this type of agentic usage, in well defined sub tasks, and the ability of a model to try many possibilities is a huge gift here.
- serbrech 4mo ago
  Now GitHub can point an agent at this blog and say “implement this” :)
- amadeus 4mo ago
  Hey thank you, appreciate your kind words! I don’t write much and was quite an effort to get all this written out!
eblanshey 4mo ago
It's cool seeing all the engineering that goes into optimizing performance of diffs. I'm working on a FreeCAD workbench that generates diffs on CAD model trees[0], and although my bottlenecks are a bit different, I can still implement some of your optimizations down the line if needed (such as deferred syntax highlighting). My main bottleneck is that I do a complete diff on all open + changed documents in the repository up front, because due to how document properties are stored, I won't know if the file has meaningful changes until I compute the full diff (FreeCAD may save the document, but not have anything meaningful change.) [0] https://github.com/eblanshey/HistoryWorkbench https://github.com/eblanshey/HistoryWorkbench
charcircuit 4mo ago
I feel like virtualization is not the right way to handle things. It adds so much complexity and makes the user experience buggy due to breaking optimizations and features of browsers. Computers are very powerful these days and have a done of resources that they can use. We should be able to handle large diffs without any crazy tricks.
- mrkcsc 4mo ago
  > I feel like virtualization is not the right way to handle things. How would you handle it?
  - charcircuit 4mo ago
    Keep things as simple as possible and put the whole thing in the DOM. Then if there are performance problems address the scaling problems themselves instead of trying to avoid scaling. For example things like only rendering what is visible should be handled by the browser and not by messing with the DOM. The DOM is not synchronized with the browser's renderer so it will always end up hacky.
    - mrkcsc 4mo ago
      OK but thats simply doesn't perform. You can't say "only rendering what is visible should be handled by the browser" and call that a solution unless you have a magic wand to make Chrome/other browsers do this. The browser doesn't do this, and so you can either do what you say and have your browser freeze when you load up a million line diff, or you can fix things within your control which is what the author is doing.
      - charcircuit 4mo ago
        I just tested it now and Chrome does not freeze when opening up a page with a million lines of code. I repeated a file from the Linux kernel 600 times to reach over 1.1 million lines and I put each line in a div so I could alternate red and green background colors. >you can fix things within your control Blink is open source. Improving the browser is fully within your control and since browser automatically updates these optimizations will make it to your end users in a relatively short amount of time.
thangalin 4mo ago
When producing TreeTrek, I went with rudimentary diffs that account for colourblind developers: https://repo.autonoma.ca/repo/treetrek/commit/3fe9360599ae23989b964b5a73ef9b5986ea141b https://repo.autonoma.ca/repo/treetrek/commit/3fe9360599ae23... The diffs rendering library looks amazing: https://diffs.com/ https://diffs.com/ Presumably the red-green issue is a simple CSS update?
- amadeus 4mo ago
  Yup! You can pick from a bunch of different themes and use css variables to override the core colors as well!
philsnow 4mo ago
> so hit play on sandstorm For a brief hopeful moment, I thought this was the .io kind of sandstorm
archargelod 4mo ago
Is this really a hard problem? Rendering some text with coloring? I believe we're just making everything harder than it needs by constraining it to a browser interface. This would be trivial as a terminal application written in native code.
- amadeus 4mo ago
  Yeah, my bad. Sorry i wasted your time :(
- mrkcsc 4mo ago
  What are you even talking about, this post is about rendering code diffs in the context of the web. Whether or not it would be "trivial" in a native terminal application is irrelevant.
gloria_mundi 4mo ago
I don't understand the point of the inverse sticky technique. Scrolling too fast still breaks the experience (content refuses to scroll), and in a way that, at least to me, feels more disruptive than blanking for a fraction of a second. I might just be too used to blanking. Also ... shouldn't browsers just be able to render the diff without any of the trickery? Is the browser's job actually that hard for long pages, or are they just not optimising for this? Or is there some other reason for the virtualisation (e.g. memory usage)?
- deleted 4mo ago
 [deleted]
- amadeus 4mo ago
 I probably didn’t explain this well enough, but your render times always have to be within the frame buffer (16.6ms for 60hz or 8.3ms for 120hz). Under normal circumstances even if you occasionally blow a frame buffer, with the over-scroll you won’t hit the sticky bounds. The only time you will is if you’re scrolling at a rate where the jumps are quite large — large and fast enough typically where you’re not going to have a frame of reference for what you should see vs what you are seeing to notice you are behind. Ultimately scrolling is managed on a separate thread from JS, which means if you do like an opt+click on the scroll bar, you’re going to make a jump that JavaScript can never keep up with, even if you’re under your frame times. And with regards to safari, if your requestAnimationFrame is capped at 60hz but your scrolling is GPU composited at 120hz, this is the only way to keep scrolling at 120hz with 60hz dom updates and never see any blanking.
- shay_ker 4mo ago
 hmm it does seem like a good point that browsers should be able to do this natively. the DOM already holds the content, and browsers manage what's rendered in the viewport.
- Joker_vD 4mo ago
 Also, I am deeply suspicious if they can properly support searching with Ctrl+F... And yes, browsers should be just able to render the diffs, if the styling is somewhat minimal and there is no JS: I have a 6.3-MiB large HTML file with complete transcript of a certain TV series, formatted like <dd>Character #1: Well, that was interesting all right.</dd> <dd>Character #2: [sigh]</dd></dl> <dl><dd>Character #1: Lemme help you.</dd> <dd>Character #3: Yeehaw!</dd> <dd>[thump]</dd> (the </dl><dl> produces a small paragraph break, to indicate scene transitions), and it scrolls and searches just fine.
andrew_kwak 4mo ago
I remember wrestling with diff tools back in the day. A good visualization can make a world of difference. How does it handle large files with lots of changes?
alexpandey 4mo ago
[flagged]
lxe 4mo ago
This is pretty awesome. I work with editors and monaco-like things a ton, and I review (look at) very large PRs very often. Having this speedy optimized interface is a delight. Check out their trees lib as well.
_ZeD_ 4mo ago
While for "simple" diffs these UIs are "fine", I'm still struggling (after 15+ years of search) for a really good tool that could help me with 3-file diffs... I'm still stuck with (k)diff3, and, while they work, I would really like to a more integrate web interface for my projects
- v3gas 4mo ago
  Can you give an example of why you need that? Genuinely curious
  - jiehong 4mo ago
    git merge conflicts?
speedgoose 4mo ago
> Safari, for example, currently caps requestAnimationFrame at 60Hz even on higher refresh-rate displays Of course. It’s often Safari.
bryanlarsen 4mo ago
The problem with large diffs is usually with the human, not the computer. Large diffs are very hard to review, so more tools that help with the understanding of a large diff would be very welcome.
Igor_Wiwi 4mo ago
this approach is quite similar to what I use for rendering huge markdown files in https://mdview.io https://mdview.io : 10MB files renders correctly and even Table of Content works. 15MB renders but scrolling is not as smooth as I wanted, will work on that. The approach described here https://igorstechnoclub.com/how-i-render-10mb-markdown-files-in-the-browser/ https://igorstechnoclub.com/how-i-render-10mb-markdown-files...
eqvinox 4mo ago
nit: there is no Linux "v6" or "v7". The first and second version component cannot be separated, it's v6.0 and v7.0. Versions 6.0 and 6.1 have as much in common as 6.19 and 7.0. You could also call them 3.60 and 3.80, or 2.6.100 and 2.6.120.
_young_grug_ 4mo ago
That video has most tastefully added backtrack of all of HN

210 points by amadeus 4mo ago

A bit of a technical deep dive into how we built CodeView, a review surface that can handle rendering diffs of immense size, all in a browser.

OutThisLife 4mo ago
big fan :)
- amadeus 4mo ago
  thank you!
AlexErrant 4mo ago
Feature request; `git diff --color-moved` uses colors to display moved chunks of code. Scanning https://diffs.com/docs https://diffs.com/docs it isn't obvious that yall support that; please add it :)

cipherself 4mo ago

For anyone else who's suffering, paste this in the console in devtools: document.getElementsByTagName('main')[0].style.margin = '0 auto';

fat 4mo ago
lmao - thank you!
- AdamTReineke 4mo ago
  Aw, Pierre shut down? Is there a write-up on that? (The code review startup idea.)
mi_lk 4mo ago
There's always reader mode when an author tries to be cute about their layout
quadrature 4mo ago
thank you, how do they live like this.
chrismorgan 4mo ago
I’m surprised to see such a reaction to a left-aligned content column. I’d have said it was pretty common, though probably not as common as it used to be. If I were to pick one problem with the presentation, it would rather be the font-size (12px is unequivocally too small), or the use of monospace (simply unsuitable for body text). And incidentally, HN suffers from two of these three problems.
- nextaccountic 4mo ago
  > I’m surprised to see such a reaction to a left-aligned content column. I’d have said it was pretty common, though probably not as common as it used to be. When it was common, columns weren't so thin I think
  - adrian_b 4mo ago
    Left aligned content was much less of a problem on the old displays with a 4:3 aspect. Nowadays, with wide displays, the browser windows may also be wide, especially when using a lot of open tabs, and left aligned content becomes much more annoying when it is thrown far away from your center of vision.

IshKebab 4mo ago

Very impressive! I doubt Github or Gitlab would ever do something as good as this but maybe there's a chance we could get it in Forgejo?

joosters 4mo ago

I was hoping that this would talk more about the logic behind generating a diff, rather than the optimisations involved in rendering the text. IMO (as someone who doesn't have to deal with the actual rendering) it would go a bit deeper into talking about deciding how to show what has changed. There's a lot of improvements that could be made there. e.g. "whitespace has changed here" so there's no real code changes involved. Or "this big list of imports has changed, and code formatting has line-wrapped the list into different lines" - gitlab for example copes poorly with this. I'd love to just see a clean diff that highlights the additional import, and not just ten lines of changes caused by adding one line to a big list of imported symbols/functions.

amadeus 4mo ago
One of our next big projects is actually to support semantic diffs, which I think will be a lot more applicable to what you're asking for here. Currently diffs just takes a normal git patch file, or generates one from 2 versions of a file.
manquer 4mo ago
> rather than the optimizations involved in rendering the text. Any views they have on this topic is going to come across as quite opinionated given their choices for text rendering for this post and general aesthetics of website.
- amadeus 4mo ago
  Naw, the truth is I'm not really smart or intelligent enough to build a semantic diff system. For that you'll need to wait on a post from one of our smarter devs, this was a post about rendering diffs in a browser.
stephbook 4mo ago
Most projects start with tree-sitter and then switch to language-native parsers. Either way, it's not something you solve yourself – you just find the language-specific implementation load megabytes of WASM on the frontend or generate it on the backend. difftastic, semanticdiff.. lots of projects like that. Obviously they can offer stuff like "function name changed" instead of showing you 30 lines of +newName -oldName

logdahl 4mo ago

Maybe an intended effect, but the header ascii art is suspiciously misaligned... :^)

shaokind 4mo ago

Semi-related: have you considered making DiffsHub a browser extension, so you can serve private diffs as well? (I say this, having done a vibe-port of the code to a browser extension, so the underlying concept works.)

nerdypepper 4mo ago

rendering massive diffs is cool but ultimately a gimmick. in what scenario are you actually reading a 500k line diff? something i'd really want to see from forges is alternate diff techniques: like AST diffing.

amadeus 4mo ago
Performance and optimization is one of many pieces, but yes, it's a meme to render 500k lines. That said though, and maybe I didn't say it well in the post, the more performant and optimized your tool is, the less burden you put on developers and users. Sure you won't review 100k lines, but maybe the diff includes a ton of testing snapshots, or maybe it's a long running feature branch and you need to just quickly jump in and look at a specific change from a specific file. The less the developer or the user needs to think about `how` to render the diff or `how to navigate the diff`, the better we did our job.
mrkcsc 4mo ago
Optimizing for the P95/99 case of performance typically makes everything better as a whole.

deleted 4mo ago

[deleted]

cvince 4mo ago

Whatever happened to all the pretext hype? I feel like that would be perfect for rendering huge diffs.

taejavu 4mo ago
Can you select and copy text with pretext?
- cvince 4mo ago
  Of course you can. It doesn't replace the dom, just changes how it's calculated. https://chenglou.me/pretext/dynamic-layout/ https://chenglou.me/pretext/dynamic-layout/
amadeus 4mo ago
Yes and no. It would help to improve things a bit when it comes the measure/reconciliation phase (unclear to say how much). However we've already done a pretty good job around batching writes vs reads. However passing a million lines of code through pretext is unlikely to be very efficient, so a lot of the work around estimation is still very important. That said, while I don't want to make pretext a direct dependency of the library, there's a good chance I'll explore the possibility of allowing devs to pass it in as an additional argument perhaps improve performance a bit. It should also be noted that we have a full API to support things like line annotations (comments, etc) that are entirely controlled by the user, so there's always a bit of a dynamic aspect there that would come into play
- cvince 4mo ago
  I think it's amazing how deep you can go in something as simple as rendering diffs, and I'm really grateful you shared that writeup (in fact I've been following you loosely since the Kiip days as a then aspiring designer+developer type). I've always been curious how products like ag-grid are able to allow you to lazy-load 10,000,000+ rows in a table without the blanking you described, and I imagine this type of method describes how it can be done.

akdor1154 4mo ago

I disagree with the theory that scrolling frame rate doesn't need to be smooth for scrolling to feel smooth. On mobile it kinda does. Scrolling diffs on mobile just kinda feels crap. I have been spoiled by years of engineer hours spent getting scrolling to be 60- or even 120Hz smooth to match my finger, and diffs just.. isn't. I know this is frustrating to hear, and that this is technically compounded by mobile probably having the lowest device performance to be playing with too, but.. There you go.

embedding-shape 4mo ago
Matters a great deal on desktop too, and laptops for that matter. Even more on platforms like macOS that smooths scrolling by default too, but very noticeable on Windows and various Linux distributions too when native scrolling is janky/choppy, and it frustrates even casual users.
amadeus 4mo ago
> disagree with the theory that scrolling frame rate doesn't need to be smooth for scrolling to feel smooth It's possible you might be misunderstanding what I was trying to say here because 120hz scrolling on a 120hz device was the goal and why one of those virtualization techniques was not acceptable to me which lead me to coming up with a novel workaround to this problem (Inverse Sticky Technique). CodeView uses a system that allows scrolling to update at your native framerate (120hz) WITHOUT needing Javascript needing to keep up at 120hz. If you're seeing stuttering while scrolling on https://diffshub.com https://diffshub.com would love to know more context (device/diff link/etc) because that is very much NOT our experience.
- akdor1154 4mo ago
  Even the linked ghostty PR on your home shows this - this is Firefox Android on a Nokia XR21 / TA-1486. It's not unuseable, but it definitely feels like 'js hacking my scrolling' and not a native surface flinging around. The experience is actually worse with smaller movements, i guess because my brain is more conscious when breaking the 'finger physically moving the text' illusion. I don't mean to be dismissive - you're working on a really hard problem, and you're clearly approaching it with a mindset of perfection. I'm posting because I know you're probably able to solve this too :) Edit: as a point of (unfair) comparison, the codemirror Huge File demo works fine: https://codemirror.net/examples/million/ https://codemirror.net/examples/million/ It does suffer from the occasional partial paint when quickly coasting, but I'm not bothered by this at all, it's far less intrusive than dropping frames / stuttering / etc.

brap 4mo ago

I don’t have much to say except I appreciate the ASCII art.

tomtom1337 4mo ago

It is SO NICE to see people working on making fast, nice-to-use tools. It's a lovely experience to use diffshub. Thank you for creating it, and than you for the great write-up! (I made it "that far" )

amadeus 4mo ago
Appreciate it, thank you!

darkamaul 4mo ago

What an interesting article. I did not assume I would read it until the end when I opened it, but the writing was super clear and easy to follow. At the end, I admire the craft and patience to try to solve code diff rendering, and wish the folks at GitHub could put the same effort to improve their platform. On a side note, I feel that we’re going to see more and more of this type of agentic usage, in well defined sub tasks, and the ability of a model to try many possibilities is a huge gift here.

serbrech 4mo ago
Now GitHub can point an agent at this blog and say “implement this” :)
amadeus 4mo ago
Hey thank you, appreciate your kind words! I don’t write much and was quite an effort to get all this written out!

eblanshey 4mo ago

It's cool seeing all the engineering that goes into optimizing performance of diffs. I'm working on a FreeCAD workbench that generates diffs on CAD model trees[0], and although my bottlenecks are a bit different, I can still implement some of your optimizations down the line if needed (such as deferred syntax highlighting). My main bottleneck is that I do a complete diff on all open + changed documents in the repository up front, because due to how document properties are stored, I won't know if the file has meaningful changes until I compute the full diff (FreeCAD may save the document, but not have anything meaningful change.) [0] https://github.com/eblanshey/HistoryWorkbench https://github.com/eblanshey/HistoryWorkbench

charcircuit 4mo ago

I feel like virtualization is not the right way to handle things. It adds so much complexity and makes the user experience buggy due to breaking optimizations and features of browsers. Computers are very powerful these days and have a done of resources that they can use. We should be able to handle large diffs without any crazy tricks.

mrkcsc 4mo ago
> I feel like virtualization is not the right way to handle things. How would you handle it?
- charcircuit 4mo ago
  Keep things as simple as possible and put the whole thing in the DOM. Then if there are performance problems address the scaling problems themselves instead of trying to avoid scaling. For example things like only rendering what is visible should be handled by the browser and not by messing with the DOM. The DOM is not synchronized with the browser's renderer so it will always end up hacky.
  - mrkcsc 4mo ago
    OK but thats simply doesn't perform. You can't say "only rendering what is visible should be handled by the browser" and call that a solution unless you have a magic wand to make Chrome/other browsers do this. The browser doesn't do this, and so you can either do what you say and have your browser freeze when you load up a million line diff, or you can fix things within your control which is what the author is doing.
    - charcircuit 4mo ago
      I just tested it now and Chrome does not freeze when opening up a page with a million lines of code. I repeated a file from the Linux kernel 600 times to reach over 1.1 million lines and I put each line in a div so I could alternate red and green background colors. >you can fix things within your control Blink is open source. Improving the browser is fully within your control and since browser automatically updates these optimizations will make it to your end users in a relatively short amount of time.

thangalin 4mo ago

When producing TreeTrek, I went with rudimentary diffs that account for colourblind developers: https://repo.autonoma.ca/repo/treetrek/commit/3fe9360599ae23989b964b5a73ef9b5986ea141b https://repo.autonoma.ca/repo/treetrek/commit/3fe9360599ae23... The diffs rendering library looks amazing: https://diffs.com/ https://diffs.com/ Presumably the red-green issue is a simple CSS update?

amadeus 4mo ago
Yup! You can pick from a bunch of different themes and use css variables to override the core colors as well!

philsnow 4mo ago

> so hit play on sandstorm For a brief hopeful moment, I thought this was the .io kind of sandstorm

archargelod 4mo ago

Is this really a hard problem? Rendering some text with coloring? I believe we're just making everything harder than it needs by constraining it to a browser interface. This would be trivial as a terminal application written in native code.

amadeus 4mo ago
Yeah, my bad. Sorry i wasted your time :(
mrkcsc 4mo ago
What are you even talking about, this post is about rendering code diffs in the context of the web. Whether or not it would be "trivial" in a native terminal application is irrelevant.

gloria_mundi 4mo ago

I don't understand the point of the inverse sticky technique. Scrolling too fast still breaks the experience (content refuses to scroll), and in a way that, at least to me, feels more disruptive than blanking for a fraction of a second. I might just be too used to blanking. Also ... shouldn't browsers just be able to render the diff without any of the trickery? Is the browser's job actually that hard for long pages, or are they just not optimising for this? Or is there some other reason for the virtualisation (e.g. memory usage)?

deleted 4mo ago
[deleted]
amadeus 4mo ago
I probably didn’t explain this well enough, but your render times always have to be within the frame buffer (16.6ms for 60hz or 8.3ms for 120hz). Under normal circumstances even if you occasionally blow a frame buffer, with the over-scroll you won’t hit the sticky bounds. The only time you will is if you’re scrolling at a rate where the jumps are quite large — large and fast enough typically where you’re not going to have a frame of reference for what you should see vs what you are seeing to notice you are behind. Ultimately scrolling is managed on a separate thread from JS, which means if you do like an opt+click on the scroll bar, you’re going to make a jump that JavaScript can never keep up with, even if you’re under your frame times. And with regards to safari, if your requestAnimationFrame is capped at 60hz but your scrolling is GPU composited at 120hz, this is the only way to keep scrolling at 120hz with 60hz dom updates and never see any blanking.
shay_ker 4mo ago
hmm it does seem like a good point that browsers should be able to do this natively. the DOM already holds the content, and browsers manage what's rendered in the viewport.
Joker_vD 4mo ago
Also, I am deeply suspicious if they can properly support searching with Ctrl+F... And yes, browsers should be just able to render the diffs, if the styling is somewhat minimal and there is no JS: I have a 6.3-MiB large HTML file with complete transcript of a certain TV series, formatted like <dd>Character #1: Well, that was interesting all right.</dd> <dd>Character #2: [sigh]</dd></dl> <dl><dd>Character #1: Lemme help you.</dd> <dd>Character #3: Yeehaw!</dd> <dd>[thump]</dd> (the </dl><dl> produces a small paragraph break, to indicate scene transitions), and it scrolls and searches just fine.

andrew_kwak 4mo ago

I remember wrestling with diff tools back in the day. A good visualization can make a world of difference. How does it handle large files with lots of changes?

alexpandey 4mo ago

[flagged]

lxe 4mo ago

This is pretty awesome. I work with editors and monaco-like things a ton, and I review (look at) very large PRs very often. Having this speedy optimized interface is a delight. Check out their trees lib as well.

_ZeD_ 4mo ago

While for "simple" diffs these UIs are "fine", I'm still struggling (after 15+ years of search) for a really good tool that could help me with 3-file diffs... I'm still stuck with (k)diff3, and, while they work, I would really like to a more integrate web interface for my projects

v3gas 4mo ago
Can you give an example of why you need that? Genuinely curious
- jiehong 4mo ago
  git merge conflicts?

speedgoose 4mo ago

> Safari, for example, currently caps requestAnimationFrame at 60Hz even on higher refresh-rate displays Of course. It’s often Safari.

bryanlarsen 4mo ago

The problem with large diffs is usually with the human, not the computer. Large diffs are very hard to review, so more tools that help with the understanding of a large diff would be very welcome.

Igor_Wiwi 4mo ago

this approach is quite similar to what I use for rendering huge markdown files in https://mdview.io https://mdview.io : 10MB files renders correctly and even Table of Content works. 15MB renders but scrolling is not as smooth as I wanted, will work on that. The approach described here https://igorstechnoclub.com/how-i-render-10mb-markdown-files-in-the-browser/ https://igorstechnoclub.com/how-i-render-10mb-markdown-files...

eqvinox 4mo ago

nit: there is no Linux "v6" or "v7". The first and second version component cannot be separated, it's v6.0 and v7.0. Versions 6.0 and 6.1 have as much in common as 6.19 and 7.0. You could also call them 3.60 and 3.80, or 2.6.100 and 2.6.120.

_young_grug_ 4mo ago

That video has most tastefully added backtrack of all of HN