9 ms·
Differential Transformer
- pikseladam 2y agoDid this mean they solved the hallucination problem of transformers? edit: not fully but it gives promising results. quiet an improvement actually.
- lafreb 2y agoThe paper says that they've improved hallucination mitigation, but not really "solved" the issue.
- Rhapso 2y ago"Hallucination" isn't really a problem that can be "fixed". Its just model error. The root problem is simply that the model doesn't capture reality, just an approximation. What we are incorrectly calling "hallucination" is just the best the model has to offer.
- dilap 2y agoit can be fixed in theory if the model knows-what-it-knows, to avoid saying things its uncertain about (this is what (some) humans do to reduce the frequency w which they say untrue things). theres some promising research using this idea, tho i dont have it at hand.
- AnimalMuppet 2y agoI'm pretty sure there's something I don't understand, but: Doesn't an LLM pick the "most probable next symbol" (or, depending on temperature, one of the most probable next symbols)? To do that, doesn't it have to have some idea of what the probability is? Couldn't it then, if the probability falls below some threshold, say "I don't know" instead of giving what it knows is a low-probability answer?
- viraptor 2y ago> Doesn't an LLM pick the "most probable next symbol" Yes, but that very rarely matters. (Almost never when it's brought up in discussions) > Couldn't it then, if the probability falls below some threshold, say "I don't know" instead of giving what it knows is a low-probability answer? A low probability doesn't necessarily mean something's incorrect. Responding to your question in French would also have very low probability, even if it's correct. There's also some nuance around what's classified as a hallucination... Maybe something in the training data did suggest that answer as correct. There are ideas similar to this one though. It's just a bit more complex than pure probabilities going down. https://arxiv.org/abs/2405.19648 https://arxiv.org/abs/2405.19648
- anticensor 2y ago> Responding to your question in French would also have very low probability, even if it's correct. It's actually a common utterance in Paris.
- anon291 2y agoYou need to separate out the LLM, which only produces a set of probabilities, from the system, which includes the LLM and the sampling methodology. Sampling is currently not very intelligent at all. The next bit of confusion is that the 'probability' isn't 'real'. It's not an actual probability but a weight that sums up to one, which is close enough to how probability works that we call it that. However, sometimes there are several good answers and so all the good answers get a lower probability because there are 5 of them. A fixed threshold is not a good idea in this case. Instead, smarter sampling methods are necessary. One possibility is that if we do have seeming confusion, to put a 'confusion marker' into the text and predict the next output and train models to refine the answer as they go along. Not sure if any work has been done here, but this seems to go along with what you're interested in
- viraptor 2y ago> However, sometimes there are several good answers and so all the good answers get a lower probability because there are 5 of them. That's the result after softmax. If you want to act on the raw results, you can still do that.
- atrus 2y agoI don't think that fixes it, even in theory, since there's always some uncertainty.
- hoosieree 2y agoLLMs can't hallucinate. They generate the next most likely token in a sequence. Whether that sequence matches any kind of objective truth is orthogonal to how models work. I suppose depending on your point of view, LLMs either can't hallucinate, or that's all they can do.
- CooCooCaCha 2y agoWhenever someone takes issue with using the word “hallucinate” with LLMs I get the impression they’re trying to convince me that hallucination is good. Why do you care so much about this particular issue? And why can’t hallucination be something we can aim to improve?
- ToValueFunfetti 2y ago>Whether that sequence matches any kind of objective truth is orthogonal to how models work. Empirically, this cannot be true. If it were, it would be statistically shocking how often models coincidentally say true things. The training does not perfectly align the model with truth, but 'orthogonal' is off by a minimum of 45 degrees.
- viraptor 2y agoIt matches the training data. Whether the training data matches truth (and whether it's correctly understood - sarcasm included) is a completely separate thing. > The training does not perfectly align the model with truth, but 'orthogonal' Nitpicky, but the more dimensions you have, the easier it is for almost everything to be orthogonal. (https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-dimensions-orthoginality https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-dime...) That's why averaging embeddings works.
- timcobb 2y agoIsn't this the same thing that happens when you train a human on truths vs falsehoods?
- ToValueFunfetti 2y ago
- visarga 2y agoThis reminds me it's easy to train similarity models, hard to train identity/equivalence prediction. Two strings can be similar in many ways, like "Address Line 1" and "Address Line 2" or "Position_X" and "Position_Y", yet distinct in meaning. That one character makes all the difference. On the other hand "Vendor Name" is equivalent with "Seller Company" even though they are pretty different lexically. The dot product, which is at the core of attention, is good for similarity not identity. I think this is why models hallucinate - how can they tell the distinction between "I have trained on this fact" and "Looks like something I trained on".
- tucnak 2y agoI'm led to believe this is mostly because "known unknowns" are not well-represented in the training datasets... I think, instead of bothering with refusals and enforcing a particular "voice" with excessive RL, they ought to focus more on identifying "gaps" in the datasets and feeding them back, perhaps they're already doing this with synthetic data / distillation.
- spencerchubb 2y agoit's not "just" model error during pre-training, there is never an incentive for the model to say "I don't know" because it would be penalized. the model is incentivized to make an educated guess large transformer models are really good at approximating their dataset. there is no data on the internet about what LLMs know. and even if there were such data, it would probably become obsolete soon that being said, maybe a big shift in the architecture could solve this. I hope!
- happypumpkin 2y ago> it would probably become obsolete soon Suppose there are many times more posts about something one generation of LLMs can't do (arithmetic, tic-tac-toe, whatever), than posts about how the next generation of models can do that task successfully. I think this is probably the case. While I doubt it will happen, it would be somewhat funny if training on that text caused a future model to claim it can't do something that it "should" be able to because it internalized that it was an LLM and "LLMs can't do X."
- spencerchubb 2y agoalso presumes that the LLM knows it is an LLM
- adwn 2y agoSystem prompts sometimes contain the information that "it" is an LLM. Maybe in the future, those prompts will include motivational phrases, like "You can do it!" or "Believe in yourself, then you can achieve anything."
- Vecr 2y agoThey're generally fine tuned not to. I'm not sure how long that will hold though.
- ykonstant 2y ago- Are you an LLM? - As a Large Language Model, I am fine tuned to be unable to answer this question.
- watsonmusic 2y agothat would be huge!
- HarHarVeryFunny 2y agoI don't think there's any narrow definition of what "hallucination" means. It generally refers to the model giving non-factual answers in contexts that are meant to be factual, but not all causes of this are going to be fixable without very major changes. The fundamental issue is that most of the time LLMs are going to be combining statistics derived from many training samples when generating a single continuation, and there is just no guarantee that this will result in a semantically coherent response. Of course the model's depth of parsing and semantic analysis usually means that each generated word is highly plausible, but this isn't the same as being factually correct, especially so in these cases where the model is drawing on multiple sources to create a mashup response, which is the normal mode of operation.
- ExxKA 2y agoVery interesting. Currently working on timeseries with Transformers. Let me know if anyone else out there is also reading it from that context.
- magicalhippo 2y agoThe visualization reveals that Transformer tends to allocate only a small proportion of attention scores to the correct answer, while disproportionately focusing on irrelevant context. [...] Specifically, we partition the query and key vectors into two groups and compute two separate softmax attention maps. Then the result of subtracting these two maps is regarded as attention scores. [...] The approach is analogous to noise-canceling headphones and differential amplifiers in electrical engineering, where the difference between two signals cancels out common-mode noise. Simple change, with seemingly decent improvements across the board.
- watsonmusic 2y ago[flagged]
- deleted 2y ago[deleted]
- msoad 2y agoLike most things in this new world of Machine Learning, I'm really confused why this works? The analogy to noise-cancelling headphones is helpful but in that case we clearly know which is signal and which is noise. Here, if we knew why would we even bother to the noise-cancelling work?
- watsonmusic 2y agothe model is supposed to learn this
- _hl_ 2y agoSome of the "prior art" here is ladder networks and to some handwavy extent residual nets, both of which can be interpreted as training the model on reducing the error to its previous predictions as opposed to predicting the final result directly. I think some intuition for why it works has to do with changing the gradient descent landscape to be a bit friendlier towards learning in small baby steps, as you are now explicitly designing the network around the idea that it will start off making lots of errors in its predictions and then get better over time.
- HarHarVeryFunny 2y agoI don't understand either. It seems the general idea is that they calculate attention twice, which due to random initialization might be expected to give two slightly different results. I'd have thought that what these two attention maps would have in common would be the signal, and where they would differ would be noise, so rather than subtracting them (resulting in all noise?!) what you really want is to add (so the common signal gets reinforced) and normalize.
- Carlseymanh 2y agoI think there might be some communalities with system engineering, where you subtract the output from the input in order to get a control signal that steers the plant to the target values. I too fail to see how that would be supposed to work in practice.
- kelseyfrog 2y ago
- watsonmusic 2y agoThe modification is simple and beautiful. And the improvements are quite significant.
- campers 2y agoThe tl;dr on high level performance improvements "The scaling curves indicate that Diff Transformer requires only about 65% of model size or training tokens needed by Transformer to achieve comparable language modeling performance." "Diff Transformer retains high performance even at reduced bit-widths, ranging from 16 bits to 6 bits. In comparison, Transformer’s accuracy significantly drops with 6-bit quantization. The 4-bit Diff Transformer achieves comparable accuracy as the 6-bit Transformer, and outperforms the 4-bit Transformer by about 25% in accuracy."
- deleted 2y ago[deleted]
- breukh 2y ago[flagged]
- digdugdirk 2y agoIs there any way to replicate this with existing models, or are we going to need to wait for models to be trained in this style? I'm imagining a smaller model examining the output tokens of a larger model and metaphorically slapping it on the wrist with a ruler if the output tokens start drifting off topic. Not quite the same, but an entertaining thought nonetheless.
- causal 2y agoIt's a different attention mechanism with a different map setup, so fundamentally a different type of model
- om8 2y agoLooks like it is a drop in replacement for attention, but models will need to be retrained for this one, yes.
- aDyslecticCrow 2y agoIt may not need to be entirely retrained. The value spans and input are the same, and no extra weights are needed. You may be able to tune an existing model with this attention mechanism and get some of the benefits. But overall... it's mainly a training change, so training is needed to make a difference.
- bionhoward 2y agoYes, I believe this is possible, you could clone weights of one or more existing models and fine tune them in groups with different random seeds for noise/drop to produce reasonable outputs under a differential transformer decoding scheme whereby tokens with disagreement receive more attention (surprisal analysis)
- patcon 2y agoI wonder what is lost here. Surely there's a trade-off... I'm wondering if there's any effect of "creativity", or ability to interpolate between concepts. Hallucination and creativity feel very related to me. I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between
- watsonmusic 2y agonot all hallucinations are creativity Imaginate that for a RAG application, the model is supposed to follow the given documents
- magicalhippo 2y ago> Surely there's a trade-off... For one, speed and memory. They have twice as many Q and K weights in the attention blocks, leading to a ~10% reduction in throughput on their H100 (table 7 in appendix A).
- karmasimida 2y agoI mean it doesn’t necessarily needs 2x QK to match that performance, in terms of accuracy, of a regular transformer right?
- lennxa 2y agothey mention similar performance to vanilla transformer with significantly reduced param count though
- dartos 2y ago> Hallucination and creativity feel very related to me. Why? I see them as just sampling errors. Sure a mistake can spark inspiration sometimes, but creativity is much more than mistakes. > I understand hallucinating as simply being misaligned with the space humans feel appropriate to interpolate between These language models are next-token predictors. The way the next token is predicted is by sampling a probability space outputted by the model. That sampling process can be non deterministic. Hallucinations are when that sampling results in tokens that come together to create a false or otherwise unintended statement. You can just as well think of everything a model outputs as a hallucination, but we train the model to output a space what we want them to hallucinate is more likely. Otherwise it just outputs meaningless noise. “Hallucinate” is really an awful word for what it’s trying to describe.
- pxdm 2y agoWhat's the comparison with conventional attention using a more aggressive (lower temperature) softmax? I can imagine that for the multi-needle retrieval test this may also give a performance boost, although at some cost other more creative tasks.
- mota7 2y agoI had the same thought: Just eye-balling the graphs, the result of the subtraction looks very close to just reducing the temperature. They're effectively doing softmax with a fixed temperature, but it's unclear that this work is going to do better than just learning a per-head temperature parameter. c.f. https://arxiv.org/abs/2010.04245 https://arxiv.org/abs/2010.04245 which shows an improvement by learning per-head temperature. The other way to think about this is that it looks like a hacked-up kinda-sorta gated attention. If that's the case, then doing softmax(alphaq_1k_1^T - log_sigmoid(betaq_2k_2^T)) might be better? (where alpha,beta are learned temperatures).
- singularity2001 2y agoAnyone remember siamese networks?
- nmacias 2y agoAdderaLLM was right there
- vsroy 2y agoIs the thing that's going on here that softmax can't push a value to 0, but by subtracting 2 softmax maps we can output 0s?
- pizza 2y agoWas just going to mention that it seems that it should be possible to make a Flash Attention version of this algorithm and was pleasantly surprised to see they already included an implementation of one :)
- iandanforth 2y agoThe key bit I didn't understand at first was what happens if the two groups of attention learn the same thing; because their attention masks are subtracted from one another if they both output similar values the attention across the board will drop to zero and this will lead to high loss. So the only way to reduce loss is if they learn to attend to different things. One of the simplest strategies they could learn (and this paper claims that they do) is for one group to focus on relevant context and the other to focus on irrelevant context. Thus one group learns the noise and the other the signal (it's not this cut and dry but is a useful simplification for understanding IMO).
- dartos 2y agoThere’s probably a small chance that they could both learn the same thing, but it’s probably not likely enough to be a major issue.
- magicalhippo 2y agoAn interesting aspect is that they don't do a plain subtraction, but rather subtract a portion of the second softmax. This makes sense, if one considers that the two copies are identical then the softmax outputs would be identical and the difference is zero everywhere. However, by subtracting a scaled copy, the normalization of the difference seems to really boost the signal value(s) over the "noise", making the signal stand out compared to pre-normalization.
- testdfkjahdfh 2y agoif two attentions A, B are identical, would (A - lambda * B) be just (1-lambda) * A, how does it "boost the signal value(s) over the "noise""?
- magicalhippo 2y agoHow embarrassing, I had one of those "autocorrect moments". I somehow put the lambda inside the softmax when thinking and trying it without realizing. So what I was playing with in a spreadsheet (so not so obvious as plain code) was softmax(A) - softmax(lambda * A) And as so happens, normalizing the output of that that with my test vectors seems to really boost the output the largest component if A and B are equal.
- dartos 2y ago> By being less distracted by irrelevant context, Diff Transformer can mitigate hallucination in question answering and text summarization I’m very interested in this claim. I was under the impression that hallucination is unavoidable in these kinds of models. IIRC proof for that was trending on HN a couple weeks ago.
- moffkalast 2y agoIt's not possible to get rid of it entirely, but if you can get the model to bullshit only 0.1% of the time instead of 5% of the time it's a massive improvement. Most of it should be happening when there's no data to draw conclusions from. E.g. STT models make up words in silence, vision models find things in lens cap noise, LLMs make up explanations when they have no data to pull from. The real solution would be more along the lines of training models to specifically ignore these cases, or in the case of LLMs to just know when to say "I don't know".
- ErikBjare 2y agoMitigate, not completely fix.
- pshc 2y agoMore broadly I think hallucination is inevitable in pure text models. We need model architectures incorporating a stream of real-world ground truth such as a live video feed or embodiment.
- nowayno583 2y agoDoes anyone understand why they are taking the difference between transformers instead of the sum? It seems to me that in a noise reducing solution we would be more interested in the sum, as random noise would cancel out and signal would be constructive. Of course, even if I'm right proper training would account to that by inverting signs where appropriate. Still, it seems weird to present it as the difference, especially seeing as they compare this directly to noise cancelling headphones, where we sum both microphones inputs.
- aDyslecticCrow 2y agoThe noise isn't truly random; it's just a matrix of small values that shouldn't be taken into account. Subtracting them cancels them out. As pointed out by a different comment, it's actually the attention we are interested in that is cancelled out *if they are both equal*. This is what the paper mentions in its abstract; > promoting the emergence of sparse attention patterns In theory, it is quite clever, and their results seem to back it up.
- watsonmusic 2y ago[dead]
- thegeomaster 2y agoI suspect that plus vs minus is arbitrary in this case (as you said, due to being able to learn a simple negation during training), but they are presenting it in this way because it is more intuitive. Indeed, adding two sources that are noisy in the same way just doubles the noise, whereas subtracting cancels it out. It's how balanced audio cables work, for example. But with noise cancelling headphones, we don't sum anything directly---we emit an inverted sound, and to the human ear, this sounds like a subtraction of the two signals. (Audio from the audio source, and noise from the microphone.)
- nowayno583 2y agoOh! It's been a good while since I've worked in noise cancelling. I didn't know current tech was at the point where we could do direct reproduction of the outside noise, instead of just using mic arrays! That's very cool, it used to be considered totally sci fi to do it fast enough in a small headset.
- aDyslecticCrow 2y agoVery clever. I like this kind of nitty-gritty detail work, and the change is small enough to be adapted easily by others. Bravo! I'm a little concerned about the last sentence of the section introduction of "2 Differential Transformer". It mentions using improvements from previous papers, but in the grammatical context, it's unclear if this improvement is added to both the normal transformer and their diff transformer. This would otherwise sully the comparisons. It's the "main difference" wording in the previous sentence that raised a flag for me. Of course, a good-faith researcher would know this and may not feel the need to clarify. But you can never be too careful about some published research in this field.
- Chirono 2y agoThe two other changes they mention have been widely adopted, and are included in at least some of the models they benchmark against. It seems they list them for completeness as changes to the original transformer architecture.
- aDyslecticCrow 2y agoNicely spotted! Then, I really look forward to seeing this method tested by others! Epic stuff.
- vessenes 2y agoYes. This looks really, really good to me. Cross the board improvements in training time, perplexity improvements per both token trained and per model size. I'm reminded of MoE architectures, in that world we're choosing an optimal small model to process part or all of the inference job; I wonder if MoE got some of the same benefits from forcing the Transformer to distinguish between alternate possibilities. In any event, I'd imagine that this will get widely adopted if the numbers hold up; like I said, this seems to be basically no downside, and should be easy to replicate.
- f38zf5vdt 2y agoThere is a downside, every attention layer has to effectively compute attention twice (run scaled_dot_product_attention). As scaled_dot_product_attention is usually one of the most expensive operations in training and inference of a model, it seems like networks using this may be substantially slow and perhaps should considered against larger networks with more attention layers. https://github.com/microsoft/unilm/blob/master/Diff-Transformer/multihead_flashdiff_1.py#L101-L102 https://github.com/microsoft/unilm/blob/master/Diff-Transfor...
- lucidrains 2y agodoes this not mean we should explore usage of talking heads (Shazeer et al) a bit more? https://arxiv.org/abs/2003.02436 https://arxiv.org/abs/2003.02436
- watsonmusic 2y ago[dead]
- x49asvk 2y agoThis concept is really interesting to me, I am very very new to transformers but would love to learn more about normal transformers and differential too. Can anyone suggest any resources?
- WithinReason 2y agoWe empirically find that the setting λᵢₙᵢₜ = 0.8 − 0.6 × exp(−0.3 · (l − 1)) works well in practice I wonder about the story behind that formula...
- Kubuxu 2y agoHmm, 0.8 works well, but let's try setting lower layers to lower initial value. Let's say 0.2. Ok, I need a formula that will go between 0.2 and 0.8, slowly approaching 0.8. Starts fiddling with numbers for 20min, I guess this can work.
- godelski 2y agoSure, but in research show some comparisons
- physicsguy 2y agoIn practice there's always a trade off between getting some result out and published and rigorously exploring every avenue of optimisation in research. Sometimes you have to say 'this is good enough and long enough already'.
- godelski 2y agoRight, but it has to be *good enough*. My issue isn't that they didn't do more work, my issue is that they didn't even report work that they did do that communicates the impact of the literal proposed method. https://news.ycombinator.com/item?id=41783013 https://news.ycombinator.com/item?id=41783013
- kridsdale3 2y agoA whole lot of things are tuned optimally by rotating an analog dial until things look / sound right.
- stellalo 2y agoLooks like this makes (at least initially in training) the “negative” attention term smaller in the early layers (smaller l) compared to later layers (larger l). Which I guess makes sense: you probably want to attend a little bit to everything before concluding that it’s really a few spots you should look at. (Although it seems the author do not discuss this choice anywhere in the paper?)
- WithinReason 2y agoHmmm, this could be expressed as 2 consecutive attentions in a residual branch: Simplified differential T. looks like: (softmax(Q₁K₁) − λ softmax(Q₂K₂)) V You can factor this into: x = softmax(Q₁K₁)V x += -λ softmax(Q₂K₂)V which is like 2 subsequent regular attentions added that are sharing V
- kelseyfrog 2y agoYou could also extrapolate this into more than two terms by squinting your eyes and saying that λ ∈ {1, -1} is close enough to λi ∈R^d ∣ ∥λi ∥=1. No idea if it would result in better performance, but that's research babyyyy!
- tananan 2y agoNow I'm wondering, isn't there usually a `num_heads x value_dim -> model_dim` projection that goes after a MHA? The W in `softmax(QK)VW`? That one can play the role of this subtraction in a vanilla transformer, no? So I wonder what kind of advantage does splitting things up like this bring.
- miven 2y agoIs there an intuitive reason why this ends up working this well compared to, say, applying some kind of thresholding to attention activations that are below average for a given head to filter that same attention noise out?
- islewis 2y ago> Differential attention takes the difference between two softmax attention functions to eliminate attention noise If I understand correctly, this architecture trades twice as much attention memory in exchange for either a higher quality model, or less parameters at a similar quality. > According to the fitted curves, 6.8B-size DIFF Transformer achieves a validation loss comparable to 11B-size Transformer, requiring only 62.2% of parameters This raises a few questions for me: - Would having only 60% of the parameters negate the double space for attention, leaving a similar memory profile as a traditional transformer? - Does that tradeoff change noticeably between training and inference?
- entropicdrifter 2y agoI think it would negate the RAM savings, but it would also reduce the amount of storage needed at rest and possibly reduce initial start up times depending on storage speed and model size. So, possibly good for low-end models on consumer devices?
- _hl_ 2y agoMy understanding was that the extra parameters required for the second attention mechanism are included in those 6.8B parameters (i.e. those are the total parameters of the model, not some made-up metric of would-be parameter count in a standard transformer). This makes the result doubly impressive! Here's the bit from the paper: > We set the number of heads h = dmodel/2d, where d is equal to the head dimension of Transformer. So we can align the parameter counts and computational complexity. In other words, they make up for it by having only half as many attention heads per layer.
- chessgecko 2y agoI think they mitigated the extra memory/compute from this by using half the number of overall heads and doubling V and O. Without actually checking the math I think it should be equivalent in flops, not counting the extra (cheap) multiply by const and subtract.
- Kubuxu 2y agoIt would double the size of the KV cache, which can be significant (multi-GB) at larger context sizes.
- Imnimo 2y agoI feel like I'm missing a key insight here. I understand the problem that regular softmax attention struggles to approach assigning zero attention to irrelevant stuff. And I get that having this subtraction formula makes it possible to assign exactly (or near) zero attention weight without having crazy outlier activations. But it seems like it also makes it very easy to have negative attention weight (which is equivalent to having positive attention weight on the negation of your value vectors). Intuitively, it just feels like a difficult balancing act to keep all the stuff you don't care about so close to zero. But Figure 1 clearly shows that it works, so I don't doubt that it is in fact possible. I'm just struggling to build a picture of how exactly the network accomplishes this.
- watsonmusic 2y agonegative values can enhance the expressibility
- Jerrrrrrry 2y agodoubt is the seed of reason
- deleted 2y ago[deleted]
- Grosvenor 2y agoRegular softmax (and attention) has an error in it. softmax should be exp()/1+∑exp() Notice the 1 added to the denominator. The difference is at the negative limit, softmax can be 0, instead of some epsilon. The same could be done by adding an extra zero value in x. Downside is, you have to retrain your model from scratch to fix this.
- impossiblefork 2y agoI've tried that in a small transformer that I trained from scratch and it didn't really make any difference. I also made a version where I made this trainable somehow, probably by replacing the 1 with a constant associated with the layer, and that didn't make any difference either. I didn't follow Miller's proposal quite as he wrote it though and I put the mechanism in all the layers rather than avoiding it at the end. My test doesn't absolutely rule out usefulness-- there's always different ways of applying something, but I saw no indication of it.
- machinelearning 2y agoThis is a good problem to solve but the approach is wrong imo. It has to be done in a hierarchical way to know what you attended to + full context. If the differential vector is being computed with the same input as the attention vector how do you know how to modify the attention vector correctly
- quantadev 2y agoDoesn't everything just get tweaked in whatever direction the back-propagation derivative says and proportionally to that "slope"? In other words, simply by having back-propagation system in effect there's never any question about which way to adjust the weights, right?
- slashdave 2y agoI don't get it. Arbitrary linear combinations are already accommodated via feed forward. What am I missing?
- michalsustr 2y agoMy hunch is that this effectively creates a differentiable minimax “search” “tree” that can be backpropagated through. Not a tree — a dag really — and not search, but learning. :)
- thatsadude 2y agoTrue, that's a yellow flag for me.
- chessgecko 2y agoI wonder how much of the value here is from canceling out the positional noise rope produces. I would love to see a table comparing an alibi version of this to an alibi baseline in addition to the rope models here. Crazy gains though congrats to the researchers
- esafak 2y agoHow is this different than using a sparsity-inducing prior?
- badsandwitch 2y agoWhat is purpose of the lambda parameter? Why isn't it a constant of 1?
- h_tbob 2y agoI wish they didn’t use swiGLU and preRMSnorm so we could have a better comparison. Then we would know how much this transformer innovation helps by itself.
- mik09 2y agor/machine learning comment thread has some interesting ideas, one of them linking this one with similar work in CV: https://www.reddit.com/r/MachineLearning/comments/1g0lnij/r_ngpt_normalized_transformer_with_representation/ https://www.reddit.com/r/MachineLearning/comments/1g0lnij/r_...