8 ms·

I'm an industrial electrician. A lot of poor electrical work is visible only to a fellow electrician, and sometimes only another industrial electrician. Bad tec

I'm an industrial electrician. A lot of poor electrical work is visible only to a fellow electrician, and sometimes only another industrial electrician. Bad technical work requires technical inspectors to criticize. Sometimes highly skilled ones.

andy99 9mo ago
I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. They are also assuming good faith.
- zdragnar 9mo ago
  This is half the basis for the replication crisis, no? Shady papers come out and people cite them endlessly with no critical thought or verification. After all, their grant covers their thesis, not their thesis plus all of the theses they cite.
- Aurornis 9mo ago
  > I don’t consider it the reviewers responsibility to manually verify all citations are real I guess this explains all those times over the years where I follow a citation from a paper and discover it doesn’t support what the first paper claimed.
- auggierose 9mo ago
  In short, a review has no objective value, it is just an obstacle to be gamed.
  - amanaplanacanal 9mo ago
    In theory, the review tries to determine if the conclusion reached actually follows from whatever data is provided. It assumes that everything is honest, it's just looking to see if there were mistakes made.
    - auggierose 9mo ago
      Honest or not should not make a difference, after all, the submitting author may believe themselves everything is A-OK. The review should also determine how valuable the contribution is, not only if it has mistakes or not. Todays reviews determine neither value nor correctness in any meaningful way. And how could they, actually? That is why I review papers only to the extent that I understand them, and I clearly delineate my line of understanding. And I don't review papers that I am not interested in reading. I once got a paper to review that actually pointed out a mistake in one of my previous papers, and then proposed a different solution. They correctly identified the mistake, but I could not verify if their solution worked or not, that would have taken me several weeks to understand. I gave a report along these lines, and the person who gave me the review said I should say more about their solution, but I could not. So my review was not actually used. The paper was accepted, which is fine, but I am sure none of the other reviewers actually knows if it is correct. Now, this was a case where I was an absolute expert. Which is far from the usual situation for a reviewer, even though many reviewers give themselves the highest mark for expertise when they just should not.
- pbhjpbhj 9mo ago
  Surely there are tools to retrieve all the citations, publishers should spot it easily. However the paper is submitted, like a folder on a cloud drive, just have them include a folder with PDFs/abstracts of all the citations? They might then fraudulently produce papers to cite, but they can't cite something that doesn't exist.
  - tpoacher 9mo ago
    how delightfully optimistic of you to think those abstracts would not also be ai generated ...
    - zzzeek 9mo ago
      sure but then the citations are no longer "hallucinated", they actually point to something fraudulent. that's a different problem.
  - michaelt 9mo ago
    > Surely there are tools to retrieve all the citations, Even if you could retrieve all citations (which isn't always as easy as you might hope) to validate citations you'd also have to confirm the paper says what the person citing it says. If I say "A GPU requires 1.4kg of copper" citing [1] is that a valid citation? That means not just reviewing one paper, but also potentially checking 70+ papers it cites. The vast majority of paper reviewers will not check citations actually say what they're claimed to say, unless a truly outlandish claim is made. At the same time, academia is strangely resistant to putting hyperlinks in citations, preferring to maintain old traditions - like citing conference papers by page number in a hypothetical book that has never been published; and having both a free and a paywalled version of a paper while considering the paywalled version the 'official' version. [1] https://arxiv.org/pdf/2512.04142 https://arxiv.org/pdf/2512.04142
- grayhatter 9mo ago
  > The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? > They are also assuming good faith. I can only relate this to code review, but assuming good faith means you assume they didn't try to introduce a bug by adding this dependency. But I would should still check to make sure this new dep isn't some typosquatted package. That's the rigor I'm responsible for.
  - chroma205 9mo ago
    > I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? No. Modern peer review is “how can I do minimum possible work so I can write ‘ICLR Reviewer 2025’ on my personal website”
    - grayhatter 9mo ago
      > No. [...] how can I do minimum possible work I don't know, I still think this describes most of the reviews I've seen I just hope most devs that do this know better than to admit to it.
    - freehorse 9mo ago
      The vast majority of people I see do not even mention who they review for in CVs etc. It is usually more akin to a volunteer based, thankless work. Unless you are an editor or sth in a journal, what you review for does not count much for anything.
  - tpoacher 9mo ago
    This is true, but here the equivalent situation is someone using a greek question mark (";") instead of a semicolon (";"), and you as a code reviewer are only expected to review the code visually and are not provided the resources required to compile the code on your local machine to see the compiler fail. Yes in theory you can go through every semicolon to check if it's not actually a greek question mark; but one assumes good faith and baseline competence such that you as the reviewer would generally not be expected to perform such pedantic checks. So if you think you might have reasonably missed greek question marks in a visual code review, then hopefully you can also appreciate how a paper reviewer might miss a false citation.
- stdbrouw 9mo ago
  The idea that references in a scientific paper should be plentiful but aren't really that important, is a consequence of a previous technological revolution: the internet. You'll find a lot of papers from, say, the '70s, with a grand total of maybe 10 references, all of them to crucial prior work, and if those references don't say what the author claims they should say (e.g. that the particular method that is employed is valid), then chances are that the current paper is weaker than it seems, or even invalid, and so it is extremely important to check those references. Then the internet came along, scientists started padding their work with easily found but barely relevant references and journal editors started requiring that even "the earth is round" should be well-referenced. The result is that peer reviewers feel that asking them to check the references is akin to asking them to do a spell check. Fair enough, I agree, I usually can't be bothered to do many or any citation checks when I am asked to do peer review, but it's good to remember that this in itself is an indication of a perverted system, which we just all ignored -- at our peril -- until LLM hallucinations upset the status quo.
  - tialaramex 9mo ago
    Whether in the 1970s or now, it's too often the case that a paper says "Foo and Bar are X" and cites two sources for this fact. You chase down the sources, the first one says "We weren't able to determine whether Foo is X" and never mentions Bar. The second says "Assuming Bar is X, we show that Foo is probably X too". The paper author likely believes Foo and Bar are X, it may well be that all their co-workers, if asked, would say that Foo and Bar are X, but "Everybody I have coffee with agrees" can't be cited, so we get this sort of junk citation. Hopefully it's not crucial to the new work that Foo and Bar are in fact X. But that's not always the case, and it's a problem that years later somebody else will cite this paper, for the claim "Foo and Bar are X" which it was in fact merely citing erroneously.
    - KHRZ 9mo ago
      LLMs can actually make up for their negative contributions. They could go through all the references of all papers and verify them, assuming someone would also look into what gets flagged for that final seal of disapproval. But this would be more powerfull with an open knowledge base where all papers and citation verifications were registered, so that all the effort put into verification could be reused, and errors propagated through the citation chain.
- andai 9mo ago
  >I don’t consider it the reviewers responsibility to manually verify all citations are real. Doesn't this sound like something that could be automated? for paper_name in citations... do a web search for it, see if it there's a page in the results with that title. That would at least give you "a paper with this name exists".
- PeterStuer 9mo ago
  I think the root problem is that everyone involved, from authors to reviewers to publishers, know that 99.999% of papers are completely of no consequence, just empty calories with the sole purpose of padding quotas for all involved, and thus are not going to put in the effort as if. This is systemic, and unlikely to change anytime soon. There have been remedies proposed (e.g. limits on how many papers an author can publish per year, let's say 4 to be generous), but they are unlikely to gain traction as thoug most would agree onbenefits, all involved in the system would stand to lose short term.
- zzzeek 9mo ago
  correct me if I'm wrong but citations in papers follow a specific format, and the case here is that a tool was used to validate that they are all real. Certainly a tool that scans a paper for all citations and verifies that they actually exist in the journals they reference shouldn't be all that technically difficult to achieve?
  - alexcdot 9mo ago
    There are a ton of edge cases and a bit of contextual understanding for what is a hallucinated citation (i.e. what if its republished from arxiv to ICLR?) But to your point, seems we need a tool that can do this
  - mike_hearn 9mo ago
    It's not, there's lots of ways to resolve citations without even using AI. I experimented a couple of years ago with getting LLMs to check citations but stopped working on it because there's no incentive. You could run a fancy expensive pipeline burning scarce GPU hours and find a bunch of bad citations. Then what? Nobody cares. No journal is going to retract any of these papers, the academics themselves won't care or even respond to your emails, nobody is willing to pay for this stuff, least of all the universities, journals or governments themselves. For example, there's a guy in France who runs a pre-LLM pipeline to discover bad papers using hand-coded heuristics like regexs or metadata analysis e.g. checking if a citation has been retracted. Many of the things it detects are plagiarism, paper mills (i.e. companies that sell fake papers to academics for a profit), or the result of joke paper creators like SciGen. https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: Other than populating an obscure database nobody knows about, this work achieved bupkis.
- figassis 9mo ago
  It is absolutely the reviewers job to check citations. Who else will check and what is the point of peer review then? So you’d just happily pass on shoddy work because it’s not your job? You’re reviewing both the authors work and if there were people to at needed to ensure citations were good, you’re checking their work also. This is very much the problem today with this “not my problem” mindset. If it passes review, the reviewer is also at fault. Not excuses.
  - dpkirchner 9mo ago
    Agreed, and I'd go further. If nobody is reviewing citations they may as well not exist. Why bother?
    - vkou 9mo ago
      1. To make it clear what is your work, and what is building on someone else's. 2. If the paper turns out to be important, people will bother. 3. There's checking for cursory correctness, and there's forensic torture.
      - figassis 9mo ago
        building on imaginary someone else? That's exactly the same as lying. Is a review not about verifying that the paper and even data is correct? I get reviewers can make mistakes, but this seems like defending intentional mistakes. I mean, in college I have had to review papers, and so took peer review lectures, and nowhere in there was it ever stated that citations are not the reviewer's job. In fact, citation verification was one to the most important parts of the lectures, as in, how to find original sources (when authoring), and how to verify them (when reviewing). When did peer review get redefined?
        vkou 9mo ago
        I'm not defending dishonesty, I'm saying that's what citations do when they are used by honest people.
  - 9mo ago
- jayess 9mo ago
  Wow. I went to law school and was on the law review. That was our precise job for the papers selected for publication. To verify every single citation.
  - _blk 9mo ago
    Thanks for sharing that. Interesting how there was a solution to a problem that didn't really exist yet.. I mean, I'm sure it was there for a reason, but I assume it was more things like wrongful attribution, missing commas etc. rather than outright invented quotes to fit a narrative or do you have more background on that? ...at least the mandatory automated checking processes are probably not far off at least for the more reputable journals, but it still makes you wonder how much you can trust the last two years of LLM-enhanced science that is now being quoted in current publications and if those hallucinations can be "reverted" after having been re-quoted. A bit like Wikipedia can be abused to establish facts.
- not2b 9mo ago
  Agreed. I used to review lots of submissions for IEEE and similar conferences, and didn't consider it my job to verify every reference. No one did, unless the use of the reference triggered an "I can't believe it said that" reaction. Of course, back then, there wasn't a giant plagiarism machine known to fabricate references, so if tools can find fake references easily the tools should be used.
- armcat 9mo ago
  I agree with you (I have reviewed papers in the past), however, made-up citations are a "signal". Why would the authors do that? If they made it up, most likely they haven't really read that prior work. If they haven't, have they really done proper due dilligence on their research? Are they just trying to "beef up" their paper with citations to unfairly build up credibility?
- rokob 9mo ago
  As a reviewer I at least skimmed the papers for every reference in every paper that I review. If it isn't useful to furthering the point of the paper then my feedback is to remove the reference. Adding a bunch of junk because it is broadly related in a giant background section is a waste of everyone's time and should be removed. Most of the time you are mostly aware of the papers being cited anyway because that is the whole point of reviewing in your area of expertise.
bdangubic 9mo ago
same (and much, much, much worse) for science
barfoure 9mo ago
I’d love to hear some examples of poor electrical work that you’ve come across that’s often missed or not seen.
- joshribakoff 9mo ago
  I am not an electrician, but when I did projects, I did a lot of research before deciding to hire someone and then I was extremely confused when everyone was proposing doing it slightly differently. A lot of them proposed ways that seem to violate the code, like running flex tubing beyond the allowed length or amount of turns. Another example would be people not accounting for needing fireproof covers if they’re installing recessed, lighting in between dwelling in certain cities… Heck, most people don’t actually even get the permit. They just do the unpermitted work.
- AstroNutt 9mo ago
  A couple had just moved in a house and called me to replace the ceiling fan in the living room. I pulled the flush mount cover down to start unhooking the wire nuts and noticed RG58 (coax cable). Someone had used the center conductor as the hot wire! I ended up running 12/2 Romex from the switch. There was no way in hell I could have hooked it back up the way it was. This is just one example I've come across.
lencastre 9mo ago
an old boss of mine used to say there are no stupid electricians found alive, as they self select darwin award style
xnx 9mo ago
No doubt the best electricians are currently better than the best AI, but the best AI is likely now better than the novice homeowner. The trajectory over the past 2 years has been very good. Another five years and AI may be better than all but the very best, or most specialized, electricians.
- legostormtroopr 9mo ago
  Current state AI doesn’t have hands. How can it possibly be better at installing electrics than anyone? Your post reads like AI precisely because while the grammar is fine, it lacks context - like someone prompted “reply that AI is better than average”.
  - xnx 9mo ago
    An electrician with total knowledge/understanding, but only the average dexterity of a non-professional would still be very useful.

andy99 9mo ago

I’ve reviewed a lot of papers, I don’t consider it the reviewers responsibility to manually verify all citations are real. If there was an unusual citation that was relied on heavily for the basis of the work, one would expect it to be checked. Things like broad prior work, you’d just assume it’s part of background. The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. They are also assuming good faith.

zdragnar 9mo ago
This is half the basis for the replication crisis, no? Shady papers come out and people cite them endlessly with no critical thought or verification. After all, their grant covers their thesis, not their thesis plus all of the theses they cite.
Aurornis 9mo ago
> I don’t consider it the reviewers responsibility to manually verify all citations are real I guess this explains all those times over the years where I follow a citation from a paper and discover it doesn’t support what the first paper claimed.
auggierose 9mo ago
In short, a review has no objective value, it is just an obstacle to be gamed.
- amanaplanacanal 9mo ago
  In theory, the review tries to determine if the conclusion reached actually follows from whatever data is provided. It assumes that everything is honest, it's just looking to see if there were mistakes made.
  - auggierose 9mo ago
    Honest or not should not make a difference, after all, the submitting author may believe themselves everything is A-OK. The review should also determine how valuable the contribution is, not only if it has mistakes or not. Todays reviews determine neither value nor correctness in any meaningful way. And how could they, actually? That is why I review papers only to the extent that I understand them, and I clearly delineate my line of understanding. And I don't review papers that I am not interested in reading. I once got a paper to review that actually pointed out a mistake in one of my previous papers, and then proposed a different solution. They correctly identified the mistake, but I could not verify if their solution worked or not, that would have taken me several weeks to understand. I gave a report along these lines, and the person who gave me the review said I should say more about their solution, but I could not. So my review was not actually used. The paper was accepted, which is fine, but I am sure none of the other reviewers actually knows if it is correct. Now, this was a case where I was an absolute expert. Which is far from the usual situation for a reviewer, even though many reviewers give themselves the highest mark for expertise when they just should not.
pbhjpbhj 9mo ago
Surely there are tools to retrieve all the citations, publishers should spot it easily. However the paper is submitted, like a folder on a cloud drive, just have them include a folder with PDFs/abstracts of all the citations? They might then fraudulently produce papers to cite, but they can't cite something that doesn't exist.
- tpoacher 9mo ago
  how delightfully optimistic of you to think those abstracts would not also be ai generated ...
  - zzzeek 9mo ago
    sure but then the citations are no longer "hallucinated", they actually point to something fraudulent. that's a different problem.
- michaelt 9mo ago
  > Surely there are tools to retrieve all the citations, Even if you could retrieve all citations (which isn't always as easy as you might hope) to validate citations you'd also have to confirm the paper says what the person citing it says. If I say "A GPU requires 1.4kg of copper" citing [1] is that a valid citation? That means not just reviewing one paper, but also potentially checking 70+ papers it cites. The vast majority of paper reviewers will not check citations actually say what they're claimed to say, unless a truly outlandish claim is made. At the same time, academia is strangely resistant to putting hyperlinks in citations, preferring to maintain old traditions - like citing conference papers by page number in a hypothetical book that has never been published; and having both a free and a paywalled version of a paper while considering the paywalled version the 'official' version. [1] https://arxiv.org/pdf/2512.04142 https://arxiv.org/pdf/2512.04142
grayhatter 9mo ago
> The reviewer is not a proofreader, they are checking the rigour and relevance of the work, which does not rest heavily on all of the references in a document. I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? > They are also assuming good faith. I can only relate this to code review, but assuming good faith means you assume they didn't try to introduce a bug by adding this dependency. But I would should still check to make sure this new dep isn't some typosquatted package. That's the rigor I'm responsible for.
- chroma205 9mo ago
  > I've always assumed peer review is similar to diff review. Where I'm willing to sign my name onto the work of others. If I approve a diff/pr and it takes down prod. It's just as much my fault, no? No. Modern peer review is “how can I do minimum possible work so I can write ‘ICLR Reviewer 2025’ on my personal website”
  - grayhatter 9mo ago
    > No. [...] how can I do minimum possible work I don't know, I still think this describes most of the reviews I've seen I just hope most devs that do this know better than to admit to it.
  - freehorse 9mo ago
    The vast majority of people I see do not even mention who they review for in CVs etc. It is usually more akin to a volunteer based, thankless work. Unless you are an editor or sth in a journal, what you review for does not count much for anything.
- tpoacher 9mo ago
  This is true, but here the equivalent situation is someone using a greek question mark (";") instead of a semicolon (";"), and you as a code reviewer are only expected to review the code visually and are not provided the resources required to compile the code on your local machine to see the compiler fail. Yes in theory you can go through every semicolon to check if it's not actually a greek question mark; but one assumes good faith and baseline competence such that you as the reviewer would generally not be expected to perform such pedantic checks. So if you think you might have reasonably missed greek question marks in a visual code review, then hopefully you can also appreciate how a paper reviewer might miss a false citation.
stdbrouw 9mo ago
The idea that references in a scientific paper should be plentiful but aren't really that important, is a consequence of a previous technological revolution: the internet. You'll find a lot of papers from, say, the '70s, with a grand total of maybe 10 references, all of them to crucial prior work, and if those references don't say what the author claims they should say (e.g. that the particular method that is employed is valid), then chances are that the current paper is weaker than it seems, or even invalid, and so it is extremely important to check those references. Then the internet came along, scientists started padding their work with easily found but barely relevant references and journal editors started requiring that even "the earth is round" should be well-referenced. The result is that peer reviewers feel that asking them to check the references is akin to asking them to do a spell check. Fair enough, I agree, I usually can't be bothered to do many or any citation checks when I am asked to do peer review, but it's good to remember that this in itself is an indication of a perverted system, which we just all ignored -- at our peril -- until LLM hallucinations upset the status quo.
- tialaramex 9mo ago
  Whether in the 1970s or now, it's too often the case that a paper says "Foo and Bar are X" and cites two sources for this fact. You chase down the sources, the first one says "We weren't able to determine whether Foo is X" and never mentions Bar. The second says "Assuming Bar is X, we show that Foo is probably X too". The paper author likely believes Foo and Bar are X, it may well be that all their co-workers, if asked, would say that Foo and Bar are X, but "Everybody I have coffee with agrees" can't be cited, so we get this sort of junk citation. Hopefully it's not crucial to the new work that Foo and Bar are in fact X. But that's not always the case, and it's a problem that years later somebody else will cite this paper, for the claim "Foo and Bar are X" which it was in fact merely citing erroneously.
  - KHRZ 9mo ago
    LLMs can actually make up for their negative contributions. They could go through all the references of all papers and verify them, assuming someone would also look into what gets flagged for that final seal of disapproval. But this would be more powerfull with an open knowledge base where all papers and citation verifications were registered, so that all the effort put into verification could be reused, and errors propagated through the citation chain.
andai 9mo ago
>I don’t consider it the reviewers responsibility to manually verify all citations are real. Doesn't this sound like something that could be automated? for paper_name in citations... do a web search for it, see if it there's a page in the results with that title. That would at least give you "a paper with this name exists".
PeterStuer 9mo ago
I think the root problem is that everyone involved, from authors to reviewers to publishers, know that 99.999% of papers are completely of no consequence, just empty calories with the sole purpose of padding quotas for all involved, and thus are not going to put in the effort as if. This is systemic, and unlikely to change anytime soon. There have been remedies proposed (e.g. limits on how many papers an author can publish per year, let's say 4 to be generous), but they are unlikely to gain traction as thoug most would agree onbenefits, all involved in the system would stand to lose short term.
zzzeek 9mo ago
correct me if I'm wrong but citations in papers follow a specific format, and the case here is that a tool was used to validate that they are all real. Certainly a tool that scans a paper for all citations and verifies that they actually exist in the journals they reference shouldn't be all that technically difficult to achieve?
- alexcdot 9mo ago
  There are a ton of edge cases and a bit of contextual understanding for what is a hallucinated citation (i.e. what if its republished from arxiv to ICLR?) But to your point, seems we need a tool that can do this
- mike_hearn 9mo ago
  It's not, there's lots of ways to resolve citations without even using AI. I experimented a couple of years ago with getting LLMs to check citations but stopped working on it because there's no incentive. You could run a fancy expensive pipeline burning scarce GPU hours and find a bunch of bad citations. Then what? Nobody cares. No journal is going to retract any of these papers, the academics themselves won't care or even respond to your emails, nobody is willing to pay for this stuff, least of all the universities, journals or governments themselves. For example, there's a guy in France who runs a pre-LLM pipeline to discover bad papers using hand-coded heuristics like regexs or metadata analysis e.g. checking if a citation has been retracted. Many of the things it detects are plagiarism, paper mills (i.e. companies that sell fake papers to academics for a profit), or the result of joke paper creators like SciGen. https://dbrech.irit.fr/pls/apex/f?p=9999:1 https://dbrech.irit.fr/pls/apex/f?p=9999:1:::::: Other than populating an obscure database nobody knows about, this work achieved bupkis.
figassis 9mo ago
It is absolutely the reviewers job to check citations. Who else will check and what is the point of peer review then? So you’d just happily pass on shoddy work because it’s not your job? You’re reviewing both the authors work and if there were people to at needed to ensure citations were good, you’re checking their work also. This is very much the problem today with this “not my problem” mindset. If it passes review, the reviewer is also at fault. Not excuses.
- dpkirchner 9mo ago
  Agreed, and I'd go further. If nobody is reviewing citations they may as well not exist. Why bother?
  - vkou 9mo ago
    1. To make it clear what is your work, and what is building on someone else's. 2. If the paper turns out to be important, people will bother. 3. There's checking for cursory correctness, and there's forensic torture.
    - figassis 9mo ago
      building on imaginary someone else? That's exactly the same as lying. Is a review not about verifying that the paper and even data is correct? I get reviewers can make mistakes, but this seems like defending intentional mistakes. I mean, in college I have had to review papers, and so took peer review lectures, and nowhere in there was it ever stated that citations are not the reviewer's job. In fact, citation verification was one to the most important parts of the lectures, as in, how to find original sources (when authoring), and how to verify them (when reviewing). When did peer review get redefined?
      - vkou 9mo ago
        I'm not defending dishonesty, I'm saying that's what citations do when they are used by honest people.
- 9mo ago
jayess 9mo ago
Wow. I went to law school and was on the law review. That was our precise job for the papers selected for publication. To verify every single citation.
- _blk 9mo ago
  Thanks for sharing that. Interesting how there was a solution to a problem that didn't really exist yet.. I mean, I'm sure it was there for a reason, but I assume it was more things like wrongful attribution, missing commas etc. rather than outright invented quotes to fit a narrative or do you have more background on that? ...at least the mandatory automated checking processes are probably not far off at least for the more reputable journals, but it still makes you wonder how much you can trust the last two years of LLM-enhanced science that is now being quoted in current publications and if those hallucinations can be "reverted" after having been re-quoted. A bit like Wikipedia can be abused to establish facts.
not2b 9mo ago
Agreed. I used to review lots of submissions for IEEE and similar conferences, and didn't consider it my job to verify every reference. No one did, unless the use of the reference triggered an "I can't believe it said that" reaction. Of course, back then, there wasn't a giant plagiarism machine known to fabricate references, so if tools can find fake references easily the tools should be used.
armcat 9mo ago
I agree with you (I have reviewed papers in the past), however, made-up citations are a "signal". Why would the authors do that? If they made it up, most likely they haven't really read that prior work. If they haven't, have they really done proper due dilligence on their research? Are they just trying to "beef up" their paper with citations to unfairly build up credibility?
rokob 9mo ago
As a reviewer I at least skimmed the papers for every reference in every paper that I review. If it isn't useful to furthering the point of the paper then my feedback is to remove the reference. Adding a bunch of junk because it is broadly related in a giant background section is a waste of everyone's time and should be removed. Most of the time you are mostly aware of the papers being cited anyway because that is the whole point of reviewing in your area of expertise.

bdangubic 9mo ago

same (and much, much, much worse) for science

barfoure 9mo ago

I’d love to hear some examples of poor electrical work that you’ve come across that’s often missed or not seen.

joshribakoff 9mo ago
I am not an electrician, but when I did projects, I did a lot of research before deciding to hire someone and then I was extremely confused when everyone was proposing doing it slightly differently. A lot of them proposed ways that seem to violate the code, like running flex tubing beyond the allowed length or amount of turns. Another example would be people not accounting for needing fireproof covers if they’re installing recessed, lighting in between dwelling in certain cities… Heck, most people don’t actually even get the permit. They just do the unpermitted work.
AstroNutt 9mo ago
A couple had just moved in a house and called me to replace the ceiling fan in the living room. I pulled the flush mount cover down to start unhooking the wire nuts and noticed RG58 (coax cable). Someone had used the center conductor as the hot wire! I ended up running 12/2 Romex from the switch. There was no way in hell I could have hooked it back up the way it was. This is just one example I've come across.

lencastre 9mo ago

an old boss of mine used to say there are no stupid electricians found alive, as they self select darwin award style

xnx 9mo ago

No doubt the best electricians are currently better than the best AI, but the best AI is likely now better than the novice homeowner. The trajectory over the past 2 years has been very good. Another five years and AI may be better than all but the very best, or most specialized, electricians.

legostormtroopr 9mo ago
Current state AI doesn’t have hands. How can it possibly be better at installing electrics than anyone? Your post reads like AI precisely because while the grammar is fine, it lacks context - like someone prompted “reply that AI is better than average”.
- xnx 9mo ago
  An electrician with total knowledge/understanding, but only the average dexterity of a non-professional would still be very useful.