Talk:Nicholas Carlini/GA1

GA Review

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Article (edit | visual edit | history) · Article talk (edit | history) · Watch

Nominator: Sohom Datta (talk · contribs) 00:28, 2 September 2024 (UTC)

Reviewer: Vacant0 (talk · contribs) 10:50, 3 September 2024 (UTC)

Hi, thanks for nominating this article. I'll review it during the course of this week. Vacant0 (talk • contribs) 10:50, 3 September 2024 (UTC)

GA review (see here for what the criteria are, and here for what they are not)
It is reasonably well written. a (prose, spelling, and grammar): b (MoS for lead, layout, word choice, fiction, and lists): It is factually accurate and verifiable, as shown by a source spot-check. a (references): b (citations to reliable sources): c (OR): d (copyvio and plagiarism): It is broad in its coverage. a (major aspects): b (focused): It follows the neutral point of view policy. Fair representation without bias: It is stable. No edit wars, etc.: It is illustrated by images, where possible and appropriate. a (images are tagged and non-free images have fair use rationales): b (appropriate use with suitable captions):
Overall: Pass/Fail:
· · ·

Initial comments

• There is unlikely any copyright violation in the article. Earwig's Copyvio Detector has reported only 20.0% in similarity.
• There are no cleanup banners, such as those listed at WP:QF, in the article.
• The article is stable.
• No previous GA reviews.

General comments

• Prose, spelling, and grammar checking.
  • No major issues were found in the article. I've made some minor improvements: See Special:Diff/1244379524.
• Checking whether the article complies with MOS.
  • Change "Research" to "Career".
    •  Done - S
  • Reword "More recently", "Recently", "sometimes" all are listed MOS:WTW violations.
    •  Partly done - S, "sometimes" is correct in its context. The original blog post [1] itself noted that the attack works "often" (possibly due to the unpredictability of the AI model).
  • Change "well known" to just "known".
    •  Done - S
  • The rest of the article complies with the MOS:LEDE, MOS:LAYOUT, and MOS:WTW guidelines. There is no fiction and embedded lists within the article, so I am skipping MOS:WAF and MOS:EMBED.
• Checking refs, verifiability, and whether there is original research.
  • Reference section with a {{reflist}} template is present in the article.
  • No referencing issues.
    • Expand Ref 5 and 10 by adding work/website.
      •  Done - S
  • Most listed references are reliable.
  • I've spotchecked the entire article:
    • Education section content is verifiable.
    • I do not have access to Ref 4 so I cannot verify what's written inside.
    • Ref 5 seems to only verify that he has worked on adversarial machine learning. I could not verify the rest of the sentences, so I assume that's written in Ref 4.
    • Ref 6 and 7 verify the sentence.
    • Ref 8 verifies the sentence.
    • Ref 9 just cites Carlini's work. I do not see any mentions that indicate that Carlini worked on the questionnaire.
    • Ref 10 verifies the sentence. Ref 11 does not mention Carlini.
    • Ref 12 verifies the sentence (Carlini is not mentioned directly in the source, though, but only as "one of the researchers").
    • Ref 13 verifies the sentence.
    • Ref 14 verifies the sentence (Again, Carlini is not mentioned directly in the source but only as one of the researchers).
    • Ref 15 verifies the sentence.
    • Ref 16–20 verify the awards.
    • Ref 21 just mentions "carlini". How do we know that it's Nicholas Carlini?
    • Refs 1, 3, and 15 to 21 all seem to be primary sources and/or not independent of the subject. This, with the two refs not mentioning Carlini directly but as "one of the researchers", leaves me divided on whether the person even meets WP:GNG.
• Checking whether the article is broad in its coverage.
  • Wikilink Carlini & Wagner attack in the lede.
  • No issues, everything is well explained. The article addresses the main aspects, and it stays focused on the topic.
• Checking whether the article is presented from an NPOV standpoint.
  • I've listed above some WTW issues that should be addressed.
  • "many other defenses" – do we know which? If not, rephrase this.
    • There isn't an RS that discusses all the defenses that had been broken using this attack. Based solely on Carlini's work during his PhD, there are at least 19 defenses that were broken using variations of the Carlini Wagner attack (7 of which were from the ICLR conference incident mentioned in the article), and the rest are mentioned in this paper by Carlini and this other paper by Carlini. I think Carlini himself claims to have broken over 30 defenses in a recent blog post [2], however, even if we take the lower estimate, the phrasing of "many defenses" is justified.
• Checking whether the article is stable.
  • As noted in the initial comments, the article has been stable.
• Checking images.
  • There are no images in the article.

Final comments

@Sohom Datta: I'll put the review on hold for a week for you to address the issues. I'm particularly divided on whether the person meets WP:GNG due to reasons listed above. Once most issues get addressed, I might ask someone for a second opinion regarding this issue. Vacant0 (talk • contribs) 20:34, 6 September 2024 (UTC)

@Vacant0 Sound good, I'll work through these over the weekend, three points:

• About Ref 4, I can send the pdf to you through email if you are interested, (also, I think The Wikipedia Library should also allow you to access almost any IEEE paper including that one)
• Wrt Ref 21, it can't really be anyone else, the feat achieved at ioccc was cursed enough that it required specialized knowledge and Carlini was the only person who has proved that the "printf" was turing complete in their 2015 paper. (Also see their github where they archived the version of the program they submitted)
• Wrt to the question of notability, I think the person happily passes WP:NPROF having had an attack named after him (pretty rare in computer security) which is also the single highest cited paper in machine learning security (9737 citations per google scholar), also see [3] not to mention the fact that their work is covered by multiple RS.

Sohom (talk) 21:39, 6 September 2024 (UTC)

Okay, thank you for clarifying regarding his notability. I've found Ref 4 on TWL and it verifies the sentences that is cited to in this article. Vacant0 (talk • contribs) 21:46, 6 September 2024 (UTC)

@Vacant0I've actioned most of the issues and left some comments for some of them. Let me know what you think. Sohom (talk) 23:09, 7 September 2024 (UTC)

Looks good to me. Promoting. Vacant0 (talk • contribs) 10:35, 8 September 2024 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.