Home | Notifications | New Note | Local | Federated | Search | Logout
Reply to @cwebber@social.coop Christine Lemmer-Webber@cwebber@social.coop (2026-07-22 21:31:00) To those saying the OpenAI + Hugging Face thing is an ad, I think that's true, but I think it's not just an ad. They're taking the marketing opportunity for sure, and it's *awfully convenient* that it's two AI companies as part of the story, right?Reply
But, despite being a big critic of this stuff, I do think that they are *very good at attacking systems*. The reason being that most attacks tend to have very similar patterns, and these models are great at blasting similar patterns over and over again, so they can ruthlessly find their way through exploits.
The bigger question to me is: which examples *aren't* there motivations for two companies to put out statements together? Where else has stuff like this happened?
And the more important one, the response I think they *want* people to hear is "our stuff is so scary and powerful, you have to use and trust even more of our vibetech".
But that's not MY takeaway...
---Reply--- Christine Lemmer-Webber@cwebber@social.coop (2026-07-22 21:34:41) I really don't think that you can fight LLM agent exposed and exploited security issues with *more vibes*. You absolutely cannot. These machines are good at finding exploits, but they are also good at *inserting* them, accidentally all over the place, and also in terms of them being the perfect machines to insert subtle, hard-to-find intentional vulnerabilities.
In terms of using them as vulnerability scanners, I think they're important tools. In terms of code generation, there is simply no way to fight vibes with more vibes.
Instead, we need to get systemic about things.
So how do we fix security issues SYSTEMICALLY?
---Replies---
Christine Lemmer-Webber@cwebber@social.coop (2026-07-22 21:39:20) Most security vulnerabilities have pretty common shapes. But we also have solutions to them:
- Ambient authority! Mitigation: capability security
- Confused deputies! Mitigation: capability security
- Memory exploits: Mitigation: use memory safe languages (Rust and etc if you gotta go low level, though languages with a GC or etc are actually even *safer* because they don't have "unsafe" blocks of code) or use CHERI
- Injection attacks: Use structured representations. Quasiquote or combinator templating is much safer than string templating, for example. Why are we speaking to our databases in sentences? Send them the AST of the query!
- Bad crypto usage: Use standardized cryptography patterns
There are a few others, but even if you just focus on the above, we can simply *eliminate* huge swaths of security issues.