It's worth realizing that, before computerized central offices, telephone wiretapping required running physical wires. Back when Rudi Giuliani was prosecuting organized time, not only did physical wires have to be run, the cops were billed for them as expensive private lines. His task force was spending about a million dollars a year with New York Telephone on wiretapping.
In one case, law enforcement didn't pay their bill, resulting in the person being wiretapped having the wiretap connection show up on their bill, blowing the case.
That resulted in the Communications Assistance to Law Enforcement Act, which mandated that central offices offer remote wiretapping. Capacity up to 1% of lines is required.
Back in the electromechanical era, the only call data that could be collected was outgoing dial pulses, using a "pen register".[1] (The one shown in Wikipedia is mine. It's a beautiful piece of antique brass telegraph technology. It records dial pulses as dashes, and has to be wound up like a clock, with a big brass key.) The Supreme Court decision allowing "pen registers" without a warrant refers to these "extremely limited" devices. That definition has been stretched and stretched by law enforcement into all non-voice data collected by telcos.
> In the real world, it does feel likely that we’re going to hit some sort of a ceiling on the number of useful bugs, and probably we’ll hit it soon.
This doesn't resonate with me. I see companies adding more sloppily written features with AI. I see more bugs in the software I use, not less. While it's plausible that software is getting both buggier and more secure, I suspect those two move in the same direction not opposite.
My guess is that we're getting better at finding _existing_ security issues with AI (and thus fixing those issues), but simultaneously adding more insecure surface areas _at a faster rate_.
mbroshi
On one side, you have pieces like this, where seemingly there are constant fights between serious actors with large and properly distributed budgets, employing top tech and top minds; on the other - regular news of the hackz, where responsible person in charge of security with root access failed to grasp basic technical knowledge (several times), ticking every checkbox in "never do this" list from security best practices, which led to every customer being pwned.
It's like two parallel worlds, that exist in the same place at the same time, but somehow don't cross.
Insimwytim
I think this comes in with the wrong assumptions from the start. The thing that US vs Apple taught us is to not demand access publicly, this puts companies in an awkward spot. If approached more tacitly, gag order etc the company has nothing to gain but everything to lose.. and more likely to comply. I hugely doubt that intentional backdoors don’t exist for the most powerful countries / people
tipsytoad
I don't think the thesis that a government will be able to do something will ultimately hold. I don't see how they can avoid "going dark" in a democracy.
we live in a world where the government can't even do much about illegal drug markets anyone can access by downloading a piece of software.
if they pass laws that mandate backdoor access and block software which doesn't conform more and more people will move to the dark networks.
and if they effectively block the dark networks (in the limit they will have to block all encrypted communications) then we will be living in a tyranny.
freedom is messy. accept that digital crime can only be solved when the criminal makes a tangible mistake. LLM's will be building profiles on criminals to help with identifying mistakes.
teravor
It breaks my heart that the governments with unlimited budgets who have hired the best and brightest will have to put in serious effort to get the bad guys, and potentially find it not worth it to casually spy on the whole world.
I am just beside myself at such an idea that people looking to feed the prison machine cannot as easily find excuses to turn normal citizens into prison feed.
Just super sad guys.
RajT88
I've always loved the ridiculousness of the "going dark" label when law enforcement can't access encrypted chats or a back door isn't built into a piece of software. When there are security cameras on the vast majority of houses, stop lights and in people's hands, and when so much meta data about people's associations are shared from Google, Facebook, any other social platform, how in the world can they say they are "going dark". How did they ever solve crimes before these things?
fitblipper
I'm supposed to be concerned that the US government and Israel won't be able to hack everyone's phones?
Gigachad
> In fact, the worst part about this dynamic is that these potential new backdoors will begin primarily useful for allowing the US to weaken its own systems, which will in turn allow foreign adversaries to find new ways to attack our communications. This deliberate self-sabotage will happen just at a moment when we’re finally learning how to defend our own infrastructure.
I don't understand how you can both argue for that law enforcement (and intelligence) agencies will force others to implement intentional backdoors AND also everyone will be using AI to find and secure ALL potential holes in the software so there won't be any vulnerabilities anymore.
Wouldn't one AI or another detect this deliberate backdoor and report it, as it'll look just like any other security vulnerability, the only difference being the intention?
I have respect for the author so I feel like I probably misunderstand something from the overall text rather than I somehow have a better perspective on this topic that the author knows very much more about than me. I felt like I nodded along all up until "So how is this a problem?" and now I'm not sure I understood correctly.
embedding-shape
I'm going to be a little epistemically silly here. Why doesn't Goedel's incompleteness theorem apply to the assertion that AI-enabled patching will prevent all exploits?
I'm quite in favor of impenetrability of software as the alternative is constant friction, to be honest.
comments (10)
That resulted in the Communications Assistance to Law Enforcement Act, which mandated that central offices offer remote wiretapping. Capacity up to 1% of lines is required.
Back in the electromechanical era, the only call data that could be collected was outgoing dial pulses, using a "pen register".[1] (The one shown in Wikipedia is mine. It's a beautiful piece of antique brass telegraph technology. It records dial pulses as dashes, and has to be wound up like a clock, with a big brass key.) The Supreme Court decision allowing "pen registers" without a warrant refers to these "extremely limited" devices. That definition has been stretched and stretched by law enforcement into all non-voice data collected by telcos.
Law enforcement still wants more.
[1] https://en.wikipedia.org/wiki/Pen_register
Animats
This doesn't resonate with me. I see companies adding more sloppily written features with AI. I see more bugs in the software I use, not less. While it's plausible that software is getting both buggier and more secure, I suspect those two move in the same direction not opposite.
My guess is that we're getting better at finding _existing_ security issues with AI (and thus fixing those issues), but simultaneously adding more insecure surface areas _at a faster rate_.
mbroshi
It's like two parallel worlds, that exist in the same place at the same time, but somehow don't cross.
Insimwytim
tipsytoad
we live in a world where the government can't even do much about illegal drug markets anyone can access by downloading a piece of software.
if they pass laws that mandate backdoor access and block software which doesn't conform more and more people will move to the dark networks.
and if they effectively block the dark networks (in the limit they will have to block all encrypted communications) then we will be living in a tyranny.
freedom is messy. accept that digital crime can only be solved when the criminal makes a tangible mistake. LLM's will be building profiles on criminals to help with identifying mistakes.
teravor
I am just beside myself at such an idea that people looking to feed the prison machine cannot as easily find excuses to turn normal citizens into prison feed.
Just super sad guys.
RajT88
fitblipper
Gigachad
I don't understand how you can both argue for that law enforcement (and intelligence) agencies will force others to implement intentional backdoors AND also everyone will be using AI to find and secure ALL potential holes in the software so there won't be any vulnerabilities anymore.
Wouldn't one AI or another detect this deliberate backdoor and report it, as it'll look just like any other security vulnerability, the only difference being the intention?
I have respect for the author so I feel like I probably misunderstand something from the overall text rather than I somehow have a better perspective on this topic that the author knows very much more about than me. I felt like I nodded along all up until "So how is this a problem?" and now I'm not sure I understood correctly.
embedding-shape
I'm quite in favor of impenetrability of software as the alternative is constant friction, to be honest.
tomrod