He then goes on to argue about impediments created by IRBs. 2/2
Blog: https://argmin.substack.com/ Webpage: https://people.eecs.berkeley.edu/~brecht/
7,290 followers 219 following 1,224 posts
view profile on Bluesky
He then goes on to argue about impediments created by IRBs. 2/2
Hah, he’s doing a funny inversion: his definition of research comes from this government website that defines human subject research www.ecfr.gov/current/titl... 1/x
Stephen, those two words can’t possibly have operationalizable meaning.
Kevin isn’t on here so I can’t ask him, but I think he’s just turning a party line is causal inference back on itself. I have heard many times that RCTs lack external validity by fiat.
This new center strikes the right tone in approaching the AI alignment problem. alignmentalignment.ai
On unquantifiable costs and inherent tradeoffs in decision theory. www.argmin.net/p/stuck-in-t...
Yeah, this is when we start to pick at what "work" means. If you can't get your arm to move into position for weeks, and your physical therapist does some pseudoscience on you, and all of a sudden you're back at top performance, did their adjustment work?
This is one of my favorite examples because chiropractic adjustments and bodywork clearly work. We just don't know how to set up an RCT that proves they work. But RCTs are incredibly limited as sources of epistemology.
This is tricky because a lot of scammers have an element of true believer in them (like supplement pushers). 2/2
I totally agree with you that medicine has many examples where unusual practices were eventually validated as scientifically effective. But I can't find any examples that started off as literal scams and then became mainstream treatments. 1/x
In your posts, there are just as many notes as there should be.
Love this example
PS @akhilrao.bsky.social Sleaze Theorem once again remaining undefeated.
This post expands on my musings on here yesterday. Thanks to everyone who brainstormed with me.
How Snake Oil Becomes Normal Technology: A Brief History of American Capitalism
Awesome. Thank you! Do you have a positive example of a technology that was once considered snake oil but has become generally liked?
You somehow go from a supplement scam to Betsy DeVos as Secretary of Education. It's just so insane!
I highly recommend Bridget Read's "Little Bosses Everywhere" for a deep dive into the history of MLMs.
A second example: multi-level marketing. What started off as culty scams became hugely influential corporations (that are still culty scams) protected by law.
Notably: cryptocurrency. It didn't necessarily start off as snake oil, but it quickly became a lot of scams and smoke and mirrors. But now (even though it's still super sketchy!) it's accepted by institutional investors and backed by law.
This was a catty subtweet, but I swear I am earnestly interested in an answer. There are a few good ones in the replies.
Holy crap.
Though I was cattily subtweeting, I was earnestly interested in the phenomenon! I also think you are being a bit generous with their argument, but your generosity is one your best attributes.
Yes. Crypto is a fascinating example.
Though this inverts the order I initially asked for, I still want to read this history.
Does anyone have recommendations for studies or books on how snake oil becomes normal technology?
Three different views of optimal decision theory yield identical decisions. Which view is right?
I wrote a short summary with some context on the blog.
In a new paper, I try to resolve the counterintuitive evidence of Meehl’s “clinical vs statistical prediction” problems: Statistics only wins because the game is rigged.
The dirty secret is Nature's AI journal only publishes 4th tier work. But that doesn't stop our friend Ethan from posting screenshots, leaning on the authority of the nature logo.
Please be rude, Kevin.
I'm confused how this war would work, considering our entire lifestyle depends on China selling us stuff. It doesn't make any sense.
He's also 80, so born before the computer. Possibly a more transformative technology.
That judge (Alsup) apparently decides all tech cases. (Oracle v Google, Waymo v Uber, etc.). www.theverge.com/2017/10/19/1...
Lecture 2 talks about actuarial prediction, why it’s easy, and how a statistical view declares all variation is “noise.”
Thank you!
@mraginsky.bsky.social needs to team up with Sasha Rakhlin to write the Russian ML story. They have actually read all of the original papers and met many of the main characters.
I write about machine learning in The Irrational Decision (preorder your copies now! LOL), but I tell a decidedly American story about it, drawing the line from Bell Labs to Neurips.
How we’ll be evaluated in the future determines our machine learning methods in the present.
fall on me.
OMG. Really kicking myself for not seeing that connection until today. You say I can't just shoot a hole into the surface of Mars? Wait until you see *this* optimizer.
absolutely. airplanes are incredibly dangerous! but when we engineered the shit out of the environment they fly in, air travel became remarkably safe.
What I love about your example is that it could be any number of people and is basically the same attitude Nate Silver has. They are all bravely converging on exactly the same worldview, by pure deduction alone, apparently.
I have had multiple AI safety tell me I'm wrong, and they have written extensively about the dangers of self-harm. I have had multiple tell me we can talk about both things at the same time. I don't know exactly what the right move is, but 'maddening' is right.
Is it all just security theater?
It's crazy, but it's true. And for me, over here in the ivory tower, I'm fascinated by what this reveals about the tasks themselves. What should our expectations of accuracy and precision be? Totally fascinating to me.
I agree with a minor caveat here: why/how do we trust the structure output of these extractions? to be fair, it makes me rethink a lot of the trust we put into "principled" structured extractions in the past...
Apologies because I'm in the middle of one of the most AI crazy parts of the world, but I'm sure you'd also agree that when AI Safety people talk only about skynet and bioweapons and never about potential self-harm, then there is a real horse race and it's completely misallocated.
Sure, but a quadrotor with an IED strapped to it is very different than a chatbot. Maybe one of the bigger issues is that when we adopt the umbrella term "AI" for anything partially automated, then scoping what it means for AI to be safe becomes intractable.
I don't think the analogy of car crashes to climate change is like the analogy of mental health to skynet. The skynet harm is completely made up and stolen from movies. Analogizing it to climate change helps those who deny that climate change exists.
This is the important point. They imagine a bunch of ludicrous scenarios about bioweapons, and spend zero time thinking about mental health, isolation, and. self-harm.
Yup. And not just philosophers. There is a whole breed of "AI Safety" researchers in CS who do the same SciFi nonsense and get enormous research funding. They got mad at me for calling them dorks complicit in death yesterday. www.argmin.net/p/the-banal-...
This graphic, from your webpage, is the banal evil I'm talking about.
Sleaze Theorem. Undefeated.
One extra AI Safety tidbit for this morning: Anthropic's Claude refused to transcribe this passage by Allen Wood (h/t @hilzoy.bsky.social) because it violated their safety protocols.
I wrote about the tragic death of Adam Raine and the venal negligence of "AI Safety." www.argmin.net/p/the-banal-...
As promised, a syllabus for this semester’s graduate course on machine learning.
you are right, but it's still astounding how full of shit that particular poll guy is and how he must clearly be making money selling the garbage statistics he compiles.
Also, a lot of optimization convergence proofs are just mindless crank turning! It's an industry that Jason Lee calls "the bashing method." And many people have written papers showing bashing is legitimately automatable... for example this dum dum: arxiv.org/abs/1408.3595
I have too many thoughts. Notably, Bubeck works at OpenAI and has a history of claiming they find amazing novel results that turn out to be already published.
There's a reason why sibling marriage is taboo.
Everything is AI now, and I hate it.
I just realized I can do the meme... Tired: We ran a genetic algorithm where the evolution step runs BFGS. WIRED: AI
I hate AI so much.
This might be too deep a cut, but the "AI" in that article is a genetic algorithm that tunes its population with 1000 parallel instantiations of the BFGS nonlinear optimizer.
Muting all Matts would probably be fine...
Optimization folks, it turns out that BFGS is sentient.
But how do you mute the word "argument?" The entire project is set up for hate-click engagement farming.
A fun fact is that he subsequently abandoned physics and did a PhD in Computer Science, specializing in graphics. (True story!)
PS. I wouldn't do my man Paul F. dirty. He's misunderstood but totally brilliant and hard to disagree with. But he's also a terrible communicator. :)
Yes, that's a great point. Moore's Law was much more a KPI than a law of nature.
LOL. So far just me. Though, to be fair, I can't read all of cs.LG on arXiv...
For what it’s worth, I was hoping to find a way to teach a new class, “How To Argue With Statisticians,” this fall. Unfortunately, I will have to postpone said plans until next year.
Should we still teach machine learning classes? Let me justify my existence.
Yeah, my stupid face will be there in person momentarily. I’ll be shilling hard for neoliberalism.
LOL, I'm sad that I know where you are and can feel the Fremdschaemen from my house.
100% agree with this.
Are linear models simple and interpretable? My answer won’t surprise you.
lol indeed! I don't know if I need to be reading that.
what's the difference?
Huh, I need to read that Dubins and Savage book, don't I?
I love this synopsis of the survey. Thank you. I'm actually teaching Learning for Control this Spring Semester. Revised hot takes coming...
I'd argue it's not fine and great because no one wants to engage with its limitations.
The latter. Because if the 2nd issue is true, the first issue doesn't matter.
Today's @lioneltrolling.bsky.social article, ICYMI www.unpopularfront.news/p/against-vu...
I'm really fond of John Ganz's coinage of "Vulgar Positivism." The Argument thinks it can make a profit embracing maximum vulgarity.
The new magazine promising to "publish people who seek truth from facts" opens with an overconfident analysis of cash transfer RCTs. Cursed timeline.
Should machine learning care about simple baselines?
Cheers, Max bsky.app/profile/been...
Sadly, the students are cooked, too. www.nytimes.com/2025/08/10/t...
For example: They say "AI diffusion in safety-critical areas is slow," but this ignores the widespread adoption by students for coursework or doctors for scribing patient conversations.
Right. Fundamentally, arguing against X-riskers forces them into a peculiar debate position. And many of their claims are outright incorrect.
What did you like about that essay? I read it, and its thesis goes against most of what Max argues in his piece. (Unless you apply the transitive property, where AI is snake oil, and AI as normal technology together means all normal technology is snake oil.)
To be fair, it's also polyvalent in the sciences, but scientists can't pull their heads out of their asses. There's a huge difference between: - In a landmark paper [1]... - This paper sucks [1]... - For related work see [3-97]... But all three have the same impact on google scholar stats.
This might be the greatest random YouTube recommendation I've ever received. www.youtube.com/watch?v=vaG5...
Evidence to your point: Google started using h-index as a metric for whether or not they'd interview people for research positions.
Good enough to make a living. But can you look yourself in the mirror?
Statistical pedantry is the devil's work.
that's right.
If machine learning is a game, it’s Calvinball. Bitter lessons from chess don’t apply.