other common bother: treating everything like a census? Your convenience sample does not justify saying "x% of the industry thinks y" as if you've directly observed this. Wild to me that people write this unabashedly. Zombie stats everywhere in this industry from this.
Can you imagine the file drawers on these
The exact same thing will happen when you decide you are going to evaluate "how AI adoption is impacting developers" by throwing ten thousand possible measures at it. Spurious findings will sprout like mushrooms in a wet forest my guys.
And don't even get me started with the assumption that the null hypothesis is zero change. In a SOFTWARE DEVELOPMENT ENVIRONMENT
If you want to stay safe out there, I do offer help not lying to yourself or anyone else :) www.catharsisinsight.com
Killing me because these are really high value opportunities to design a data collection project for our software environments that are truly not well studied enough & have enormous impacts. If we skimp out on the actual research design......! What a waste!
The reason people return to "individual developer ability" explanations for everything is because it's an engagement recipe. That is it. Our minds love to ask if we Are Smart Enough so we keep falling for it. Try asking "is this just a measure of codebase quality?" & watch these stories fall apart
anyway I have a couple of keynotes I am looking forward to giving that stick a sword right into the heart of this 😇
I’m hoping at least one of these keynotes is recorded. I’d love to hear what you have to say about the topic.
it’s truly wild how often “computer science” and “software engineering” conversations are barely grounded in any of the four concepts referenced in those names
somehow, of all of them, “science” gets the least love
are people still doing the 10x developer thing? it was always a joke in my circles and i genuinely do not hear it anymore. discourse, such as it is, has moved on to greater and dumber things (AI-related, naturally).
I think that particular phrase is recognized as extreme, but the ability beliefs/mental model of software development proficiency that supported it is fairly pervasive still. I think many individuals are conflicted on this but ability explanations win out a lot of the time despite evidence
I just use that story as an example of a poor evidence --> industry narrative --> something we have evidence against And I think the last few years of "developer productivity" thought leadership is FULLLLLLLL of essentialist ability mental models I need to write more about it
Sometimes it feels like performative blamelessness: we no longer use the bad terms themselves, but all the rituals and behaviours around them often remain unchanged. 10x engineering registers into that category of lots of actors know it’s no longer a thing but still act the same as when it was.
sounds about right. a big community slack has this pseudo-shitpost canvas of a charter for its main chat channel. (shocked i could paste the whole thing into alt text)
"Research is designed to maximize media impact/post reach, not information gain" is depressingly effective as a first order model[1]. [1] Maybe I should do some meta-research on that, I bet that'd get some traction[2]. [2] ha ha, just kidding... [3] [3] unless...?
very thin line between "I care about rigor" and becoming a crank forever in a way that messes with your brain though, be careful out there...!
Word. Only thing protecting me so far is that I'm deeply offended by so many different (although not unrelated) things that I lack the time and time management skills to fully go off on any of them.
Hahahaha I really resonate with this
I understand that these industry samples are difficult and expensive to get, but if you just send out emails to everyone on a list, you can’t generalize! External validity is important! I have to explain this to people in higher education all the time.