Ever stared at a table of regression coefficients & wondered what you're doing with your life? Very excited to share this gentle introduction to another way of making sense of statistical models (w @vincentab.bsky.social) Preprint: doi.org/10.31234/osf... Website: j-rohrer.github.io/marginal-psy...
📌
📌
Just shared this on the Stan forums to answer a question
📌
It's that "age as a predictor" plot from like a month ago! Looks great!
hahaha omg that meme choice is just perfect. Yes, that's what I've been mainly working on for the last months.
And many thanks 😊
📌
Thank you, and bravo, you are amazing (both).
Many thanks, glad if you liked it!
I am reading, and I feel super proud: In my last study (here www.medrxiv.org/content/10.1...), I model incidence of ED use with exponential on distance. I actually did counterfactual approach, and named it "counterfactual"!! (wanted to see ED use if everybody was "living" 5 km from the ED)
Sometimes I stare at a table of regression coefficients & wonder what calcified disciplinary conventions are doing with all of our lives.
Are you??? Your midlife crisis must be a lot more sophisticated than mine.
lol, doubtful. Someone must be capable of sophisticated crises, but personally I think that's only in novels. I suppose the therapists would be able to tell us.
📌
📌
sometimes I stare at my email inbox and watch the colors change from across the room as new messages pile in - tears filling my eyes. Then I’ll unstandardize all my regression coefficients
Yes!
📌
This is a really great paper - thank you, @dingdingpeng.the100.ci and @vincentab.bsky.social!
Thank you for the kind words Fred!
That's how I've been teaching it for a while. Glad to see that there is now a paper on it that I can use to justify this ☀️
Ah! I’ve been using this idea for a while now, but havnet found the time to get into marginaleffects yet (all my early attempts crashed against the issues it has with models using the gr(,by) notation). This is my chance, thanks! Also,
Not sure what gr() is, but if you post a problematic minimal reproducible example to github, I'll look into it.
I keep getting the equivalent of this (on much more complex models, see screenshot): github.com/vincentarelb... But I'm currently unable to compile the latest dev version, so I'll need to wait til I solve that issue to see if my new repr ex would have an issue w that.
What do you think of eg zero inflated models on this? I always end up first adopting the prediction machine perspective and then writing an appendix deconstructing the impact of the components of the mixture.
I don’t have much experience with these models but my first impulse would be to use marginaleffects to try to crank out predictions 😂
I do the same, but the I second guess myself and go through predictions at single component level, just to be sure I actually understand what’s going on 🤷 but I haven’t seen any discussion of best practices or anybody else doing that
For many multi-part models, you can use the `type` argument to specify which part you are interested in. So you can learn about the effect of X on Pr(S=1) or on Pr(Y=1) separately.
We do zi models all the time and we always use this approach.
"zi models" -- for a second I though you were imitating a strong German accent
"Return of the ZI" (because I can pronounce 'z' properly 😉)
I thought that was a French model...
"draw me like one of zi French models"
I will never forget the groans Niels Waller elicited by teaching “A correlation is just a covariance computed in France—between z scores!”
My day is ruined now, thank you.
Rest assured I taught all my students this terrible joke too
Wow what omg
That's a dad joke quality joke.
As opposed to my stereotype of Deutsch which would be something like "Die Fit Indices sind zu Groẞ!" which in pigeon english would be "die you fit indices you're too gross."
As a native speaker, I have nothing to add to this. Except I would replace Fit Indices with Gütekriterien.
As a native speaker who didn't study in Germany, I have nothing to add to this. Except I wouldn't replace Fit Indices with Gütekriterien.
Gütekreterien as an R package, generate any fit criterion for anything, always fails on install because ü
I'm confused by the implicit optimism of Gütekriterien. It seems to conflict with the German demeanor.
Just perfect
In my head I actually read the sentence "We do zi models all zi time und ve alvays use zis apprhoach," because it showed up in my feed without the context above it
Honestly that's a spot on impression of me.
Check out my paper for zero-inflated beta models with marginaleffects: osf.io/preprints/ps...
fantastic title 10/10
it's on the reading list :-)
I will say that for hefty models with lots of data marginaleffects struggles (e.g., Weiner diffusion).
You may be in World #3 from here. In any case, if you have a simple example with public data, I'd like to profile and see. Most of the time, the issue is that `predict()` takes a long time, but there might be something we can do to make things faster. bsky.app/profile/vinc...
I should have added that it is fast when you calculate MEM but if you want AME it can take a looongg time.
If your data are massive, you can fit the AME to a sample.
and if predictors are categorical, shrink it to unique combinations and re-expand by using the `wts` argument.
Or could it be possible to calculate the derivatives with sparse matrices? There are fancy computation speed-ups in the ML packages like PyTorch etc.
Also in the works, but every prediction and aggregation function needs to be written manually from scratch, so it's a big project. On big datasets, it can be crazy fast. see benchmarks. github.com/arcruz0/marg...
Hmm good idea. Might even be worth a convenience function...
There's code in `marginaleffects` already but it's not activated currently. When I'm done with this major refactor, I'll work on bringing this back as an automatic convenience when it works.
This looks great! Really clearly written and nice plots: thanks for sharing
Many thanks 😊
This is fantastic! Bravo both 👏👏👏👏
Oh, many thanks 😊
Two points! When varying multiple covariates (for a model that's locally differentiable) - it can make a big difference whether you make a first or second order approximation to estimate the local effect - and that's crucial to understand how to interpret the estimate.
Second point! For people that are not ML practicioners - if you are using a flexible non-linear model, it's absolutely crucial to assess goodness of fit on held out data. Any sufficiently flexible non-linear model can trivially get a perfect R^2 on the dataset it's fit on.
For your second point — I do wonder whether there’s anything systematic in the extent to which that actually affects focal quantities? We do some additional variations in the website and the comparison of interest seems completely unchanged even when a covariate is overfitted probably a lot.>
I think what is probably happening is that - if you fit a "standard" non-linear model in R (like, a spline, or a GBT) - the fitting procedure has defaults to regularize the fit to mitigate overfitting and make it easier to use out of the box. However...
If you roll your arbitrary non-linear model (even something as simple as a linear model + higher order polynomial features) and then fit it with a gradient based optimizer with no regularization, you will almost certainly overfit and obtain absurd results for your sensitivities.
I did some search on that but I couldn’t really find the intersection of concerns about overfitting when what’s of interest is not the model as a whole, but rather something like an average comparison.
PS there’s a (mildly) updated version available now: osf.io/g4s2a_v2
This looks interesting. I shall check it out. This pubmed.ncbi.nlm.nih.gov/7171691/ Is a favourite paper of mine. Turning regression into predictions has been a feature of Genstat ever since the 1980s. Peter Lane did the programming.
Oh wow! Yes, that seems like they were already working on this very idea. That's amazing, thank you for the pointer!
This is in a certain sense the follow-up to a blog post I wrote 3 years ago (!) after I had looked into the topic for a paper with Felix Elwert, who is thus once again to blame for large chunks of my output. www.the100.ci/2022/05/27/%...
📌
📌
Need to cite this in my Beta paper!
Speaking as a person just (belatedly) learning regression: er, yes!
This is really nice! Just a tiny hint, there is a miniscule error here: j-rohrer.github.io/marginal-psy... The link to the helper script on the top is leading to a 404: github.com/j-rohrer/mar...
Oh, many thanks for the pointer! I think that's because the repository was private (while the page wasn't); it should work now (or in a bit; I have the feeling settings on GitHub sometimes take some minutes to actually go into effect)
No worries, I think it'll work soon enough. + just a suggestion: Maybe consider adding "download as ZIP" after "Replication Package" for clarity. I didn't know there is no link to another website but immedeiately the download at first click but this could be "on me". Sorry if that's useless
No it's not on you 😂 I'd also probably be surprised, lemme quickly update this...
Link in menu leading to ZIP is a bit like when you text somebody "Sure, let's meet some day" and suddenly the person is standing in front of your door.
Phew, happy you see it like this :)
I’ve read the paper but I’m still wondering what I’m doing with my life.
Me too, which is why I never implied any logical connection between the two sentences 😋
Foolish me. But your paper is awesome. It will save me having soooo many conversations!
Oh gosh, stuff like this makes me sad I'm not teaching students anymore and able to utilise such great resources for those purposes... Looks great though so will channel that into introducing colleagues and collaborators to the wonders of marginaleffects 😊
Thank you for the kind words 😊
Haha
Thank you for writing this - it looks great! I can't wait to dig in.
Hope you’ll enjoy reading it!
Never. That’s for you stats people.
📌
📌
Nice paper! I would add some more context on the term "counterfactual", probably some/many readers are not familiar with that term, and I think it's an important information because - unlike the pure coefficients - counterfactual predictions involve some "data manipulation".
I.e., while for simple predictions, it is easily possible to calculate expected outcomes with "y = b0 + b1 * x1 + b2 * x2...", setting non-focal "x" to mean or zero (for factors), counterfactual predictions "clone" the sample.
V important point. I think I've bugged @vincentab.bsky.social with so many questions about counterfactuals over the years in an attempt to get these that he could have blocked me by now and id have said "fair" :p
Good point, will add that to my notes for the next iteration!
100%. This is a great suggestion.
Super cool!✨️
Thank you 🦄
📌
Doing the Lord's work, Julia & Vincent! The marginaleffects package is so damn good, happy I can refer to this paper now in reviews :-)
Thanks Fabian, writing papers that are referenced during the peer-review process is sort of my thing 😂
📌
This is great! Thank you!
📌
@andrew.heiss.phd
Some pretty graphs here! 🤩
omg I'm such a fan. And you commented on a paper of mine! (Julia drew all the graphs.)
Wait, of mine? But you're doing such cool stuff. Is it legal for people to be fans of each other?
fantastic! Straight into the reading list for graduate stats. One thing that might be useful is a conceptual paragraph about how statistical power/sample size estimation changes. I can imagine (enthusiastic) students stuck on how to adjust what they know about study planning.
If you stare really hard at the cutpoint coefficients from a partial proportional odds ordered logistic, they stare back
This looks really interesting!
Very cool! I'm intrigued by the treatment of ordinal variables, and I have a question. Will this approach: avg_comparisons(mod_ord, variables = "partner", hypothesis = ~ I(sum(x * 1:5))) also give a good answer when the response-distribution is unbalanced, e.g. most people responded on level 1?
I would be so disappointed if that one doesn’t spark controversy 😂 in my mind it gives you a good answer to exactly the question it’s asking, but apparently that’s not the smartest question to ask (Vincent made me move it to online-only)
I was surprised it was there as well. In a thread on the marginaleffects github, I asked about computing the expected value in this manner and Vincent said "it can be done, but don't do it" :p
wrote a bit fast, that's probably not an issue if one assumes the proportional odds assumtions holds and models the data accordingly, as done in the example. If one relaxes the assumption and models category specific effects, things are probably different.
You may be interested in the article by Mize and Han cited at the top of this page (and also might want to look at the examples on the page). marginaleffects.com/bonus/total_...
This is amazing! My one wish is that you’d have also touched on the difference between average marginal effect/slope vs marginal effect at the mean. In sociology, model almost always have large-ish number of controls and dealing with them is a hassle.
TLDR: avg mfx if your sample is representative; weighted avg mfx if you need weights; mfx at mean only if your computer is slow.
Petition to rename the package mfx
You can be on @gmcd.bsky.social 's "I love short aliases" team, and I'll stay here by myself screaming "tab completion works well enough". 😂
It’s not the brevity I like. It’s the alternative pronunciation
lol
"emeffex" compromise solution?
Oh that’s a good one tho, I’ll add a note that we add this when we revise it! Only takes a couple of words
Thanks for the pointer, you'll get an acknowledgment 😋
Well...no...but I'm still glad y'all wrote this.
For you the question would be: Ever stared at people staring at regression coefficients and wondered what you're doing with your life?
📌