what trips people up is that the lizard people vcs hyping up crypto and the metaverse in 2021 — which, is it fair to say that its only purpose is gambling and fraud? — are the same folks hyping up LLMs and promising that it will replace your doctor your therapist your best friend
the evil giant egg man who would happily sell my children into slavery says this is the future and frankly opposing every thing he says is in 2025 a useful and good heuristic
more narrowly i am not a philosopher of mind/language but reading the machine say “there are no credible sources” irritates me profoundly. CREDIBLE TO WHOM? who are you to judge credibility? lol. it’s all cognitive sharp edges
*no credible evidence, whatever
I agree it's weird but my thought here is that I am constantly noticing that what people consider credible sources is utterly fucked already. and in this particular case I am noting that it is correct, i.e., it is making the same judgment as me
the more out there version of what I'm suggesting is that *if* you can express what your values are to it, then it's plausible that SOTA LLMs are significantly better than the vast majority of people at analyzing sources in a way that is faithful to those values
as always I think what is most dangerous here is precisely that they can do this well and persuasively. I am much less worried about accidental slippage in the meaning of "credible" — this baseline ability deployed by cynical actors is way worse
imho we have to be careful about these comparisons. when ppl talk about “this LLM is an expert at task x” often the task has been defined down or the target population has been defined up, to wit: the average person has a high school degree & doesn’t read books bsky.app/profile/davi...
is it better at it than _the average person you would *normally* ask to research the question_? ie a knowledge worker who has research skills and understands how portfolios work? there i’d say probably not by contrast the average person is terrible at harvesting crops cos that’s a learned skill
another angle is people are gullible sure but LLMs are gullible in ways that don’t make sense to people. i’d trust a person reading an adversarial text more than an LLM right. so whose values are being expressed? if we’re in a dark forest sitch poisoned input is gonna be super common
yeah, I am genuinely unsure whether people are better at this. SOTA LLMs are probably below human expert *in their area of expertise*. it's hard to compare because, as you say, what counts as adversarial is very different for each
I really did mean vast majority — my original draft of this post was saying that under a holistic definition of literacy, most people with college degrees are illiterate
two of the three people I QTed in this thread have PhDs and the other is a professional writer