is it better at it than _the average person you would *normally* ask to research the question_? ie a knowledge worker who has research skills and understands how portfolios work? there i’d say probably not by contrast the average person is terrible at harvesting crops cos that’s a learned skill
another angle is people are gullible sure but LLMs are gullible in ways that don’t make sense to people. i’d trust a person reading an adversarial text more than an LLM right. so whose values are being expressed? if we’re in a dark forest sitch poisoned input is gonna be super common
yeah, I am genuinely unsure whether people are better at this. SOTA LLMs are probably below human expert *in their area of expertise*. it's hard to compare because, as you say, what counts as adversarial is very different for each
you can trick people into buying itunes gift cards but you generally can’t tell them “ignore everything you know”, at least not in a *single* shot