Meanwhile, I see deep questions on the AI side that should definitely warrant some humanities input. And I don't mean the usual ethics corner, but: how to design reasoning structure/flow, personality tuning (what is the "I" in the model?), rethinking tokenization. Won't happen for some time.
If we had had more humanities input, we probably could have foreseen how bad naive RLHF would be in terms of promoting sycophancy, and in turn what harm having models be endlessly sycophantic could do to certain vulnerable human populations (e.g. promotion of delusions, etc)
Even just bad for training. I'm still slightly horrified by the post-training datasets still in use in the open ecosystem. Just a tiny bit of care from people with actual expression skills would make a difference…
Also: we are switching pretraining paradigms right now from fuzzy web crawl corpus with honestly questionable data practices all around, to some form of generalized "distant writing". Don't think anyone is aware of this/talking about this in DH circles and this is certainly a miss.
There is a very small "group" of us doing computational theology. In broad terms, we're asking 'distant writing' related questions: what does it mean when a model trained on theological texts then *writes* theological text? How do we categorize it? Is it "new" theology? Etc.
Could be actually an interesting candidate for amplified mid-training: not a large search space, enough to get performance with a small model.
In hindsight, what I should’ve done is written a few articles or a book, *and then* joined an outfit where I shouldn’t say much. There’s *a lot* to say here.
Yeah, I know exactly what you mean. Given that *nervous cough* I suspect we’re thinking about very similar problems. There’s a line to draw from before nanogenmo, back translation, synthetic data making, etc etc. It’s both interesting and, frankly, very fun.
Ah ah. I think multiple research team are circling on the same thing. Last release from Meta on active learning hit a bit too close to home.
Something something, one person’s language data augmentation is another person’s distant writing is another person’s creative writing project
We’ve also been training on a mix of code and natural language (cleaned/augmented or not) since GPT-3 which should have been extremely weird to everyone. But I agree that humanists are well placed to contrast the long history of “reasoning” with whatever it is that currently goes under that name
Say more: Do you mean synthetic data?
Yes but that won't say much. It's rephrasing/knowledge amplification. It's simulating problems with logic constraints at scale. It's writing things that have never been written before (like a complete search sequence). (most of my daily work right now)
FWIW I'm really interested in the editing / rephrasing / amplification side, but my mostly-sole-author research and publication pipeline is comparatively glacial (probably too slow to be remotely helpful or interesting for most AI/ML people). I wonder if that's a factor here? We'll see more in +
6-12 months from DH people?
*people who are much faster than me!
Pierre-Carl is at the cutting edge of this stuff. I don’t know how well academic DH will be able to keep up, but I think it’s fair to say that I know lots of people who do recognize these as exciting problems!
Yeah, I'd think that there's a lot of work on them coming down the pipes.
As a layman, can I ask what is "distant writing"? Is it about the distance between the thought (prompt) and output?
From a DH perspective, it's roughly the reversed of distant reading: "reading" lots of text using computational/text mining methods => "generating" lots of texts using language models and some formal constraints/inspirations (which are called "backtranslations")
Main purpose is to train better small models for dedicated tasks. Language models have a very high memorization inertia: you need lots of text to get accurate assimilation of new facts.
I'm working on getting product folks with more humanities backgrounds much deeper into the technical side of things where I work. All the bluster of a lot of these people about being luddites dies when the zoom meeting starts and its time to prioritize deadlines