If LLM developers wish to claim it manipulates ideas, they should adopt cleanroom practices: train LLM A on copyright content and train LLM B exclusively on content licensed for the purpose (or public domain). LLM A can create an auditable intermediate representation that would be read into LLM B.
User queries only ever go to LLM B, which has never accessed the copyright materials. Many researchers in this area would agree that LLM "quality" would drop precipitously in this setting. There are many studies supporting the thesis that LLMs do not, and perhaps can not, manipulate ideas.
Existing tests for substantial similarity are circumstantial evidence of the mechanism by which a human produced the allegedly-infringing work. I believe those tests lack construct validity for machine-generated content.
(Having done some work on this, I think this type of clean room practice isn’t actually super feasible in practice. Even if you get curation like this right—a huge if—I’m finding that these types of counterfactuals are really brittle in practice)