Perhaps they are slightly more sophisticated but as a data scientist I sort of doubt it. Neural networks are nothing new they have been around since at least the 70s. The weren’t practical when theorized due to the lack of computing power and training data. These guys haven’t “invented” anything.
Moreover, having tried to use neural nets instead of a less “sophisticated” model usually results in lower predictive performance. The area where neural networks are really necessary is cases where the data needs to be encoded for example images or language.
However, at this point pre-trained encoding layers exist and image identification models can be trained very efficiently by only retraining the last few layers of the network using PyTorch. Training is the very costly part of the process and these LLMs are retrained all the time.
I agree. This is neither new nor revolutionary. It is, however, an evolution of something that came before. Like most discoveries. And it will cause a lot of further harm before it is allowed to implode.