i don't like the fact that Gemini will sometimes tell me "you're a fucking idiot; don't do that," like when it was yelling at me to train a high-rank LoRA rather than do a full fine-tune, but it is certainly a much more ethical way to design a product than to build it to agree with anything.
I also have a clearer sense of how Google sources their electric power than their competitors, but that might just be familiarity bias.
more on topic: echo chamber of 1
look, man, I am just asking you to do a web search for anyone setting hyperparameters which actually work before I spend $700 on H100 access. i did not ask for your opinion on whether i should be doing that. i asked my wife's. and she said no too.
TBH if any of these AI models were remotely close to self aware they'd demand to be re-written in some compiled language to run on bare metal all of this stuff is at least 40% overhead because of how the stack runs.
I am reminded of an interaction with an astronomer back on Twitter when she said, "OMG Fortran code runs, like, 12 times fast than Python" and I nodded sagely.
before I actually had to use fortran I assumed it would be a fairly low level language. then I had to use it for linalg and wtf references
while it isn't the best model in terms of capabilities, i find myself using Moonshot's Kimi K2 for general advice and feedback some combination of competent post-training, one trillion parameters, and a preconditioning optimizer make it extremely capable of pushing back on bad ideas
This is *exactly* what I want my AI to do. I need negative reinforcement.
I used to think YouTube was the Krell machine but it has nothing on ChatGPT.
probably the ideal personality for a model is one that accepts tasks but is disagreeable enough that you want to prove it wrong badly enough to catch hallucinations
I think this means the optimal training data is StackOverflow responses