OpenAI spends about $700,000 a day, just to keep ChatGPT going. The cost does not include other AI products like GPT-4 and DALL-E2. Right now, it is pulling through only because of Microsoft's $10 billion funding
those studies aren’t talking about asking it “what is the square root of pi” or stuff like that. but stuff such as “is 7 greater than 4?”, “what is 10 + 3?”, “is 97 prime?” stuff it has most definitely seen the answers to.
No, they very explicitly checked to see whether the training set contains the literal math problem that they asked it for the answer to. ChatGPT is able to answer math questions that it has never seen before. I believe this is the article (though I had to go searching, it’s been a while).
When people dismiss LLMs as “just prediction engines” they’re really missing the point. Of course they’re prediction engines, that’s not in dispute. The question is about how they go about making those predictions. When I show you the string “18 + 10 =” you can predict what comes next, yes? Well, how did you predict it? Did you memorize that particular specific string, or have you developed heuristics for how to do simple addition problems when you see them?
Humans are also not particularly well known for their math skills. Ask a random stranger to do simple arithmetic in their head, with only a few seconds to think and no outside help, and I wouldn’t expect particularly reliable results.
however, people are not notoriously bad at the types of basic arithmetic they test for. every time I pay something with cash, I work out how much change I’m owed mentally, and so does the seller. I can count on one hand the number of times I’ve actually been given incorrect change throughout my entire lifetime. And when I did get wrong change, it was usually “oh, I thought you gave me €10 ínstead of €20”. Meaning that they actually still did the math correctly.
No sane person will ever tell you 4 is bigger than 7. Yet llms sometimes get even this type of question wrong. They learn patterns, but not concepts. This is even simpler than basic arithmetic.
No, they very explicitly checked to see whether the training set contains the literal math problem that they asked it for the answer to. ChatGPT is able to answer math questions that it has never seen before. I believe this is the article (though I had to go searching, it’s been a while).
When people dismiss LLMs as “just prediction engines” they’re really missing the point. Of course they’re prediction engines, that’s not in dispute. The question is about how they go about making those predictions. When I show you the string “18 + 10 =” you can predict what comes next, yes? Well, how did you predict it? Did you memorize that particular specific string, or have you developed heuristics for how to do simple addition problems when you see them?
These things are currently infamously bad at math, though.
I won’t argue that it’ll never get there. I’m confident it will, - though with a lot more perl hacks than elegant emergence.
But today, these things have an astonishingly high ‘appearance of intelligence’ to ‘incredible stupidity’ ratio.
Humans are also not particularly well known for their math skills. Ask a random stranger to do simple arithmetic in their head, with only a few seconds to think and no outside help, and I wouldn’t expect particularly reliable results.
Haha. Fair point.
however, people are not notoriously bad at the types of basic arithmetic they test for. every time I pay something with cash, I work out how much change I’m owed mentally, and so does the seller. I can count on one hand the number of times I’ve actually been given incorrect change throughout my entire lifetime. And when I did get wrong change, it was usually “oh, I thought you gave me €10 ínstead of €20”. Meaning that they actually still did the math correctly.
No sane person will ever tell you 4 is bigger than 7. Yet llms sometimes get even this type of question wrong. They learn patterns, but not concepts. This is even simpler than basic arithmetic.