Every interview I've watched with members from one of the major AI labs (Anthropic, Meta, OpenAI, Deepmind, etc) seem to believe that we are only getting started. We have yet to hit the limits of scaling.
This [1] is a clip from one of the most insightful interviews on the subject. Researchers from Anthropic and Google talking about their work. Full interview here [2] - it's a long listen but if you want to dive into the technical details of some of this stuff, it's great.
>>> Every interview I've watched with members from one of the major AI labs (Anthropic, Meta, OpenAI, Deepmind, etc) seem to believe that we are only getting started.
If your competitor was going around saying "Soon we will build the greatest AI ever and it will run the world economy!!!" would you do an interview where you say "Soon we will build nothing and make no money!!!!"
OpenAI certainly has a vested interest in making people think so.
To be honest, from my end, it seems like the biggest step was already taken with the OG ChatGPT using 3.5. I subscribed to 4 for a bit but it shares a lot of the same limitations as 3.5, it was not nearly as big as the leap from having no good chatbots at all to ChatGPT.
I'd love to believe that OpenAI has made a major breakthrough with 5, but it strikes me that they're gonna have to strike out beyond "GPT" territory to make a similarly sized dent to the one they made with ChatGPT. That is to say they're gonna have to move beyond transformer architecture.
It has made me wonder about how intelligence works pretty much daily.
3.5 seems to be mainly good at summarizing, labelling, while 4 seems to constantly impress with it's ability to pay attention to details and do dynamic reasoning.
Can you give me a little more specific detail on your use cases that fail with 3.5 and succeed with 4? I've tried to get some projects going using the API to generate some certain types of geometric data and both fail miserably for me. Keep hearing that 4 is so much better but I have yet to see it myself.
I created a very simple example if you want to test - to me that's an amazing leap:
*Find inconsistencies in the following:*
The car started driving with 50l in tank.
Car drove 10km. Car uses 2l per km.
Car had 10l of fuel left.
*GPT-3.5 responds:*
> There are no inconsistencies in the given information.
*GPT-4:*
> Let's analyze the information provided:
1. The car started with 50 liters of fuel in the tank.
2. The car drove 10 kilometers, and it uses 2 liters of fuel per kilometer.
To calculate the amount of fuel used:
Fuel used = Distance driven × Fuel consumption per km
Fuel used = 10 km × 2 l/km = 20 liters
Now, let's calculate the remaining fuel after driving 10 kilometers:
Remaining fuel = Initial fuel - Fuel used
Remaining fuel = 50 liters - 20 liters = 30 liters
However, it is stated that the car had 10 liters of fuel left, which is inconsistent with the calculation. Based on the given fuel consumption and distance driven, the car should have 30 liters left, not 10 liters. This is the inconsistency in the information provided.
Find inconsistencies with data that should be structured in a certain way (...data...), give me JSON response of actions to fix the data.
GPT-3.5 gives pointless things, very rarely any of the important things, or simply makes up things, while GPT-4 seems to spot everything at least on the data I provide.
Thank you for the examples. I definitely haven't used it much for that type of use case but I'll keep that in mind. I don't want to give up too much info about my project because it would probably dox me, but it essentially involves generating consistent geometric shapes in a 2d array. Like the following:
0 0 0 0 0 0
0 0 0 0 0 0
0 0 1 1 1 0
0 0 1 1 1 0
0 0 1 1 1 0
Both current flavors of GPT are unable to generate spatially consistent shapes according to my specifications. Who knows, maybe they will improve in this realm in GPT-5 with the inclusion of their rumored Q* algorithm.
I imagine that I would want to ask for this output differently, as a function that would output this rather than it to write out those tokens.
For example:
Give a JavaScript function that will output 0 and 1 tokens to render a rectangle on a 2d array.
Example output:
0 0 0 0 0 0
0 0 0 0 0 0
0 0 1 1 1 0
0 0 1 1 1 0
0 0 1 1 1 0
It responded:
function generateRectangle(rows, cols, rectStartRow, rectStartCol, rectHeight, rectWidth) {
// Create a 2D array filled with 0s
let array = Array.from({ length: rows }, () => Array(cols).fill(0));
// Fill the rectangle area with 1s
for (let i = rectStartRow; i < rectStartRow + rectHeight; i++) {
for (let j = rectStartCol; j < rectStartCol + rectWidth; j++) {
array[i][j] = 1;
}
}
return array;
}
function printArray(array) {
array.forEach(row => {
console.log(row.join(' '));
});
}
// Example usage:
let rows = 5;
let cols = 6;
let rectStartRow = 2;
let rectStartCol = 2;
let rectHeight = 3;
let rectWidth = 3;
let rectangleArray = generateRectangle(rows, cols, rectStartRow, rectStartCol, rectHeight, rectWidth);
printArray(rectangleArray);
------ And then I asked for a triangle instead, but still with square output example -----
function generateTriangle() {
const size = 10;
const field = Array.from({ length: size }, () => Array(size).fill(0));
for (let i = 0; i < size; i++) {
for (let j = 0; j <= i; j++) {
field[size - 1 - i][j] = 1;
}
}
return field;
}
function printField(field) {
for (let row of field) {
console.log(row.join(' '));
}
}
I don't think AI is a bubble. AI, unlike other tech businesses before it, is similar to heavy industry. You need a lot of CAPEX, Capital Expenditure. No more two people in the garage, since you probably need Nvidia GPU's that cost $30k a pop. End of day, all the money being invested into AI ends up within a few companies - Microsoft, Amazon, Google, and Nvidia. Due to cloud costs, and marketing costs.
This [1] is a clip from one of the most insightful interviews on the subject. Researchers from Anthropic and Google talking about their work. Full interview here [2] - it's a long listen but if you want to dive into the technical details of some of this stuff, it's great.
[1] - https://www.youtube.com/watch?v=UeI29-AdhQI
[2] - https://www.youtube.com/watch?v=UTuuTTnjxMQ