You certainly know more than I do about the various AI systems. I’ve barely messed with anything outside of LLMs.
As someone that rode a bicycle to commute full time for several years and was an amateur road racer until I was disabled by the 7th car that hit me, roads and hazards are very unpredictable and when things go wrong it happens fast. I’ve experienced a lot of the unbelievably stupid inadequacies in the infrastructure on the fringes. I do not think any system can be trained for these situations and the environment on small hardware. I see that as a direct conflict with capitalism that is motivated to minimize cost at the expense of hundreds of milliseconds. I don’t trust any auto manufacturer to use adequate hardware for the task.
The kinds of accuracy and efficiency required for running tensor math should still apply the same, as far as I know. I don’t know enough about other systems. Maybe they are not dependant on such heavy parallel vector math. Assuming they are based on tensors, it still results in the inadequate data throughput problem with the L2 to L1 caches of current compute architectures. That leaves the same hacks needed with GPU options as far as consumer level hardware.
I think we still need real dedicated tensor math hardware before anything like self driving cars will be ethically possible. With the ~10 year cycle of any new silicon we are probably 5-8 years out before a real solution to the problem… As far as I can tell.
I do not think any system can be trained for these situations and the environment on small hardware. I see that as a direct conflict with capitalism that is motivated to minimize cost at the expense of hundreds of milliseconds. I don’t trust any auto manufacturer to use adequate hardware for the task.
Don’t get me wrong, I do think this a valid cocnern, but I also don’t think we need to achieve “perfection” before this should be allowed on the road. I don’t trust every random human to pay attention, be able to see well enough, to notice the weird stuff, or to know how to react to it adaquetly. I think the sheer number of bicycle accidents alone show that our current systems and infrastructure don’t work for this.
If cars were fully autonomous, we could give them all strict rules. It would be easier to make up rules for how cyclists should be treated by moving vehicles, and riders could count on that being the case. We try to do this with road rules now, but many drivers just straight don’t listen. And this makes cycling hard because you never have any idea what any single driver is going to do.
A bit more soap boxy, but self driving cars should immediately abort any time they see shit they don’t recognize. Sure, in some ways, that’s easier said than done, but having good mechanisms for “how to safely stop when something weird happens” is critical here. And it makes a lot more of the “what do we do about weird shit in the street” a lot easier.
And to another point, maybe cars just need hard dedicated lanes that cyclists aren’t allowed in, more like a tram or city subway. And if people know to only cross at designated areas, it makes a lot of this a lot easier too.
And yes, capitalism makes a lot of this harder. I totally agree with you there. But this is something that should drastically save lives in the long run, but we might need to hold the fucking capitalist machine at bay while we do it.
I think we still need real dedicated tensor math hardware before anything like self driving cars will be ethically possible. With the ~10 year cycle of any new silicon we are probably 5-8 years out before a real solution to the problem… As far as I can tell.
This gets into the “too hard to articulate through text” zone, but I’ll just say that I think this is less far off than you think. For one, dedicated tensor hardware does exist and has existed for almost ten years at this point. It’s been commercially available for at least 5 IIRC. And for another, while lots of LLM type work is extremely intensive, lots of this object recognition type stuff is actually much easier. Lots of the “training” type stuff is the real expense, but it really only needs to be done “once” by the manufacturer and exported to each car. The amount of power needed by each car is much lower. And that type of stuff has been done pretty fast on consumer hardware for many years. It’s definitely still hard but it’s not like we’re talking about orders of magnitude out of reach where we need significant hardware breakthroughs - were essentially at the “this is mostly manageable with current hardware, but hardware evolves quickly anyway” stage.
having good mechanisms for “how to safely stop when something weird happens” is critical here. And it makes a lot more of the “what do we do about weird shit in the street” a lot easier.
I don’t know how vision may be different, but how do they know what they don’t know, differently that LLMs? That’s like the main problem with small LLMs, the next most probable token is always the next most probable token. Sure there is a bit more nuance available at lower levels, but the basic problem remains. The threshold of token choice is a chosen metric and that choice is heavily influenced by cost. If there was more cost effective tensor hardware I would have bought it. I mean I’m sure an FPGA could be used but if it is more cost effective than a GPU, I think we’d all be using it. I know there was some chip announced by IBM, but since when has IBM done anything remotely relevant in the consumer space. I think of IBM as a subsidiary of Red Hat more than anything else now.
You certainly know more than I do about the various AI systems. I’ve barely messed with anything outside of LLMs.
As someone that rode a bicycle to commute full time for several years and was an amateur road racer until I was disabled by the 7th car that hit me, roads and hazards are very unpredictable and when things go wrong it happens fast. I’ve experienced a lot of the unbelievably stupid inadequacies in the infrastructure on the fringes. I do not think any system can be trained for these situations and the environment on small hardware. I see that as a direct conflict with capitalism that is motivated to minimize cost at the expense of hundreds of milliseconds. I don’t trust any auto manufacturer to use adequate hardware for the task.
The kinds of accuracy and efficiency required for running tensor math should still apply the same, as far as I know. I don’t know enough about other systems. Maybe they are not dependant on such heavy parallel vector math. Assuming they are based on tensors, it still results in the inadequate data throughput problem with the L2 to L1 caches of current compute architectures. That leaves the same hacks needed with GPU options as far as consumer level hardware.
I think we still need real dedicated tensor math hardware before anything like self driving cars will be ethically possible. With the ~10 year cycle of any new silicon we are probably 5-8 years out before a real solution to the problem… As far as I can tell.
Don’t get me wrong, I do think this a valid cocnern, but I also don’t think we need to achieve “perfection” before this should be allowed on the road. I don’t trust every random human to pay attention, be able to see well enough, to notice the weird stuff, or to know how to react to it adaquetly. I think the sheer number of bicycle accidents alone show that our current systems and infrastructure don’t work for this.
If cars were fully autonomous, we could give them all strict rules. It would be easier to make up rules for how cyclists should be treated by moving vehicles, and riders could count on that being the case. We try to do this with road rules now, but many drivers just straight don’t listen. And this makes cycling hard because you never have any idea what any single driver is going to do.
A bit more soap boxy, but self driving cars should immediately abort any time they see shit they don’t recognize. Sure, in some ways, that’s easier said than done, but having good mechanisms for “how to safely stop when something weird happens” is critical here. And it makes a lot more of the “what do we do about weird shit in the street” a lot easier.
And to another point, maybe cars just need hard dedicated lanes that cyclists aren’t allowed in, more like a tram or city subway. And if people know to only cross at designated areas, it makes a lot of this a lot easier too.
And yes, capitalism makes a lot of this harder. I totally agree with you there. But this is something that should drastically save lives in the long run, but we might need to hold the fucking capitalist machine at bay while we do it.
This gets into the “too hard to articulate through text” zone, but I’ll just say that I think this is less far off than you think. For one, dedicated tensor hardware does exist and has existed for almost ten years at this point. It’s been commercially available for at least 5 IIRC. And for another, while lots of LLM type work is extremely intensive, lots of this object recognition type stuff is actually much easier. Lots of the “training” type stuff is the real expense, but it really only needs to be done “once” by the manufacturer and exported to each car. The amount of power needed by each car is much lower. And that type of stuff has been done pretty fast on consumer hardware for many years. It’s definitely still hard but it’s not like we’re talking about orders of magnitude out of reach where we need significant hardware breakthroughs - were essentially at the “this is mostly manageable with current hardware, but hardware evolves quickly anyway” stage.
I don’t know how vision may be different, but how do they know what they don’t know, differently that LLMs? That’s like the main problem with small LLMs, the next most probable token is always the next most probable token. Sure there is a bit more nuance available at lower levels, but the basic problem remains. The threshold of token choice is a chosen metric and that choice is heavily influenced by cost. If there was more cost effective tensor hardware I would have bought it. I mean I’m sure an FPGA could be used but if it is more cost effective than a GPU, I think we’d all be using it. I know there was some chip announced by IBM, but since when has IBM done anything remotely relevant in the consumer space. I think of IBM as a subsidiary of Red Hat more than anything else now.