With a CPU or even a GPU, there is a bunch of inefficiencies for every task as they’re designed to be able to do pretty much anything - your H265 media decoder isn’t going to be doing much when you’re keeping a running sum of the number of a certain type of bond in a list of chemicals
With ASICs and a lesser extent FPGAs, you can make it so every single transistor is being used at every moment which makes them wildly efficient for doing a single repetitive task, such as running statistical analysis on a huge dataset. This is because rather than being limited by the multiprocessing ability of the CPU or GPU, you can design the “program” to run with as much multiprocessing ability as is possible based on the program, meaning if you stream one input per clock cycle, after a delay you will get one input per clock cycle out, including your update function so long as it’s simple enough (eg moving average, running sum or even just writing to memory)
This is one specific application of FPGAs (static streaming) but it’s the one that’s relevant here
So it sounds like we’re designing the instruction pipeline for maximum parallelism for our task. I was surprised to learn that the first commercial FPGAs were available as early as the '80s. I can see how this would have been an extremely effective option before CUDA became available.
With a CPU or even a GPU, there is a bunch of inefficiencies for every task as they’re designed to be able to do pretty much anything - your H265 media decoder isn’t going to be doing much when you’re keeping a running sum of the number of a certain type of bond in a list of chemicals
With ASICs and a lesser extent FPGAs, you can make it so every single transistor is being used at every moment which makes them wildly efficient for doing a single repetitive task, such as running statistical analysis on a huge dataset. This is because rather than being limited by the multiprocessing ability of the CPU or GPU, you can design the “program” to run with as much multiprocessing ability as is possible based on the program, meaning if you stream one input per clock cycle, after a delay you will get one input per clock cycle out, including your update function so long as it’s simple enough (eg moving average, running sum or even just writing to memory)
This is one specific application of FPGAs (static streaming) but it’s the one that’s relevant here
So it sounds like we’re designing the instruction pipeline for maximum parallelism for our task. I was surprised to learn that the first commercial FPGAs were available as early as the '80s. I can see how this would have been an extremely effective option before CUDA became available.