Hacker News
points by avaer 3 days ago | 0 comments
It's not more complicated, you have things like CUDA which is basically C/C++ that compiles to GPU.

But GPUs have thousands of cores, which means if you want any benefit, the smallest data type you must use is something like array[n * 1024], or your GPU is wasted doing nothing. So you must shape your programs into that memory model or you're just wasting your time (and GPU).

That's >50% of the problem.

[–] proc0 3 days ago
Right, that makes sense. There's a ton of overhead in the setup though, I'm assuming that's because everything runs on the CPU by default. Is this then an OS design choice? Thanks for your input.
[–] avaer 3 days ago
There can be overhead, though GPU memory + drivers are some of the most optimized software in the world.

Some OS/hardware have separate GPU memory (Nvidia), some have unified memory (Macos, modern consoles), but moving memory is usually not the problem.

Things don't "run on CPU by default", it's up to the programmer where to run it. But the CPU and GPU paradigm of "running a program" are very different, since on GPU you're running SIMD (i.e. running your code on arrays) or else there is no point to use a GPU.

[–] tvbusy 3 days ago
For me, if-branching is the hardest to wrap my head around. "If"s are so natural to CPU programming that it's second nature to any developer but on GPU, it's expensive.
[–] __del__ 3 days ago
instead of iterating over an array and if-branching to one of two paths for each item, conceive of your conditionals as logic operations which test for set membership (i.e. "is this array item a member of the set i care about?") and with this array mask of "do the thing or don't do the thing" flags, you can mutate your data 0 or 1 times.

the key for me is thinking in terms of sets—much of your conditional logic is probably about sorting things into sets anyway.