It's mostly hardware design limitations. GPU hardware is way more diverse than CPU hardware, so you need to distribute shaders as either source code or a portable intermediate representation, in both cases compiled at runtime for the user's actual GPU. There are also many fixed function parts of the graphics pipeline with wildly different behavior on different hardware which you need to abstract over. And it all runs in parallel with the CPU, often with a separate pool of RAM, so you need implicit or explicit synchronization and data transfer which adds a lot of complexity especially in error cases.

If you really want to see why all the cruft is there, and how much you could remove if you dropped support for older hardware, look no further than Sebastian Aaltonen's "No Graphics API" blog post (which he actually implemented too): https://www.sebastianaaltonen.com/blog/no-graphics-api/

I think most of it comes down to 'historical accidents' tbh.

E.g. if CPUs would have closed ISAs and heavily guarded IPs, you'd also need a 'driver' and an 'OpenCPU API' to make any use of them (of course the public ISA of a CPU is essentially the driver interface baked into the hardware).

The other big difference is focus on decent single-thread performance for CPUs versus performing thousands of small tasks in parallel on GPUs, but each one slowly. This difference causes a different programming model (e.g. running an entire application in a single 'main' function from start to finish, versus spawning small specialized tasks which are running across hundreds of threads).

And the specific problem with Vulkan is that it tried to solve two conflicting requirements: on one hand being an explicit low-level API without 'magic abstractions', on the other hand supporting very different GPU architectures (especially the mobile GPU zoo). The resulting mess was quite predictable to be honest...

part of the reason is historical cruft & history path dependency. GPUs didn't start as general computing device for parallelism but a device for the specific purpose of accelerating graphics with very specific fixed functions for that job and only that. with time that fixed pipeline became more complex and more configurable, not fixed/more flexible and hacks were in parallel used to make use of it for tasks outside graphics those hacks were later made into something more formal (CUDA/OpenCL).

CUDA being the winner (closed source, not really usable on non-nvidia GPUs) between those 2 and nvidia not wanting to open their competitive advantage to the competition for the niche of non gamings uses / scientific computing (and later AI, which became a huge win for nvidia).

vulkan came around after as this open thing (based on a previous attempt by AMD called mantle) that's trying to break down the leaky abstractions and the divide between language for graphics & video games and separate language/frameworks for gpu non-graphic computing with the promise of giving more control and more room for squeezing performance from the hardware at the cost of more complexity with the idea that you'll probably build your own abstractions on top of it for your subset of applications.

If GPU were from the get go designed as a parallelism tool first and video game tools second the hardware architecture and the language would probably be different than today and simpler (and if the competition between the duopoly didn't sabotage things)

don't forget that the drivers for GPUs would until recently get updates to patch specific video games for performance issues (not sure if they still do that ?)

One thing that helped me understand it was to reframe it as a network application, where you have a client running on the CPU and a server running on the GPU (the shaders).

A bunch of the complexity comes from having to write both client and server, and the communication protocol between them, then some more comes from the client being responsible for spinning up the server. As with every network service, client/server traffic is the slowest part of the architecture, so a bunch of complexity then comes from having to minimise the amount of network calls (or draw calls, in this case).

Then you hit the part where all of this is performance critical (soft-realtime requirements), so the whole thing is geared towards “writing your own userspace IP stack in C++” rather than “writing a flask application in Python”.

The thing that makes it "hard" is that it's all indirect. You compile a program for the GPU (a shader). Then you have to setup data (textures and buffers) for it. Then, you need to spend a bunch of time telling the GPU how to use that data, offsets where each piece of data is, what formats the data is in,, how to advance to the next piece of data, etc.. etc..

There are a few libraries that try to abstract over all this indirection but all the graphics APIs themselves are lower-level so you're left to do all of that yourself.

https://docs.swmansion.com/TypeGPU/

https://github.com/google/toucan

If you care about every last bit of performance, CPUs are difficult to program for as well, possibly more difficult than a GPU.

Generally GPUs punish you more by bad performance if you don't optimise your code.

It's not more complicated, you have things like CUDA which is basically C/C++ that compiles to GPU.

But GPUs have thousands of cores, which means if you want any benefit, the smallest data type you must use is something like array[n * 1024], or your GPU is wasted doing nothing. So you must shape your programs into that memory model or you're just wasting your time (and GPU).

That's >50% of the problem.

Vulkan, OpenGL and DirectX make GPU programming more difficult than it has to be for multiple reasons, some of which other people here have mentioned. They are also low level, focused on high performance, and are primarily designed and written for game engines. Making them easy to learn for newcomers is pretty low on the priority list. ShaderToy makes it easy by taking care of everything except the GLSL, you might like it. You can also try CUDA, which is generally a lot easier than Vulkan. For a really easy/fun time with CUDA, try Warp - it’s CUDA in Python.

It's partially historical; I'm sure if we redesigned GPGPU today from the ground up, we'd have an entirely different pipeline. And partially due to hardware vendor lock-in; you can't really design an alternative to Vulkan/d3d12 even if you had the skill and drive to do so.

But also partly in the fact that the GPU is specialized for certain operations over a CPU. It wants and need to obtain data in a different way from a CPU to maximize throughput. So language constructs are designed around making sure developers stay on this fast path. The complexity can be seen as some intentional gatekeeping to signal that this kind of programming is only for specific kinds of problems.