I dunno, looks to me like AMD has better optimized drivers, which implies the nVidia has a good chunk of potential performance wasted due to driver overhead
That's not really the case.
The reason why NV has always had better DX9/11/OpenGL performance than AMD is because their drivers do a lot more behind the scenes to optimise their GPU performance.
So the draw call the app does ends up being sent to effectively a 'server' process which will be doing things like on-demand shader recompiling, caching, draw call rewriting and reordering (per game/engine), and correcting mistakes/issues that can happen.
AMD does a lot less of this aggressive optimisation which is why their performance on DX9/11/OpenGL has been traditionally lower, or doesn't increase like NVs would so often do.
This is also the reason why DX12/Vulkan saw AMD suddenly jump up performance wise, the driver basically got 'out of the way' and NV lost their 'server' advantage as those APIs are very much "do what I say" where as previous APIs are a bit more "do what I mean" - and NV has also been far more forgiving in that regard; my rule of thumb tends to be if it works on NV and not on AMD I've done something wrong, if it works on AMD and not on NV then NV have an optimisation which has done something wrong.
That's not to say that the drivers don't make a difference, however how much of a difference they can make and when they can do it is now heavily constrained.
Anyway, the point of all that is that this isn't "poorly optimised", it's just doing a lot of useful work which, yes, on lower performance CPUs might cause an issue.