It's the PSU, nothing wrong with it exactly, it's just coil whine (amongst other things probably). That's why replacing the PSU didn't fix it, both PSU's will be fine.
It's annoying but it can go away with time. I recently shell swapped a ps4 pro where the psu never made an audible sound, I gave it a quick clean out while I had it open and ever since then the damn thing won't shut up whining like that when certain games are played.
It's changing depending on different menu's, scenes and just general stuff on screen because of the different wattage requirements for each scene, like it's revving up and down depending on how much power it needs.
Some people get lucky and have PSU without whine, or less of it. That and the APU itself could also be making some noise too, happens on my laptop but normally can only hear it in quiet environments, especially the GPU when pushing it. I assume the same thing could be happening here.
So yeah it's normal, nothing to worry about unless your PSU actually starts failing, but hey at least you have a spare now.
(Either that, or this is the part where someone tells me I'm totally wrong and makes me look real stupid haha)
