How Can I Tell if My GPU or PCIe Port is Faulty?

It seems more likely that the power supply is the issue here rather than the GPU or PCIe slot. When a system shuts down unexpectedly and won't restart until the GPU is removed, it's often a sign that the power supply can't handle the load or that some safety mechanism is kicking in. A faulty GPU can cause similar issues, but given your previous power supply problems, I’d be suspicious of that again. PCIe slot failures are pretty rare and usually wouldn’t cause a whole system shutdown.
You could try using your GPU in another PC to see if the same issues arise. If it works fine in another setup, then you'll want to look closer at the power delivery in your current system. Make sure you're using the correct PCIe cables without any splitters, and that everything is seated securely. Based on what you've described, I'd bet there's still a power delivery problem here.

It would help if you could share your system specs, especially the type of power supply you just installed. It's important to identify if it meets the requirements for your build. Also, it's generally better not to jump to conclusions about others' expertise until we have all the facts. If you ended up with a subpar power supply, that could definitely be part of the problem.