Is there a downside to using Flatpaks for most apps?

There isn't really a point where using too many Flatpaks becomes a problem by itself. If the apps work well for you and disk space isn't tight, there's no need to switch just for the sake of it.

They can take more space because they use runtimes and dependencies separate from system packages. Some runtimes are shared between apps, though, so it isn't necessarily a full set of dependencies for every single app.

The practical annoyance is integration. A sandboxed app may need permission to see a particular folder or use a device, and themes don't always match the rest of the desktop. If something works in a native package but not in the Flatpak, permissions are one thing worth checking.

That separation can also be useful. An app can bring dependencies that differ from the ones Fedora provides, which may avoid some package conflicts or broken system dependencies.

Keep an eye on updates too. Flatpak apps and runtimes have their own updates, separate from Fedora's regular packages. That doesn't make them hard to maintain, but it's another set of things to keep current.

Performance is worth checking per app rather than assuming they'll all be slower. Some may take longer to start, and running apps that use different runtimes can increase memory use. Whether you notice depends on the apps and your machine.

If one feels sluggish, compare it with the Fedora package before changing everything else. There's not much reason to optimize a problem you aren't seeing.

The package itself matters as much as the format. A Flatpak is maintained and configured by whoever publishes it, so its defaults and available features may differ from Fedora's version. If an app behaves oddly or is missing a feature, trying the other package is a reasonable troubleshooting step.