How can I recover my PostgreSQL data in Kubernetes after a node restart?

0
11
Asked By CuriousCoder99 On

I'm currently running a small PostgreSQL database in my Kubernetes development cluster using Longhorn for storage. It's set up with a StatefulSet along with a Persistent Volume Claim (PVC), Persistent Volume (PV), and a Longhorn volume. After restarting the nodes, I noticed that my PostgreSQL pod came back without any data, even though the PV is set to retain mode and the Longhorn volume shows it's still over 150MB in size. I also tried restoring from a one-week-old Longhorn backup, but PostgreSQL still starts off as if it's a fresh install. Since the PV is in retain mode and I do have backups, is there any way for me to recover the actual PostgreSQL data files? I'll share my YAML configuration and volume details in the comments. Thanks!

5 Answers

Answered By CloudNinja43 On

If the PV is still intact, you might consider defining a PVC independently instead of having a claim directly within your StatefulSet. You can directly specify the `volumeName` of your PV in your new PVC. Keep in mind, that could involve a bit of manual work. You might want to look into CloudNativePG as well; it has several custom resource definitions (CRDs) to handle cluster definitions, image configurations, backups, and more, which could save you some effort in the long run.

Answered By DevGuru88 On

I wonder if you’ve updated to version 18, as it has a different folder structure. It's worth checking if a minor update of version 17 happened after your data was created. That could potentially explain the data loss.

Answered By FixItFelix On

Just a heads-up, I found a solution for my issue. The real problem was mounting the path `/var/lib/postgresql`, which caused PostgreSQL to create a temporary data directory. Fortunately, I had backups and mounted the disk in another VM (in read-only mode) to access the container's overlay filesystem. That's where I found my data.

Answered By DataWhisperer21 On

Did you delete the StatefulSet or the PVC at any point? If that's the case, your old PV might be orphaned and you'd need to reattach it to a new PVC to access your data properly.

Answered By TechieTommy On

How exactly did you restart the nodes? Since you have backups, restoring your database shouldn't be too difficult; you can use the `psql` command for that.

Related Questions

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.