I'm really struggling with creating on-demand memory dumps for my production pods that are running on the openjdk/jdk:21-distroless image. My application is built on Spring and the security context is set with a specific configuration, including a runAsUser of 1000. I've tried various commands using kubectl debug, but the closest I got was using a specific command that attempts to launch a bash shell in a separate image, only to face issues with file permissions — the process can't create a PID file because it's trying to write to /tmp, which isn't accessible due to the permissions set for the user in the pod. Is it even possible to get a proper memory dump with my current setup, or have I completely locked myself out?
3 Answers
You might consider adding a volume at /tmp and adjusting the permissions so your specified user can write there. Alternatively, if you're following best practices with a read-only root file system, you'll need to mount a writable volume for the dump. The tools you're using shouldn’t be hardcoded to just /tmp, so try checking the configuration.
The /tmp directories can differ since they are on separate file systems. You could go with the previous suggestion or try using Spring Boot Actuator to create a heap dump by exposing a heapdump endpoint. Another option is using the jmap tool with a force flag, but keep in mind that you’d need root access for ptrace on user 1000.
Are you just looking for the heap dump or do you need a full RAM dump? If it’s just the heap, you can check out the Spring Boot Actuator documentation for how to do that through the actuator endpoints.
Unfortunately, a heap dump alone won’t cut it for my needs.