Our development team really needs the capability to clone production databases into a separate development database for better debugging and testing. Currently, our development environment shares a database and a Keycloak tenant with our staging environment, which is not ideal. The devs want to replicate certain aspects of the production environment to a new dev database, and they're also asking for a dedicated Keycloak instance for development. However, I'm concerned about the various integrations we have—like with Google and Xero—and how this could all work without exposing sensitive data. I'm not sure what the best approach is or what questions I should even be asking. Any insights?
4 Answers
I completely agree that copying production data isn't the way to go unless you're careful. It opens up so many security issues that can quickly escalate to compliance violations. If the devs need real data to test against, creating a 'golden dataset' filled with mock data can offer a safe alternative. That way, your dev environment can stay protected without compromising on testing capabilities.
In most cases, directly copying production data to a development environment is a risky move, especially concerning sensitive customer data. Instead of cloning, consider creating a sanitized version of the production database. This would involve: 1. Taking a snapshot of the production database, 2. Masking or redacting any sensitive information while keeping the integrity of the data, 3. Loading this sanitized data into your development environment, and 4. Making sure that the development configurations are set up correctly to treat this database as a safe dev environment. This approach minimizes security risks while allowing the devs to work with data that reflects production.
Sounds like a solid plan! Just make sure to document the sanitization processes so everyone knows what data is safe to use.
We handle this by doing monthly sanitized extracts from production. We create a snapshot, then thoroughly scrub any sensitive fields before moving it to dev. It ensures that we have realistic data without infringing on customer privacy. If new sensitive fields are added in production, we have to ensure our sanitization processes adapt accordingly. Setting up a dedicated Keycloak instance for dev is also a good idea for testing configurations and upgrades without affecting production.
Has that process been efficient for you? There's a lot of overhead but worth it for keeping the dev environment safe.
You should definitely avoid this practice. Aside from the legal and compliance headaches, replicated production data can lead to unintended actions affecting real customer accounts. Focus on building robust monitoring and debugging tools that give the developers the visibility they need without exposing any sensitive information. And as mentioned before, using anonymized or obfuscated data is a much safer bet if you must do it at all.
Exactly! Plus, it can be a lot cheaper to generate mock data than to deal with potential security breaches.