I've been trying a new method for handling secrets in CI/CD pipelines using Python's AST (Abstract Syntax Tree) instead of usual regex approaches. Typically, detection of hardcoded secrets is done, but fixing them remains a manual effort. My goal is to see if we can automate the remediation process for common safe cases directly in the CI pipelines.
Here's what my experimental tool does:
- It scans Python repositories for hardcoded secrets.
- It analyzes assignment statements using Python's AST.
- It replaces detected secrets with environment variable references if the change is structurally safe.
- If the safety of the change can't be assured, it leaves the file intact and just reports the finding.
For instance, it transforms:
SENDGRID_API_KEY = "SG.live-abc123xyz987"
into:
SENDGRID_API_KEY = os.environ["SENDGRID_API_KEY"]
But it won't attempt to change something like:
token = "Bearer " + "sk-live-abc123"
I'm mainly looking for insights on these points:
- Should CI support automatic remediation for hardcoded secrets?
- Or should it only handle detection and leave fixes to developers?
- Are there teams already practicing automated fixes like this in their workflows?
I'm interested in hearing how others navigate this issue in real CI pipelines.
5 Answers
Been in AppSec for a while, and while your method has lower noise, be aware of the coverage gaps. There are always config files or YAMLs where secrets can sometimes hide, which need to be covered too.
Using AST for secret detection is smart! Regex can’t keep up and often leads to too many false positives. I’d suggest running your tool as a pre-commit hook rather than relying on it in CI—it’ll catch issues earlier in the dev cycle. Have you considered expanding support for other types of assignments, like kwargs?
Pre-commit seems like the better way to go, less overhead in CI. As for kwargs or attributes, that’s tricky! I’m trying to be cautious and only tackle the simplest cases right now.
I’m against tools making direct changes in the CI environment. It's safer to create a merge request for review instead. That way, you've got oversight on any changes being made.
Totally agree! Giving tools that kind of power feels risky. Merging changes with a suggested fix seems like the best compromise. Do you think teams are actually adopting this approach, or is it still mostly manual fixes after a build fails?
I don’t support CI pipelines modifying code automatically. It'd be better as a pre-commit hook instead. That way, developers can stay in control of their changes.
Yeah, that's the path I'm leaning towards too. CI changing code feels like a huge risk. Pre-commit hooks are much more trustworthy since they handle changes locally.
Clever approach using AST! It avoids false positive headaches that regex often brings. Most teams I worked with prefer a CI that blocks potential issues instead of automatically rewriting code—better to catch than to guess! The safety in your transformations means you might save a ton of time, as long as it’s all guaranteed to be safe.
Exactly! It's a delicate balance, and proving that the transformation is safe is key before trusting it in CI.
True, coverage is crucial. Right now, my tool is focused mainly on handling safe cases in Python, since expanding into YAML or Terraform could overcomplicate things.