Hi! This is my first time using Bash, and I'm trying to run a Python script on every file in /data/folder. I want each result to be written to another folder while keeping the input filename, but I'm not sure what to put after --out. I also want to make sure the loop syntax, quoting, and command-line options are correct.
#!/bin/bash
for i in /data/folder/*; do
/folder/munge_sumstats.py
--sumstats "$i"
--chunksize 500000
--snp rsID
--out "another/folder/..."
done
3 Answers
Another clear option is to use basename to get only the input file’s name:
for i in /data/folder/*; do
name=$(basename "$i")
/folder/munge_sumstats.py
--sumstats "$i"
--chunksize 500000
--snp rsID
--out "another/folder/$name"
done
If the input files have a suffix that should be replaced, you can strip it and add a new one, for example: name=$(basename "$i" .old); then use --out "another/folder/$name.new".
Use regular straight quotes and write the options with two hyphens, such as --sumstats and --out. To keep the original filename while changing its directory, you can remove everything through the last slash with parameter expansion:
for i in /data/folder/*; do
/folder/munge_sumstats.py
--sumstats "$i"
--chunksize 500000
--snp rsID
--out "another/folder/${i##*/}"
done
The quotes around "$i" are important in case a filename contains spaces. Make sure the output directory already exists.
I only used the odd spacing because I was typing this on my phone. Thanks, this explanation helped a lot!
Check the paths carefully as well. The script path and every input/output path should be valid from the directory where you run the Bash script. Using absolute paths can avoid confusion, especially if the Python program resolves files relative to the current working directory.
That makes sense too—thank you!