I have several folders in an S3 bucket named with dates, such as 2025-12-10/, 2025-12-11/, and 2025-12-12/. I want to automate copying selected folders to a local disk. I tried using a shell loop, but I'm unsure how to provide the list of dates correctly:
for i in $("2025-12-10"); do mkdir "$i" && aws s3 cp --recursive "s3:///$i" "./$i"; done
What is the correct way to write this?
3 Answers
If the dates follow a predictable range, generate them separately and feed them into the loop. For example, with GNU date:
start=$(date -d "2025-12-29" +%s)
for ((n=0; n<5; n++)); do
date=$(date -d "@$((start + n * 86400))" +%F)
mkdir -p "$date"
aws s3 cp --recursive "s3:///$date" "./$date"
done
This handles month and year boundaries instead of manually constructing potentially invalid dates.
Another option is aws s3 sync from the bucket root to the local directory, using --exclude and --include patterns to select the date folders. That can be convenient when copying a larger set, but for a small explicit list, a loop is clearer and easier to control.
Put the date values directly in the loop, separated by spaces. You can also use mkdir -p so the command succeeds when the destination already exists:
for date in 2025-12-10 2025-12-11 2025-12-12 2025-12-13; do
mkdir -p "$date"
aws s3 cp --recursive "s3:///$date" "./$date"
done
The $(...) in the original command is command substitution. It tries to execute 2025-12-10 as a command and then loops over that command’s output, which is not what you want.
That makes sense—I was treating the date like a value, but the shell was trying to run it as a command.