I'm having trouble controlling how long curl takes for a file upload. I have this loop set up to attempt the upload three times, and here's what my code looks like:
```bash
for i in {1..3}
do
echo $'n'"`date`" Upload Attempt "$i" - $(hostname) >> "$log"
curl -s -S -v -u mgnewman:
--connect-timeout 30
--max-time 30
--pubkey ~/ .ssh/id_rsa.pub
-T $file $host >> $log
err=$?
echo $'n'"`date`" Upload Ended "$err" - $(hostname) >> "$log"
touch /home/pi/webcam/webcam.webp
if [ $err -eq 0 ]; then
break
fi
done
```
I know that I'm using a loop instead of curl's retry option because I need to refresh a file to keep a watchdog from rebooting the machine if it goes unnoticed for ten minutes.
I'm trying to limit each curl operation to 30 seconds but it seems like it's taking longer. My logs show that the upload attempt started at around 23:37 and ended around 23:48, which seems way off. What am I missing?
4 Answers
From what I see, after your client authenticated, curl timeout occurred because the server isn’t receiving data. Consider using SSH or SFTP instead of curl for these uploads; they’re more suited for that purpose. If you want strict time limits, you might try different mechanisms in your script to enforce that without relying solely on curl's capabilities.
Hang tight with the loop, but make sure to fine-tune everything. You might want to log how long each curl command runs. This could help you isolate what's causing delays and fix it!
It looks like your curl is actually timing out at 30 seconds, but the long gap you see is from the overall loop, not from curl. Here's the deal:
1. `--connect-timeout 30` sets a limit for the connection phase only.
2. `--max-time 30` limits the full curl operation time.
From your logs, curl is telling you that the timeout happens after 30 seconds, which means the upload attempt failed due to a timeout, not your script's fault. The timestamps don't match up because of the time taken in your loop. You should consider logging the start and end times around the curl command to pinpoint any delays inside the loop that might be adding time beyond your expected 30 seconds.
That could help! Just be mindful that if the upload is still failing due to network issues, retries won’t fix the root problem.
If curl isn't working for your case, try doing the uploads via SFTP or SCP. You’ll have much fewer issues and can usually troubleshoot problems better. Don't forget, you could enforce time limits in other ways, like using a shell timeout. That way, you're not solely dependent on the program's internal timeout features.
Great clarification! So, should I just switch to curl's retry option for more consistent timings?