How can I safely read a file line by line into an array with Bash 3.2?
Asked by
MellowCedar42
I have a Bash function that reads each line of a text file into an array by using read -r, but it relies on features available in newer Bash versions. On macOS, the system Bash is still version 3.2, and my attempts to make the function portable keep corrupting backslashes and other escaped characters. I suspect eval is interpreting the contents more than once. What is the safest way to read every line literally in Bash 3.2, or should I require users to install a newer Bash?
Answers
A normal loop such as while IFS= read -r line; do ...; done < "$file" works in Bash 3.2 and preserves backslashes, whitespace, and empty lines. The tricky part is assigning to an array whose name is supplied as a function argument; that is where eval becomes dangerous and causes the escaping problems. Also remember that a loop fed through a pipeline may run in a subshell, so redirect the file into the loop instead of using cat | while read.
Reply to this answer
The original function can appear to work in a small test, so make sure the test file includes literal backslashes, quotes, spaces, and possibly a final line without a newline. Those cases expose the problem. read -r is the right choice; the unsafe part is expanding the captured text through eval.
Reply to this answer
The backslashes are being interpreted by eval, not by read. If you need to keep the dynamic variable name, avoid constructing an array assignment with eval. In Bash 3.2, you can use printf -v with an indexed array element and a %s format so the line is stored literally. Otherwise, use a fixed array or pass the data another way instead of evaluating file contents as shell code.
Reply to this answer
The simplest practical solution on macOS is to install a current Bash through Homebrew or MacPorts and use that explicitly. Bash 3.2 lacks several conveniences available in modern releases, and there is no real need to keep fighting the system version if your script can declare a newer Bash dependency.
Reply to this answer
Add your answer
Answers are reviewed before publication.
MellowCedar42: That is what I ended up doing. Once I ran the script with the newer Bash, the original function behaved as expected.