bloggb

Argument list too long error with GNU parallel

When the number of arguments is too long, GNU parallel raises a Argument list too long error.

A possible solution is to use the find command:

    find directory -name "*.dat" |\
    parallel -k --eta "grep Chi {} | awk -F'Chi = ' '{print $2}'" > out.txt

instead of the classical syntax:

    ls directory/*.dat |\
    parallel -k --eta "grep Chi {} | awk -F'Chi = ' '{print $2}'" > out.txt

or

    parallel -k --eta "grep Chi {} | awk -F'Chi = ' '{print $2}'" ::: directory/*.dat > out.txt
If you want to ask me a question or leave me a message add @bougui505 in your comment.