Linux / shell

Unquoted variables — when a script silently changes the arguments

Written and reviewed by Sahil Srivastav

Word splittingPathname expansionBash arrays
argc=2
<quarterly>
<report.csv>

What this error actually means

The header is the output of the small argument-inspection example below. A value containing quarterly report.csv is intended to be one filename. When expanded unquoted as $file, Bash can split it at IFS separators into two words. Those words can then undergo pathname expansion, so wildcard characters stored in data may match files in the current directory.

Double quotes preserve the result of a parameter expansion as one argument, including spaces and wildcard characters. They also preserve an empty string as an empty argument. An unquoted empty expansion can disappear completely, changing the number and positions of arguments received by the program.

Shell quoting is syntax applied while parsing the command. Quote characters stored inside a variable do not become shell syntax during ordinary expansion. A string such as options='--label "nightly run"' is therefore not a reliable argument list. Using eval to reinterpret it introduces a second parse and can turn data into executable shell code.

Causes, most common first

  1. 1A path or user-provided value is expanded without quotes. Commands such as cp $source $destination expose data to splitting and globbing before cp starts. The receiving program cannot reconstruct the original boundaries because the shell has already changed them.
  2. 2An argument list is stored as one string. Several options plus their values are not a single scalar. Expanding the string unquoted breaks values containing spaces; expanding it quoted passes the entire list as one argument. A Bash array preserves the intended list structure.
  3. 3A loop parses line-oriented command output as words. for file in $(find ...) applies splitting and expansion to a whole output string. Filenames can contain spaces and newlines, so textual reconstruction loses information. Use find -exec or a NUL-delimited interface for filesystem names.
  4. 4Quoting is mistaken for option validation. Even a correctly quoted value beginning with a dash can be interpreted as an option by the receiving utility. Where supported, use -- to end option parsing. Quoting preserves boundaries; it does not define the utility’s argument semantics.

When you see it

  • A job works with simple names but fails for a customer file containing spaces
  • An asterisk in input selects unrelated files from the current directory
  • Empty optional input shifts positional arguments or changes command behaviour
  • Adding quote characters inside a variable does not preserve the intended boundaries

How to diagnose it

Step 1

Inspect the argument vector directly

Run this harmless example in Bash. It prints the count and each argument separately, making the missing boundary visible. Repeat with "$file" to see one argument containing the original space.

show_args() { printf 'argc=%s\n' "$#"; printf '<%s>\n' "$@"; }
file='quarterly report.csv'
show_args $file

Step 2

Use a shell-aware static check

If ShellCheck is installed, it flags many unquoted expansions and array mistakes. Review the warning in context rather than suppressing it because current fixtures contain only simple names. The tool does not know your business contract for empty or missing inputs.

shellcheck ./process-files.sh

Step 3

Test wildcard and empty values in isolation

Use the same show_args helper and a disposable directory. Observe how the current directory affects an unquoted wildcard and how an empty value changes the argument count. No files need to be modified to reproduce the parsing problem.

file='*.csv'
show_args "$file"
file=''
show_args "$file"

Step 4

Inspect how the downstream tool parses arguments

After boundaries are correct, read the utility’s option rules and confirm whether it supports --. A leading-dash filename and an absent required operand are semantic cases the script must handle separately from whitespace.

The fix

Quote scalar expansions used as arguments: "$source", "$destination" and "$value". Forward a caller’s argument list using "$@", which preserves each argument separately. Do not replace it with "$*" when the receiving command needs the original boundaries.

Use a Bash array for a command assembled from multiple arguments. Append each option and value as separate array elements, then expand with "${args[@]}". If the script must be portable POSIX sh, use positional parameters with set -- rather than assuming Bash arrays are available.

Validate required values before invoking the utility. The example requires two arguments, validates the source as a regular file, and uses -- for GNU cp. It still leaves normal overwrite policy to the caller; quoting is not a complete file-transfer workflow. Choose destination and overwrite rules according to the application.

For filesystem iteration, avoid command substitution over names. find -exec passes paths directly, and NUL-delimited streams preserve names that include newlines. Do not attempt to escape every unusual character manually or use eval to resurrect quotes embedded in data.

#!/bin/bash
set -euo pipefail
source_file=${1:?usage: copy-report SOURCE DESTINATION}
destination_file=${2:?usage: copy-report SOURCE DESTINATION}
[[ -f "$source_file" ]] || {
  printf 'Source is not a regular file: %s\n' "$source_file" >&2
  exit 1
}
copy_args=(-- "$source_file" "$destination_file")
cp "${copy_args[@]}"

How to stop it coming back

  • Include spaces, empty strings, literal wildcard characters, leading dashes and newlines in argument-handling fixtures.
  • Keep command names and argument lists separate from data; avoid building executable shell text from user input.
  • Run ShellCheck in script review and explain the few intentional unquoted expansions locally.

Practise production debugging in a real repository

Reading about a failure and reproducing one are different skills. Gronex ships broken backend repositories with failing test suites that encode the real invariant, so you debug from evidence instead of memorising symptoms.

FAQ

Does set -f solve this?

It disables pathname expansion but does not prevent word splitting or preserve an unquoted empty value. Quoting each scalar argument handles the intended boundary directly without changing glob behaviour for the entire script.

Are variables always split unless quoted?

No. Bash has contexts with different expansion rules, including assignment words and [[ ... ]]. The dangerous case discussed here is ordinary unquoted expansion into command arguments. Follow the rules of the actual context rather than mechanically adding escapes.

Why does a quoted wildcard stop matching files?

Because the wildcard is now literal data. If the script intentionally selects files by a pattern, write that selection explicitly using a glob or find predicate. A value representing one pathname should not accidentally become a pattern.

Related

Argument list too longShell CSV processing interview

Other errors engineers hit next to this one

Full error and symptom index →