Shell Script Hack #10: Parameter Expansion – String Magic Without External Tools

You’re constantly typing ${variable} but did you know Bash has powerful built-in string manipulation that can replace, extract, default values, and transform text without calling external commands? Let me show you parameter expansion tricks that will change how you write shell scripts.

The Problem

You need to manipulate a filename, extract a substring, or provide default values. The traditional approach uses multiple external commands:

filename="document.txt"
basename=$(echo "$filename" | cut -d. -f1)
extension=$(echo "$filename" | cut -d. -f2)
# Slow, clunky, spawns subprocesses

The Hack: Parameter Expansion

Bash has built-in parameter expansion that’s fast and elegant:

filename="document.txt"
basename="${filename%.*}"    # document
extension="${filename##*.}"  # txt

No external commands, instant results, pure Bash!

Default Values

Use Default if Unset

# ${var:-default} - use default if var is unset or empty
PORT=${PORT:-8080}
echo "Server running on port $PORT"

# If PORT is not set, uses 8080
# If PORT=3000, uses 3000

# Real-world usage
DB_HOST=${DB_HOST:-localhost}
DB_PORT=${DB_PORT:-5432}
DB_NAME=${DB_NAME:-myapp}

echo "Connecting to $DB_HOST:$DB_PORT/$DB_NAME"

Assign Default if Unset

# ${var:=default} - assign default if var is unset
LOG_LEVEL=${LOG_LEVEL:=INFO}
# Now LOG_LEVEL is set to INFO if it wasn't before

# Useful for script configuration
CONFIG_FILE=${CONFIG_FILE:=/etc/app/config.yml}
DATA_DIR=${DATA_DIR:=/var/lib/app}

Error if Unset

# ${var:?error message} - exit with error if unset
API_KEY=${API_KEY:?API_KEY environment variable is required}
DB_PASSWORD=${DB_PASSWORD:?Database password not set}

# Script exits with error if these aren't set

Use Alternative Value

# ${var:+alternative} - use alternative if var IS set
DEBUG=${DEBUG:+--verbose}
# If DEBUG is set (to anything), DEBUG becomes "--verbose"

# Usage
echo "Running: ./app $DEBUG"

String Removal

Remove from Beginning

path="/usr/local/bin/script.sh"

# ${var#pattern} - remove shortest match from beginning
echo "${path#*/}"        # usr/local/bin/script.sh

# ${var##pattern} - remove longest match from beginning
echo "${path##*/}"       # script.sh (basename!)

# Real examples
url="https://example.com/page"
echo "${url#https://}"   # example.com/page

prefix="test_"
name="test_user_data"
echo "${name#$prefix}"   # user_data

Remove from End

filename="document.backup.tar.gz"

# ${var%pattern} - remove shortest match from end
echo "${filename%.*}"    # document.backup.tar

# ${var%%pattern} - remove longest match from end
echo "${filename%%.*}"   # document

# Get file extension
extension="${filename##*.}"  # gz

# Remove extension
basename="${filename%.*}"    # document.backup.tar

Pattern Replacement

# ${var/pattern/replacement} - replace first match
text="hello world hello"
echo "${text/hello/hi}"      # hi world hello

# ${var//pattern/replacement} - replace all matches
echo "${text//hello/hi}"     # hi world hi

# ${var/#pattern/replacement} - replace at beginning
echo "${text/#hello/hi}"     # hi world hello

# ${var/%pattern/replacement} - replace at end
echo "${text/%hello/hi}"     # hello world hi

# Real-world examples
path="/home/user/docs/file.txt"
echo "${path//\//_}"         # _home_user_docs_file.txt

url="http://example.com"
echo "${url/#http:/https:}"  # https://example.com

Case Modification

text="Hello World"

# ${var^} - uppercase first character
echo "${text^}"      # Hello World

# ${var^^} - uppercase all
echo "${text^^}"     # HELLO WORLD

# ${var,} - lowercase first character
echo "${text,}"      # hello World

# ${var,,} - lowercase all
echo "${text,,}"     # hello world

# ${var~} - toggle case of first character
echo "${text~}"      # hello World

# ${var~~} - toggle case of all
echo "${text~~}"     # hELLO wORLD

Substring Extraction

# ${var:offset:length}
text="Hello World"

# From position 6, length 5
echo "${text:6:5}"       # World

# From position 6 to end
echo "${text:6}"         # World

# Last 5 characters (note the space before minus)
echo "${text: -5}"       # World

# All except last 6 characters
echo "${text:0:-6}"      # Hello

# Real examples
timestamp="2025-10-31T20:00:00"
date="${timestamp:0:10}"     # 2025-10-31
time="${timestamp:11:8}"     # 20:00:00

String Length

# ${#var}
name="John Doe"
echo "${#name}"          # 8

password="secret"
if [ ${#password} -lt 8 ]; then
    echo "Password too short"
fi

# Validate input length
username="admin"
if [ ${#username} -ge 3 ] && [ ${#username} -le 20 ]; then
    echo "Valid username length"
fi

Real-World Use Cases

File Path Manipulation

#!/bin/bash

filepath="/var/log/app/debug.log"

# Get directory
dir="${filepath%/*}"           # /var/log/app

# Get filename
filename="${filepath##*/}"     # debug.log

# Get basename (no extension)
basename="${filename%.*}"      # debug

# Get extension
extension="${filename##*.}"    # log

# Change extension
newfile="${filepath%.*}.txt"   # /var/log/app/debug.txt

echo "Directory: $dir"
echo "Filename: $filename"
echo "Basename: $basename"
echo "Extension: $extension"
echo "New file: $newfile"

URL Parsing

#!/bin/bash

url="https://user:pass@example.com:8080/path/to/page?query=value#anchor"

# Remove protocol
temp="${url#*://}"             # user:pass@example.com:8080/path/to/page?query=value#anchor

# Extract credentials
creds="${temp%%@*}"            # user:pass
username="${creds%%:*}"        # user
password="${creds#*:}"         # pass

# Extract host and port
hostport="${temp#*@}"
hostport="${hostport%%/*}"     # example.com:8080
host="${hostport%%:*}"         # example.com
port="${hostport##*:}"         # 8080

# Extract path
path="${temp#*@}"
path="${path#*/}"
path="/${path%%\?*}"           # /path/to/page

echo "Username: $username"
echo "Password: $password"
echo "Host: $host"
echo "Port: $port"
echo "Path: $path"

Configuration with Defaults

#!/bin/bash

# Application configuration with sensible defaults
APP_NAME=${APP_NAME:-myapp}
APP_ENV=${APP_ENV:-development}
APP_PORT=${APP_PORT:-3000}
APP_HOST=${APP_HOST:-0.0.0.0}

DB_HOST=${DB_HOST:-localhost}
DB_PORT=${DB_PORT:-5432}
DB_NAME=${DB_NAME:-${APP_NAME}_${APP_ENV}}
DB_USER=${DB_USER:-${APP_NAME}}
DB_PASSWORD=${DB_PASSWORD:?Database password required}

REDIS_HOST=${REDIS_HOST:-localhost}
REDIS_PORT=${REDIS_PORT:-6379}

LOG_LEVEL=${LOG_LEVEL:-INFO}
LOG_DIR=${LOG_DIR:-/var/log/${APP_NAME}}

# Display configuration
cat << EOF
Application Configuration:
  Name: $APP_NAME
  Environment: $APP_ENV
  Host: $APP_HOST:$APP_PORT

Database:
  Host: $DB_HOST:$DB_PORT
  Database: $DB_NAME
  User: $DB_USER

Redis:
  Host: $REDIS_HOST:$REDIS_PORT

Logging:
  Level: $LOG_LEVEL
  Directory: $LOG_DIR
EOF

Backup Script with Date Formatting

#!/bin/bash

SOURCE_DIR="/var/www/html"
BACKUP_BASE="/backup"

# Get timestamp
timestamp=$(date +%Y%m%d_%H%M%S)

# Extract date components
year="${timestamp:0:4}"
month="${timestamp:4:2}"
day="${timestamp:6:2}"

# Create organized backup structure
BACKUP_DIR="${BACKUP_BASE}/${year}/${month}"
mkdir -p "$BACKUP_DIR"

# Create backup filename
backup_file="${BACKUP_DIR}/backup_${timestamp}.tar.gz"

echo "Creating backup..."
tar czf "$backup_file" "$SOURCE_DIR"

echo "Backup created: $backup_file"

# Clean old backups (keep last 30 days)
find "$BACKUP_BASE" -name "backup_*.tar.gz" -mtime +30 -delete
echo "Old backups cleaned"

Advanced Patterns

Bulk Rename Files

#!/bin/bash

# Rename all .txt files to .md
for file in *.txt; do
    mv "$file" "${file%.txt}.md"
done

# Add prefix to files
for file in *.jpg; do
    mv "$file" "photo_${file}"
done

# Replace spaces with underscores
for file in *\ *; do
    mv "$file" "${file// /_}"
done

# Convert to lowercase
for file in *; do
    mv "$file" "${file,,}"
done

Environment Variable Validation

#!/bin/bash

# Validate required environment variables
required_vars=(
    "DATABASE_URL"
    "API_KEY"
    "SECRET_KEY"
)

for var in "${required_vars[@]}"; do
    # This will exit if variable is not set
    : "${!var:?ERROR: $var is not set}"
done

# Validate format
email=${EMAIL:?Email is required}
if [[ ! "$email" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}$ ]]; then
    echo "Invalid email format"
    exit 1
fi

String Sanitization

#!/bin/bash

# Clean user input
user_input="Hello! @#$ World%^&"

# Remove special characters
clean="${user_input//[^a-zA-Z0-9 ]/}"
echo "$clean"  # Hello  World

# Create safe filename
filename="My Document (2025).txt"
safe_filename="${filename//[^a-zA-Z0-9._-]/_}"
echo "$safe_filename"  # My_Document__2025_.txt

# URL encoding (basic)
text="hello world"
encoded="${text// /%20}"
echo "$encoded"  # hello%20world

Performance Comparison

Parameter expansion vs external commands for 10,000 operations:

• Parameter expansion: 0.5 seconds
• Using cut/sed/awk: 45 seconds
• Using basename: 38 seconds

Parameter expansion is 90x faster!

Complete Example: File Processor

#!/bin/bash

process_file() {
    local filepath="$1"
    
    # Validate file exists
    [[ -f "$filepath" ]] || {
        echo "Error: File not found: $filepath"
        return 1
    }
    
    # Extract components
    local dir="${filepath%/*}"
    local filename="${filepath##*/}"
    local basename="${filename%.*}"
    local extension="${filename##*.}"
    
    # Generate output filename with timestamp
    local timestamp=$(date +%Y%m%d_%H%M%S)
    local output="${dir}/${basename}_processed_${timestamp}.${extension}"
    
    # Process based on extension
    case "${extension,,}" in
        txt|log)
            # Remove blank lines and process
            grep -v '^$' "$filepath" > "$output"
            ;;
        jpg|png)
            # Resize image
            convert "$filepath" -resize 800x600 "$output"
            ;;
        *)
            echo "Unsupported file type: $extension"
            return 1
            ;;
    esac
    
    echo "Processed: $filepath -> $output"
}

# Process all files passed as arguments
for file in "$@"; do
    process_file "$file"
done

Quick Reference

# Default values
${var:-default}    # Use default if unset
${var:=default}    # Assign default if unset
${var:?error}      # Error if unset
${var:+alt}        # Use alt if set

# Removal
${var#pattern}     # Remove shortest from start
${var##pattern}    # Remove longest from start
${var%pattern}     # Remove shortest from end
${var%%pattern}    # Remove longest from end

# Replacement
${var/old/new}     # Replace first
${var//old/new}    # Replace all
${var/#old/new}    # Replace at start
${var/%old/new}    # Replace at end

# Case
${var^}            # Uppercase first
${var^^}           # Uppercase all
${var,}            # Lowercase first
${var,,}           # Lowercase all

# Substring
${var:offset}      # From offset to end
${var:offset:len}  # From offset, length len
${var: -n}         # Last n characters

# Length
${#var}            # Length of string

Pro Tips

• Use quotes: Always quote expansions: "${var}"
• Prefer built-ins: Much faster than external commands
• Test patterns: Use echo to verify before using in scripts
• Remember the symbols: # = beginning, % = end
• Double for greedy: ## and %% for longest match

Common Mistakes

# Wrong - no space before minus
${var:-5}  # Last 5 chars, needs space: ${var: -5}

# Wrong - unquoted expansion
basename=${filename%.*}  # Should be: basename="${filename%.*}"

# Wrong - wrong symbol order
${var*pattern}  # Should be: ${var#pattern} or ${var%pattern}

Conclusion

Parameter expansion is one of Bash's most powerful yet underutilized features. It's fast, elegant, and eliminates the need for external commands in many situations. Master these patterns and your scripts will be faster, cleaner, and more maintainable.

Stop reaching for sed, awk, or cut for simple string operations. Parameter expansion can handle it all!

References

• GNU Bash Manual - Parameter Expansion
• Advanced Bash-Scripting Guide - Parameter Substitution
• Bash Hackers Wiki - Parameter Expansion