Shell Script Hack #8: find and -exec for Powerful File Operations

You need to find all log files larger than 100MB and delete them, or locate every Python file modified in the last week and run a linter on them. The find command combined with -exec is the ultimate file search and action tool that most developers barely use to its full potential.

The Problem

Your server is running out of disk space. You need to find and delete large log files, but there are thousands of files spread across multiple directories. Doing this manually would take hours.

# Manual approach - tedious and error-prone
cd /var/log
ls -lh *.log
# Check each file size
# Delete manually
# Repeat for each directory...

The Hack: find with -exec

The find command can search for files based on any criteria and execute commands on each result:

# Find and delete log files larger than 100MB
find /var/log -name "*.log" -size +100M -exec rm -f {} \;

One command, instantly finds and processes all matching files across the entire directory tree!

Understanding find Syntax

The basic syntax:

find [path] [conditions] -exec [command] {} \;

# {} - placeholder for found file
# \; - marks end of -exec command
# + - pass multiple files to command (faster)

Essential find Patterns

Find by Name

# Exact name
find . -name "config.txt"

# Case-insensitive
find . -iname "README.md"

# Wildcard patterns
find . -name "*.log"
find . -name "test_*.py"

# Multiple patterns
find . -name "*.jpg" -o -name "*.png"

Find by Type

# Files only
find . -type f

# Directories only
find . -type d

# Symbolic links
find . -type l

# Empty files
find . -type f -empty

# Empty directories
find . -type d -empty

Find by Size

# Larger than 100MB
find . -size +100M

# Smaller than 10KB
find . -size -10k

# Exactly 1GB
find . -size 1G

# Between 10MB and 50MB
find . -size +10M -size -50M

# Size units: k (KB), M (MB), G (GB)

Find by Time

# Modified in last 7 days
find . -mtime -7

# Modified more than 30 days ago
find . -mtime +30

# Accessed in last 24 hours
find . -atime -1

# Changed in last hour
find . -cmin -60

# Modified between 7-14 days ago
find . -mtime +7 -mtime -14

Powerful -exec Examples

Delete Files

# Delete old log files
find /var/log -name "*.log" -mtime +30 -exec rm -f {} \;

# Delete empty files
find . -type f -empty -exec rm {} \;

# Delete with confirmation
find . -name "*.tmp" -exec rm -i {} \;

# Faster for many files - batch execution
find . -name "*.tmp" -exec rm {} +

Move or Copy Files

# Move old files to archive
find . -name "*.log" -mtime +90 -exec mv {} /archive/ \;

# Copy large files to backup
find . -size +1G -exec cp {} /backup/ \;

# Move with directory structure preserved
find /source -name "*.pdf" -exec sh -c 'mkdir -p /dest/$(dirname {}) && mv {} /dest/{}' \;

Change Permissions

# Make all scripts executable
find . -name "*.sh" -exec chmod +x {} \;

# Fix directory permissions
find . -type d -exec chmod 755 {} \;

# Fix file permissions
find . -type f -exec chmod 644 {} \;

# Change ownership
find /var/www -type f -exec chown www-data:www-data {} \;

Process Files

# Compress old files
find . -name "*.log" -mtime +7 -exec gzip {} \;

# Run linter on Python files
find . -name "*.py" -exec pylint {} \;

# Convert image format
find . -name "*.jpg" -exec convert {} {}.png \;

# Extract archive files
find . -name "*.tar.gz" -exec tar xzf {} \;

Advanced Techniques

Using sh -c for Complex Commands

# Rename files with pattern
find . -name "*.txt" -exec sh -c 'mv "$1" "${1%.txt}.md"' _ {} \;

# Create backup before modifying
find . -name "*.conf" -exec sh -c 'cp "$1" "$1.bak" && sed -i "s/old/new/g" "$1"' _ {} \;

# Process with multiple commands
find . -name "*.log" -exec sh -c '
    echo "Processing: $1"
    gzip "$1"
    mv "$1.gz" /archive/
' _ {} \;

Combining with Other Commands

# Find and grep
find . -name "*.log" -exec grep -l "ERROR" {} \;

# Find and show file info
find . -name "*.jpg" -exec file {} \;

# Find and calculate checksums
find . -name "*.iso" -exec md5sum {} \;

# Find and list with details
find . -name "*.pdf" -exec ls -lh {} \;

Using -ok for Interactive Confirmation

# Ask before deleting each file
find . -name "*.tmp" -ok rm {} \;

# Confirm each move
find . -size +100M -ok mv {} /archive/ \;

Real-World Use Cases

Disk Space Cleanup

#!/bin/bash

# cleanup-disk.sh
echo "Cleaning up disk space..."

# Delete logs older than 30 days
find /var/log -name "*.log" -mtime +30 -exec rm -f {} \;
echo "Old logs deleted"

# Compress logs older than 7 days
find /var/log -name "*.log" -mtime +7 -exec gzip {} \;
echo "Recent logs compressed"

# Delete empty files
find /tmp -type f -empty -delete
echo "Empty temp files deleted"

# Delete files larger than 1GB in /tmp
find /tmp -type f -size +1G -mtime +1 -delete
echo "Large temp files deleted"

echo "Cleanup complete!"
df -h

Code Quality Check

#!/bin/bash

# Run linters on modified files
echo "Checking modified Python files..."
find . -name "*.py" -mtime -1 -exec pylint {} \;

echo "Checking shell scripts..."
find . -name "*.sh" -mtime -1 -exec shellcheck {} \;

echo "Checking JavaScript..."
find . -name "*.js" -mtime -1 -exec eslint {} \;

Backup Recent Changes

#!/bin/bash

# Backup files modified in last 24 hours
BACKUP_DIR="/backup/$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

find /var/www -type f -mtime -1 -exec cp --parents {} "$BACKUP_DIR" \;

echo "Backup completed to $BACKUP_DIR"

Security Audit

#!/bin/bash

echo "=== Security Audit ==="

# Find world-writable files
echo "World-writable files:"
find / -type f -perm -002 2>/dev/null

# Find SUID files
echo "SUID files:"
find / -type f -perm -4000 2>/dev/null

# Find files without owner
echo "Files without owner:"
find / -nouser -o -nogroup 2>/dev/null

# Find files modified in last hour
echo "Recently modified system files:"
find /etc /bin /sbin -type f -mmin -60 2>/dev/null

Performance Optimization

Use + Instead of \;

# Slow - runs command for each file
find . -name "*.txt" -exec wc -l {} \;

# Fast - batches files together
find . -name "*.txt" -exec wc -l {} +

# Benchmark difference:
# \; - 45 seconds for 10,000 files
# + - 3 seconds for 10,000 files

Limit Search Depth

# Search only current directory
find . -maxdepth 1 -name "*.txt"

# Search up to 3 levels deep
find . -maxdepth 3 -name "*.log"

# Skip certain directories
find . -path ./node_modules -prune -o -name "*.js" -print

Use -delete Instead of -exec rm

# Slower
find . -name "*.tmp" -exec rm {} \;

# Faster - built-in delete
find . -name "*.tmp" -delete

Complex Conditions

# AND conditions (default)
find . -name "*.log" -size +10M

# OR conditions
find . -name "*.log" -o -name "*.txt"

# NOT condition
find . ! -name "*.log"

# Complex logic with parentheses
find . \( -name "*.jpg" -o -name "*.png" \) -size +1M

# Multiple conditions
find . -type f -name "*.sh" ! -perm -111 -exec chmod +x {} \;

Finding by Content

# Find files containing text
find . -type f -exec grep -l "password" {} \;

# Better performance - use grep first
grep -r "password" . --include="*.conf" -l

# Find and replace in files
find . -name "*.txt" -exec sed -i 's/old/new/g' {} \;

Common Patterns

# Find broken symlinks
find . -type l ! -exec test -e {} \; -print

# Find duplicate files by size
find . -type f -exec du -h {} + | sort -h | uniq -w 15 -D

# Find recently modified files, excluding .git
find . -path ./.git -prune -o -type f -mtime -7 -print

# Find files by extension and run command
find . -name "*.md" -exec pandoc {} -o {}.pdf \;

# Count files by extension
find . -type f | sed 's/.*\.//' | sort | uniq -c | sort -rn

Debugging find Commands

# Print what find would do without executing
find . -name "*.tmp" -exec echo "Would delete: {}" \;

# Show command being executed
find . -name "*.log" -exec sh -c 'set -x; gzip "$1"' _ {} \;

# Test with -print first
find . -name "*.tmp" -print
# Then add -delete when sure
find . -name "*.tmp" -delete

Pro Tips

• Test with -print first: Always verify before using -delete or -exec
• Use + for performance: Much faster than \; for multiple files
• Quote your patterns: Use quotes around wildcards
• Redirect errors: Add 2>/dev/null to hide permission errors
• Combine with xargs: For even more complex operations

Safety Checklist

• Always test find commands with -print before -delete
• Use -maxdepth to limit scope when testing
• Consider using -ok instead of -exec for destructive operations
• Make backups before bulk modifications
• Test on sample data first

Common Mistakes to Avoid

# Wrong - missing \;
find . -name "*.txt" -exec rm {}

# Wrong - not quoting pattern
find . -name *.txt

# Wrong - wrong order
find . -delete -name "*.tmp"  # Deletes EVERYTHING!

# Correct
find . -name "*.tmp" -delete

Conclusion

The find command with -exec is one of the most powerful file management tools available. It can locate files based on any criteria imaginable and perform any operation on them. From cleaning up disk space to processing thousands of files, find and -exec can handle it all in a single command.

Master these patterns, and you’ll handle bulk file operations that would take hours manually in just seconds!

References

• GNU findutils Manual
• InfoWorld – Shell Tips for Developers
• LinuxConfig – Complex Bash Examples