Linux Shell Basics: Pipes, Redirection, and Command Chaining

Table of Contents

linux-beginner - This article is part of a series.

Part 1: The Linux Filesystem: Structure and Navigation

Part 2: Linux Package Management: APT and DNF

Part 3: Linux Users, Groups, and Permissions

Part 5: This Article

The Shell
#

The shell is the program that reads your commands and executes them. When you open a terminal, you’re interacting with a shell — typically bash (Bourne Again Shell) on most Linux distributions.

The shell does more than just run commands. It connects programs together, redirects input and output, expands variables and patterns, and provides control structures. Understanding these features is what separates typing commands from building efficient workflows.

Standard Streams
#

Every process has three standard streams:

Stream	Number	Default	Purpose
stdin	0	Keyboard	Input to the program
stdout	1	Terminal	Normal output
stderr	2	Terminal	Error messages

This separation is fundamental — it lets you redirect output and errors independently.

Output Redirection
#

Redirect stdout to a file
#

# Write output to a file (overwrites)
ls /etc > file-list.txt

# Append output to a file
echo "new entry" >> log.txt

Redirect stderr to a file
#

# Errors only (file descriptor 2)
find / -name "*.conf" 2> errors.txt

# Errors to one file, output to another
find / -name "*.conf" > results.txt 2> errors.txt

Redirect both stdout and stderr
#

# Both to the same file
command > output.txt 2>&1

# Shorthand (bash 4+)
command &> output.txt

# Append both
command >> output.txt 2>&1

Discard output
#

# Discard stdout
command > /dev/null

# Discard stderr
command 2> /dev/null

# Discard everything
command &> /dev/null

/dev/null is a special file that discards anything written to it.

Input Redirection
#

Read stdin from a file
#

# Feed a file as input
sort < unsorted.txt

# Combine with output redirection
sort < unsorted.txt > sorted.txt

Here documents (heredoc)
#

Pass multi-line input inline:

cat << EOF
Hello, this is
a multi-line
message.
EOF

Useful for generating config files in scripts:

cat << EOF > /etc/myapp/config.toml
[server]
host = "0.0.0.0"
port = 8080
EOF

Here strings
#

Pass a single string as stdin:

grep "error" <<< "this is an error message"

Pipes
#

Pipes connect stdout of one command to stdin of the next. This is the core mechanism for composing commands:

command1 | command2 | command3

Practical pipe examples
#

# Find the 10 largest files
du -ah /home | sort -rh | head -10

# Count lines in a file
cat access.log | wc -l
# Simpler:
wc -l < access.log

# Search logs for errors, show unique messages
grep "ERROR" app.log | sort -u

# List processes using the most memory
ps aux | sort -k4 -rn | head -10

# Show only usernames from /etc/passwd
cut -d: -f1 /etc/passwd | sort

# Count files by extension in current directory
find . -type f | sed 's/.*\.//' | sort | uniq -c | sort -rn

Pipe to common utilities
#

Utility	Purpose	Example
`grep`	Filter lines by pattern	`
`sort`	Sort lines	`
`uniq`	Remove duplicates (requires sorted input)	`
`wc`	Count lines/words/chars	`
`head`	First N lines	`
`tail`	Last N lines	`
`cut`	Extract columns	`
`awk`	Column processing	`
`sed`	Text substitution	`
`tee`	Write to file AND pass through	`
`xargs`	Convert stdin to arguments	`

`tee` — Write and pass through
#

tee sends output to a file while still passing it through the pipe:

# Save intermediate results without breaking the pipeline
ps aux | tee all-processes.txt | grep nginx

# Append instead of overwrite
command | tee -a log.txt

`xargs` — Build commands from stdin
#

# Delete all .tmp files found by find
find /tmp -name "*.tmp" | xargs rm

# Safer with filenames containing spaces
find /tmp -name "*.tmp" -print0 | xargs -0 rm

# Run a command for each line
cat servers.txt | xargs -I{} ssh {} uptime

Command Chaining
#

Sequential execution (`;`)
#

Run commands one after another regardless of success:

mkdir project; cd project; git init

Conditional AND (`&&`)
#

Run the next command only if the previous one succeeded:

make && make install

apt update && apt upgrade -y

cd /opt/myapp && ./start.sh

Conditional OR (`||`)
#

Run the next command only if the previous one failed:

cd /opt/myapp || echo "Directory not found"

grep -q "error" log.txt || echo "No errors found"

Combining AND and OR
#

# Try to start the service, report success or failure
systemctl start nginx && echo "Started" || echo "Failed to start"

Command Substitution
#

Capture a command’s output and use it as part of another command:

# Using $()
echo "Today is $(date +%A)"
echo "You are on $(hostname)"

# Create a dated backup
cp file.txt "file-$(date +%F).txt"

# Use output as an argument
kill $(pgrep myapp)
du -sh $(which python3)

Globbing (Filename Patterns)
#

The shell expands patterns before the command sees them:

Pattern	Matches
`*`	Any number of characters
`?`	Exactly one character
`[abc]`	One of a, b, or c
`[0-9]`	One digit
`[!abc]`	Not a, b, or c
`{a,b,c}`	Brace expansion: a, b, or c

# All .log files
ls *.log

# Single character difference
ls file?.txt    # file1.txt, fileA.txt

# Range
ls report_[0-9][0-9].pdf

# Brace expansion
mkdir -p project/{src,tests,docs}
cp config.{toml,toml.bak}

Quoting
#

Quoting controls how the shell interprets special characters:

Quote type	Behavior
`"double"`	Expands variables and command substitution, preserves spaces
`'single'`	No expansion — everything is literal
`\`	Escape a single character

NAME="world"

echo "Hello $NAME"      # Hello world
echo 'Hello $NAME'      # Hello $NAME
echo "File: \"$NAME\""  # File: "world"
echo "Price: \$5"       # Price: $5

Rule of thumb: Use double quotes around variables to prevent word splitting and glob expansion:

# Bad — breaks if filename has spaces
cat $FILE

# Good
cat "$FILE"

Practical Workflows
#

Monitor a log file for errors
#

tail -f /var/log/syslog | grep --line-buffered "error"

Find and replace across multiple files
#

grep -rl "old_string" ./src/ | xargs sed -i 's/old_string/new_string/g'

Check disk usage and alert
#

USAGE=$(df -h / | awk 'NR==2 {print $5}' | tr -d '%')
[ "$USAGE" -gt 90 ] && echo "Disk usage is ${USAGE}%!"

Download a list of URLs
#

cat urls.txt | xargs -I{} -P4 wget -q {}

The -P4 runs 4 downloads in parallel.

Process a CSV file
#

# Skip header, extract second column, sort, count unique values
tail -n +2 data.csv | cut -d',' -f2 | sort | uniq -c | sort -rn

Best Practices
#

Always quote your variables — "$VAR" prevents word splitting surprises
Use && instead of ; when commands depend on the previous one succeeding
Use set -e in scripts to stop on first error
Redirect stderr separately when you need to log errors without cluttering output
Prefer $(command) over backticks — it’s more readable and nests properly
Use tee to save intermediate output while debugging pipelines
Build pipelines incrementally — run each step alone first, then chain them together
Use xargs -0 with find -print0 for filenames with spaces or special characters

linux-beginner - This article is part of a series.

Part 1: The Linux Filesystem: Structure and Navigation

Part 2: Linux Package Management: APT and DNF

Part 3: Linux Users, Groups, and Permissions

Part 4: Linux Processes and Services

Part 5: This Article

The Shell#

Standard Streams#

Output Redirection#

Redirect stdout to a file#

Redirect stderr to a file#

Redirect both stdout and stderr#

Discard output#

Input Redirection#

Read stdin from a file#

Here documents (heredoc)#

Here strings#

Pipes#

Practical pipe examples#

Pipe to common utilities#

tee — Write and pass through#

xargs — Build commands from stdin#

Command Chaining#

Sequential execution (;)#

Conditional AND (&&)#

Conditional OR (||)#

Combining AND and OR#

Command Substitution#

Globbing (Filename Patterns)#

Quoting#

Practical Workflows#

Monitor a log file for errors#

Find and replace across multiple files#

Check disk usage and alert#

Download a list of URLs#

Process a CSV file#

Best Practices#