Linux Programming

A Comprehensive Guide to System Development

Getting Started

Linux programming begins with understanding the fundamental concepts of the operating system. Linux is built on Unix principles, emphasizing modularity, simplicity, and the philosophy that "everything is a file."

            Essential Requirements
            A Linux distribution (Ubuntu, CentOS, Fedora, etc.)
Essential development tools (gcc, make, gdb)
A text editor or IDE
Basic command-line familiarity

        

Installation Commands

bash

            # Ubuntu/Debian
            sudo apt-get install build-essential

            # CentOS/RHEL
            sudo yum groupinstall "Development Tools"
        

Shell Programming

Shell programming is fundamental to Linux development. The shell acts as both a command interpreter and a programming language, allowing you to automate tasks and create powerful scripts.

Basic Shell Script Structure

Every shell script follows a standard structure that makes it readable, maintainable, and executable across different systems.

bash

#!/bin/bash
# =============================================================================
# Script Name: example_script.sh
# Description: Demonstrates basic shell script structure
# Author: Your Name
# Date: $(date +%Y-%m-%d)
# Version: 1.0
# =============================================================================

# Exit on any error
set -e

# Exit on undefined variables
set -u

# Enable debug mode (optional)
# set -x

# Global variables
readonly SCRIPT_NAME=$(basename "$0")
readonly SCRIPT_DIR=$(dirname "$0")
readonly LOG_FILE="/var/log/${SCRIPT_NAME%.sh}.log"

# Function definitions
show_usage() {
    cat << EOF
Usage: $SCRIPT_NAME [OPTIONS] [ARGUMENTS]

Description:
    This script demonstrates proper shell script structure

Options:
    -h, --help      Show this help message
    -v, --verbose   Enable verbose output
    -d, --debug     Enable debug mode

Examples:
    $SCRIPT_NAME --help
    $SCRIPT_NAME --verbose /path/to/file
EOF
}

# Main script logic
main() {
    echo "Starting $SCRIPT_NAME..."

    # Your script logic here
    echo "Script completed successfully!"
}

# Parse command line arguments
while [[ $# -gt 0 ]]; do
    case $1 in
        -h|--help)
            show_usage
            exit 0
            ;;
        -v|--verbose)
            VERBOSE=true
            shift
            ;;
        -d|--debug)
            set -x
            shift
            ;;
        *)
            echo "Unknown option: $1" >&2
            show_usage
            exit 1
            ;;
    esac
done

# Execute main function
main "$@"

Key Features

Variables

Shell variables are dynamically typed and don't require explicit declaration. Understanding variable types, scoping, and best practices is crucial.

bash

# ========================================
# Variable Declaration and Assignment
# ========================================

# Simple variable assignment (no spaces around =)
name="John Doe"
age=25
is_active=true

# Read-only variables (constants)
readonly PI=3.14159
readonly CONFIG_FILE="/etc/myapp.conf"

# Environment variables (available to child processes)
export PATH="$PATH:/usr/local/bin"
export DATABASE_URL="postgresql://localhost/mydb"

# Array variables
fruits=("apple" "banana" "orange" "grape")
declare -a numbers=(1 2 3 4 5)

# Associative arrays (key-value pairs)
declare -A colors
colors["red"]="#FF0000"
colors["green"]="#00FF00"
colors["blue"]="#0000FF"

# ========================================
# Variable Usage and Expansion
# ========================================

# Basic variable expansion
echo "Hello, $name!"
echo "Hello, ${name}!"  # Preferred for clarity

# Default values
echo "Database: ${DATABASE_NAME:-myapp_db}"  # Use default if unset
echo "Port: ${PORT:=8080}"                   # Set and use default

# String length
echo "Name length: ${#name}"

# Substring extraction
filename="document.pdf"
echo "Extension: ${filename##*.}"     # pdf
echo "Basename: ${filename%.*}"       # document

# Case conversion (Bash 4+)
text="Hello World"
echo "Upper: ${text^^}"               # HELLO WORLD
echo "Lower: ${text,,}"               # hello world

# ========================================
# Variable Validation and Testing
# ========================================

# Check if variable is set
if [[ -n "${DATABASE_URL:-}" ]]; then
    echo "Database URL is configured"
fi

# Check if variable is empty
if [[ -z "${DEBUG:-}" ]]; then
    DEBUG=false
fi

# Numeric variables
declare -i count=0
count+=1
echo "Count: $count"

# ========================================
# Command Substitution
# ========================================

# Modern syntax (preferred)
current_date=$(date +%Y-%m-%d)
file_count=$(ls -1 | wc -l)
current_user=$(whoami)

# Legacy syntax (avoid in new scripts)
old_date=`date`

echo "Today is $current_date"
echo "Current user: $current_user"
echo "Files in directory: $file_count"

Control Structures

Shell scripts support various control structures for conditional execution, loops, and flow control.

bash

# ========================================
# Conditional Statements (if/else)
# ========================================

# Basic if statement
if [[ $age -ge 18 ]]; then
    echo "You are an adult"
elif [[ $age -ge 13 ]]; then
    echo "You are a teenager"
else
    echo "You are a child"
fi

# File and directory tests
if [[ -f "$CONFIG_FILE" ]]; then
    echo "Config file exists"
elif [[ -d "/etc/config" ]]; then
    echo "Config directory exists"
else
    echo "No configuration found"
fi

# String comparisons
if [[ "$environment" == "production" ]]; then
    echo "Running in production mode"
elif [[ "$environment" =~ ^(dev|development)$ ]]; then
    echo "Running in development mode"
else
    echo "Unknown environment: $environment"
fi

# Multiple conditions
if [[ -r "$file" && -w "$file" ]]; then
    echo "File is readable and writable"
fi

# ========================================
# Case Statements
# ========================================

action="$1"
case $action in
    start|run|execute)
        echo "Starting the application..."
        ;;
    stop|halt|terminate)
        echo "Stopping the application..."
        ;;
    restart|reload)
        echo "Restarting the application..."
        ;;
    status|info)
        echo "Checking application status..."
        ;;
    *)
        echo "Unknown action: $action"
        echo "Valid actions: start, stop, restart, status"
        exit 1
        ;;
esac

# ========================================
# For Loops
# ========================================

# Iterate over a list
for fruit in apple banana orange; do
    echo "Processing: $fruit"
done

# Iterate over array
for item in "${fruits[@]}"; do
    echo "Fruit: $item"
done

# C-style for loop
for ((i=1; i<=10; i++)); do
    echo "Number: $i"
done

# Iterate over files
for file in /etc/*.conf; do
    if [[ -f "$file" ]]; then
        echo "Processing config: $file"
    fi
done

# Iterate with index
for i in "${!fruits[@]}"; do
    echo "Index $i: ${fruits[$i]}"
done

# ========================================
# While and Until Loops
# ========================================

# While loop
counter=1
while [[ $counter -le 5 ]]; do
    echo "Counter: $counter"
    ((counter++))
done

# Read file line by line
while IFS= read -r line; do
    echo "Line: $line"
done < "$CONFIG_FILE"

# Until loop (opposite of while)
attempts=0
until [[ $attempts -eq 3 ]]; do
    echo "Attempt $((attempts + 1))"
    ((attempts++))
done

# ========================================
# Loop Control
# ========================================

for i in {1..10}; do
    if [[ $i -eq 5 ]]; then
        continue  # Skip iteration
    fi
    if [[ $i -eq 8 ]]; then
        break     # Exit loop
    fi
    echo "Processing: $i"
done
            

Functions

Functions in shell scripts allow code reuse, better organization, and modular programming practices.

bash

# ========================================
# Function Definition and Basic Usage
# ========================================

# Simple function
greet() {
    echo "Hello, World!"
}

# Function with parameters
greet_user() {
    local name="$1"
    local greeting="${2:-Hello}"
    echo "$greeting, $name!"
}

# Function with return value
add_numbers() {
    local num1="$1"
    local num2="$2"
    local result=$((num1 + num2))
    echo "$result"  # Return via stdout
}

# Function with exit status
file_exists() {
    local filename="$1"
    [[ -f "$filename" ]]  # Returns 0 if true, 1 if false
}

# ========================================
# Advanced Function Features
# ========================================

# Function with local variables
process_data() {
    local input_file="$1"
    local output_file="$2"
    local temp_file="/tmp/processing_$$"

    # Process data
    sort "$input_file" > "$temp_file"
    uniq "$temp_file" > "$output_file"

    # Cleanup
    rm -f "$temp_file"

    echo "Processed $input_file -> $output_file"
}

# Function with variable arguments
log_message() {
    local level="$1"
    shift  # Remove first argument
    local message="$*"  # All remaining arguments
    local timestamp=$(date '+%Y-%m-%d %H:%M:%S')

    echo "[$timestamp] [$level] $message" | tee -a "$LOG_FILE"
}

# Function with error handling
safe_copy() {
    local source="$1"
    local destination="$2"

    # Validate arguments
    if [[ $# -ne 2 ]]; then
        echo "Error: safe_copy requires exactly 2 arguments" >&2
        return 1
    fi

    # Check source exists
    if [[ ! -f "$source" ]]; then
        echo "Error: Source file '$source' not found" >&2
        return 1
    fi

    # Check destination directory exists
    local dest_dir=$(dirname "$destination")
    if [[ ! -d "$dest_dir" ]]; then
        echo "Error: Destination directory '$dest_dir' not found" >&2
        return 1
    fi

    # Perform copy
    if cp "$source" "$destination"; then
        echo "Successfully copied '$source' to '$destination'"
        return 0
    else
        echo "Error: Failed to copy '$source' to '$destination'" >&2
        return 1
    fi
}

# ========================================
# Function Usage Examples
# ========================================

# Call simple function
greet

# Call function with arguments
greet_user "Alice" "Welcome"
greet_user "Bob"  # Uses default greeting

# Capture function output
result=$(add_numbers 15 25)
echo "Sum: $result"

# Use function return status
if file_exists "/etc/passwd"; then
    echo "System password file exists"
else
    echo "System password file not found"
fi

# Use logging function
log_message "INFO" "Application started successfully"
log_message "ERROR" "Database connection failed"

# Use function with error handling
if safe_copy "/etc/hosts" "/tmp/hosts.backup"; then
    echo "Backup completed successfully"
else
    echo "Backup failed" >&2
    exit 1
fi

# ========================================
# Recursive Functions
# ========================================

# Calculate factorial
factorial() {
    local n="$1"
    if [[ $n -le 1 ]]; then
        echo 1
    else
        local prev=$(factorial $((n - 1)))
        echo $((n * prev))
    fi
}

echo "Factorial of 5: $(factorial 5)"

# ========================================
# Function Libraries
# ========================================

# Source external function library
# source "/usr/local/lib/my_functions.sh"

# Check if function exists before calling
if declare -f "my_custom_function" >/dev/null; then
    my_custom_function
else
    echo "Function my_custom_function not available"
fi

File Operations

File operations are central to shell scripting. Master reading, writing, processing, and manipulating files efficiently.

bash

# ========================================
# File Testing and Information
# ========================================

check_file_properties() {
    local file="$1"

    echo "Checking properties of: $file"

    # File existence and types
    [[ -e "$file" ]] && echo "  ✓ File exists"
    [[ -f "$file" ]] && echo "  ✓ Regular file"
    [[ -d "$file" ]] && echo "  ✓ Directory"
    [[ -L "$file" ]] && echo "  ✓ Symbolic link"
    [[ -p "$file" ]] && echo "  ✓ Named pipe"
    [[ -S "$file" ]] && echo "  ✓ Socket"

    # Permissions
    [[ -r "$file" ]] && echo "  ✓ Readable"
    [[ -w "$file" ]] && echo "  ✓ Writable"
    [[ -x "$file" ]] && echo "  ✓ Executable"

    # Special permissions
    [[ -u "$file" ]] && echo "  ✓ SUID bit set"
    [[ -g "$file" ]] && echo "  ✓ SGID bit set"
    [[ -k "$file" ]] && echo "  ✓ Sticky bit set"

    # File size and age
    [[ -s "$file" ]] && echo "  ✓ File has content"
    [[ -z "$file" ]] && echo "  ✓ File is empty"
}

# ========================================
# Reading Files
# ========================================

# Read entire file into variable
read_file_content() {
    local file="$1"
    local content

    if [[ -r "$file" ]]; then
        content=$(cat "$file")
        echo "File content:"
        echo "$content"
    else
        echo "Cannot read file: $file" >&2
        return 1
    fi
}

# Read file line by line
process_file_lines() {
    local file="$1"
    local line_number=0

    while IFS= read -r line; do
        ((line_number++))
        echo "Line $line_number: $line"

        # Process each line here
        if [[ "$line" =~ ^# ]]; then
            echo "  -> This is a comment"
        fi
    done < "$file"
}

# Read file with different field separator
process_csv_file() {
    local csv_file="$1"
    local line_number=0

    while IFS=',' read -r name age city; do
        ((line_number++))
        if [[ $line_number -eq 1 ]]; then
            echo "Header: $name, $age, $city"
            continue
        fi
        echo "Record $((line_number-1)): Name=$name, Age=$age, City=$city"
    done < "$csv_file"
}

# ========================================
# Writing Files
# ========================================

# Create and write to file
create_config_file() {
    local config_file="$1"
    local app_name="$2"

    # Create directory if needed
    mkdir -p "$(dirname "$config_file")"

    # Write configuration
    cat > "$config_file" << EOF
# Configuration file for $app_name
# Generated on $(date)

[database]
host=localhost
port=5432
name=${app_name}_db

[logging]
level=INFO
file=/var/log/$app_name.log

[server]
host=0.0.0.0
port=8080
workers=4
EOF

    echo "Configuration written to: $config_file"
}

# Append to file
log_event() {
    local log_file="$1"
    local event="$2"
    local timestamp=$(date '+%Y-%m-%d %H:%M:%S')

    # Ensure log directory exists
    mkdir -p "$(dirname "$log_file")"

    # Append log entry
    echo "[$timestamp] $event" >> "$log_file"
}

# Write with proper permissions
create_secure_file() {
    local secure_file="$1"
    local content="$2"

    # Create file with restrictive permissions
    umask 077  # Remove all permissions for group and others
    echo "$content" > "$secure_file"

    # Set specific permissions
    chmod 600 "$secure_file"  # Owner read/write only

    echo "Secure file created: $secure_file"
}

# ========================================
# File Processing and Manipulation
# ========================================

# Search and filter files
find_and_process() {
    local search_dir="$1"
    local pattern="$2"

    # Find files matching pattern
    find "$search_dir" -name "$pattern" -type f | while read -r file; do
        echo "Processing: $file"

        # Get file info
        local size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file" 2>/dev/null)
        local modified=$(stat -f%Sm "$file" 2>/dev/null || stat -c%y "$file" 2>/dev/null)

        echo "  Size: $size bytes"
        echo "  Modified: $modified"

        # Process file content
        local line_count=$(wc -l < "$file")
        echo "  Lines: $line_count"
    done
}

# Backup files
backup_file() {
    local original_file="$1"
    local backup_dir="${2:-./backups}"
    local timestamp=$(date +%Y%m%d_%H%M%S)
    local filename=$(basename "$original_file")
    local backup_file="$backup_dir/${filename}.backup_$timestamp"

    # Create backup directory if it doesn't exist
    mkdir -p "$backup_dir"

    # Copy the file with error handling
    if cp "$original_file" "$backup_file"; then
        echo "Backup created: $backup_file"
        return 0
    else
        echo "Failed to create backup of $original_file" >&2
        return 1
    fi
}

# Archive multiple files
archive_files() {
    local archive_name="$1"
    shift  # Remove first argument
    local files_to_archive=("$@")

    # Create temporary directory
    local temp_dir="/tmp/archive_temp_$$"
    mkdir -p "$temp_dir"

    # Copy files to temp directory
    for file in "${files_to_archive[@]}"; do
        if [[ -f "$file" ]]; then
            cp "$file" "$temp_dir/"
        else
            echo "Warning: File not found - $file" >&2
        fi
    done

    # Create archive
    tar -czf "$archive_name" -C "$temp_dir" .
    local status=$?

    # Cleanup
    rm -rf "$temp_dir"

    if [[ $status -eq 0 ]]; then
        echo "Archive created: $archive_name"
    else
        echo "Failed to create archive" >&2
        return 1
    fi
}

Loops in Shell Programming

Loops are essential control structures that allow you to repeat commands or operations multiple times. Shell scripting supports several types of loops, each suited for different scenarios.

For Loops

For loops are the most common type of loop, perfect for iterating over lists, arrays, ranges, or command outputs.

bash
# Basic for loop with list
    for name in John Sara Michael Thomas; do
    echo "Hello, $name!"
    done

    # For loop with array
    users=("admin" "developer" "tester" "guest")
    for user in "${users[@]}"; do
    echo "Processing user: $user"
    done

    # For loop with range (Bash 4+)
    for i in {1..10}; do
    echo "Number: $i"
    done

    # For loop with range and step (Bash 4+)
    for i in {1..20..2}; do  # Odd numbers from 1 to 20
    echo "Odd number: $i"
    done

    # C-style for loop
    for ((i=0; i<5; i++)); do
    echo "Iteration $i"
    done

    # For loop with command output
    for file in $(ls *.txt); do
    echo "Processing file: $file"
    done

    # For loop to iterate through directories
    for dir in */; do
    echo "Found directory: $dir"
    done

For loops are versatile and can be used for many different iteration needs. They work well when you know exactly what you're iterating over.

While Loops

While loops continue to execute as long as a specified condition remains true. They're ideal for situations where you don't know in advance how many iterations are needed.

bash
# Basic while loop with counter
    counter=1
    while [ $counter -le 5 ]; do
    echo "Counter: $counter"
    ((counter++))
    done

    # While loop with user input
    while true; do
    read -p "Enter a command (or 'quit' to exit): " cmd

    if [ "$cmd" = "quit" ]; then
        echo "Exiting..."
        break
    fi

    echo "Executing: $cmd"
    eval "$cmd"
    done

    # While loop for reading files line by line
    while IFS= read -r line; do
    echo "Processing: $line"
    # Process the line here
    done < input.txt

    # Using while with command output
    cat /etc/passwd | while read -r line; do
    echo "User entry: $line"
    done

    # While loop with multiple conditions
    count=1
    max=10
    while [ $count -le $max ] && [ $count -ne 5 ]; do
    echo "Number: $count"
    ((count++))
    done

While loops are particularly useful for processing user input, reading files, or in any situation where the loop should continue until a specific condition is met.

Until Loops

Until loops are the opposite of while loops - they continue to execute as long as a specified condition remains false. They run until the condition becomes true.

bash
# Basic until loop with counter
    counter=1
    until [ $counter -gt 5 ]; do
    echo "Counter: $counter"
    ((counter++))
    done

    # Until loop for checking file existence
    until [ -f /tmp/signal_file ]; do
    echo "Waiting for signal file to appear..."
    sleep 5
    done
    echo "Signal file found! Continuing..."

    # Until loop with process checking
    pid=1234
    until ! ps -p $pid > /dev/null; do
    echo "Process $pid is still running, waiting..."
    sleep 10
    done
    echo "Process $pid has finished"

    # Until loop with return status
    until ping -c 1 -W 1 google.com > /dev/null; do
    echo "Network connection not available, retrying..."
    sleep 5
    done
    echo "Network connection established!"

Until loops are particularly useful for waiting scenarios, such as waiting for a file to be created, a process to finish, or a resource to become available.

Select Loops

The select loop is a unique construct that creates a numbered menu from which users can select options. It's primarily used for creating interactive shell scripts.

bash
# Basic select loop for menu options
    echo "Please select an operation:"
    select operation in "Add" "Subtract" "Multiply" "Divide" "Quit"; do
    case $operation in
        "Add")
            read -p "Enter first number: " num1
            read -p "Enter second number: " num2
            echo "Result: $(($num1 + $num2))"
            ;;
        "Subtract")
            read -p "Enter first number: " num1
            read -p "Enter second number: " num2
            echo "Result: $(($num1 - $num2))"
            ;;
        "Multiply")
            read -p "Enter first number: " num1
            read -p "Enter second number: " num2
            echo "Result: $(($num1 * $num2))"
            ;;
        "Divide")
            read -p "Enter first number: " num1
            read -p "Enter second number: " num2
            if [ $num2 -eq 0 ]; then
                echo "Error: Division by zero"
            else
                echo "Result: $(($num1 / $num2))"
            fi
            ;;
        "Quit")
            echo "Exiting..."
            break
            ;;
        *)
            echo "Invalid option"
            ;;
    esac
    done

    # Select loop with arrays
    options=("View Files" "Check Disk Space" "Check Memory" "Check Network" "Exit")
    echo "System Maintenance Menu"
    select opt in "${options[@]}"; do
    case $REPLY in
        1) ls -la ;;
        2) df -h ;;
        3) free -m ;;
        4) ifconfig ;;
        5) echo "Goodbye!" && break ;;
        *) echo "Invalid option. Try another one." ;;
    esac
    done

    # Custom PS3 prompt
    PS3="Choose your favorite color (1-4): "
    select color in "Red" "Green" "Blue" "Yellow"; do
    if [ -n "$color" ]; then
        echo "You selected $color!"
        break
    else
        echo "Invalid selection. Please try again."
    fi
    done

The select loop automatically displays a numbered menu and prompts users for input. It's especially useful for creating interactive command-line interfaces and user-friendly scripts.

Key Points About Select Loops

The PS3 variable controls the prompt displayed to the user (default is "#? ").
The selected option is stored in the variable specified after the "select" keyword.
The user's numeric choice is stored in the REPLY variable.
Select loops continue indefinitely until explicitly terminated with "break".
They work well with case statements for handling different options.