Automating Kafka and Redis Startup in WSL: Shell Scripts for Seamless Development

As developers working with distributed systems, we often need multiple services running simultaneously. Manually starting Kafka and Redis every time we fire up our WSL environment becomes tedious and time-consuming. In this comprehensive guide, I’ll show you how to create shell scripts that automatically start Kafka (in KRaft mode) and Redis clusters when WSL starts, streamlining your development workflow.

Why Automate Service Startup?

Before diving into the implementation, let’s understand the compelling reasons for automating service startup:

• Time-saving: Eliminates repetitive manual startup commands on every WSL restart
• Consistency: Services start with identical configurations every time
• Productivity: Jump straight into development without setup overhead
• Error reduction: Removes human error from startup sequences
• Team alignment: Ensures consistent environments across development teams

Prerequisites

Before implementing these automation scripts, ensure you have:

• Apache Kafka installed and configured in KRaft mode
• Redis server installed on your system
• Basic understanding of shell scripting and Linux commands
• WSL2 running on Windows with appropriate permissions
• Sufficient system resources for running both services

Understanding KRaft Mode

KRaft (Kafka Raft) represents Apache Kafka’s evolution beyond Apache Zookeeper dependency. This new consensus protocol simplifies deployment and reduces operational complexity, making it perfect for development environments.

Kafka KRaft Mode Startup Script

Let’s create our automated Kafka startup script that handles KRaft mode configuration:

#!/bin/bash

# Start Kafka in KRaft mode
KAFKA_HOME="/home/chandan/kafka"
LOG_DIR="/home/chandan/kafka-logs"

# Create log directory if it doesn't exist
mkdir -p "$LOG_DIR"

# Start Kafka server in KRaft mode
echo "Starting Kafka server (KRaft mode)..."
nohup $KAFKA_HOME/bin/kafka-server-start.sh $KAFKA_HOME/config/kraft/server.properties > "$LOG_DIR/kafka.log" 2>&1 &
KAFKA_PID=$!

# Save PID for later reference
echo $KAFKA_PID > "$LOG_DIR/kafka.pid"

echo "Kafka started successfully in KRaft mode"
echo "Kafka PID: $KAFKA_PID"
echo "Logs available at: $LOG_DIR"

Key Features of the Kafka Script:

1. KRaft Configuration: Uses KRaft-specific server properties, eliminating Zookeeper dependency
2. Background Execution: Runs Kafka as a daemon using nohup for persistent operation
3. Comprehensive Logging: Redirects both stdout and stderr to organized log files
4. PID Management: Saves process ID for easy service management and monitoring
5. Directory Management: Automatically creates necessary directories with proper permissions

Redis Cluster Startup Script

Redis clusters provide high availability, automatic sharding, and fault tolerance. Our comprehensive script sets up both a standalone cache server and a complete cluster topology:

#!/bin/bash

# Start Redis cache and cluster nodes
REDIS_HOME="/home/chandan/redis"
LOG_DIR="/home/chandan/redis-logs"

# Create directories if they don't exist
mkdir -p "$LOG_DIR"
mkdir -p "$REDIS_HOME/config"

# Create basic Redis cache configuration
cat > "$REDIS_HOME/config/redis-cache.conf" << EOF
port 6379
bind 127.0.0.1
daemonize no
logfile "$LOG_DIR/redis-cache.log"
dir "$REDIS_HOME/data"
save 900 1
save 300 10
save 60 10000
EOF

# Create Redis cluster node configurations (3 master + 3 replica nodes)
for port in 7001 7002 7003 7004 7005 7006; do
    cat > "$REDIS_HOME/config/redis-$port.conf" << EOF
port $port
bind 127.0.0.1
daemonize no
cluster-enabled yes
cluster-config-file nodes-$port.conf
cluster-node-timeout 5000
appendonly yes
logfile "$LOG_DIR/redis-cluster-$port.log"
dir "$REDIS_HOME/cluster-$port"
EOF
    mkdir -p "$REDIS_HOME/cluster-$port"
done

# Create data directory
mkdir -p "$REDIS_HOME/data"

echo "Starting Redis cache server..."
nohup redis-server "$REDIS_HOME/config/redis-cache.conf" > "$LOG_DIR/redis-cache.log" 2>&1 &
CACHE_PID=$!
echo $CACHE_PID > "$LOG_DIR/redis-cache.pid"

echo "Starting Redis cluster nodes..."
for port in 7001 7002 7003 7004 7005 7006; do
    echo "Starting Redis cluster node on port $port..."
    nohup redis-server "$REDIS_HOME/config/redis-$port.conf" > "$LOG_DIR/redis-cluster-$port.log" 2>&1 &
    CLUSTER_PID=$!
    echo $CLUSTER_PID > "$LOG_DIR/redis-cluster-$port.pid"
    sleep 1
done

echo "Redis services started successfully"
echo "Cache server PID: $CACHE_PID (port 6379)"
echo "Cluster nodes running on ports: 7001-7006"
echo "Logs available at: $LOG_DIR"
echo ""
echo "To create the cluster, run:"
echo "redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1"