Redis: Caching and Queuing in Web Applications

Redis is a powerful in-memory database that revolutionizes how we manage temporary data and asynchronous tasks in web applications. This article demonstrates how to effectively use Redis for caching and implementing queuing systems.

What is Redis?

Redis (Remote Dictionary Server) is an open-source key-value database operating in RAM. It features:

Lightning-fast performance - RAM operations
Versatility - strings, hashes, lists, sets, sorted sets
Persistence - optional disk write
Pub/Sub - built-in messaging system
Atomic operations - thread-safe operations

Redis as Cache - Boost Performance by 100x

Basic Caching Strategies

1. Cache-Aside (Lazy Loading)

Most popular pattern - application manages cache:

async function getUser(userId) {
  // Check cache
  const cached = await redis.get(`user:${userId}`);
  if (cached) {
    return JSON.parse(cached);
  }
  
  // Fetch from database
  const user = await db.users.findById(userId);
  
  // Save to cache (TTL 1h)
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
  
  return user;
}

2. Write-Through Cache

Write simultaneously to cache and database:

async function updateUser(userId, data) {
  // Update database
  const user = await db.users.update(userId, data);
  
  // Update cache
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
  
  return user;
}

Advanced Caching Techniques

Cache Invalidation

async function invalidateUserCache(userId) {
  await redis.del(`user:${userId}`);
  await redis.del(`user:${userId}:posts`);
  await redis.del(`user:${userId}:comments`);
}

Cache Warming

async function warmCache() {
  const popularUsers = await db.users.findPopular(100);
  
  const pipeline = redis.pipeline();
  for (const user of popularUsers) {
    pipeline.setex(`user:${user.id}`, 3600, JSON.stringify(user));
  }
  await pipeline.exec();
}

Redis as Queuing System

Task Queue Implementation

Basic FIFO Queue

// Producer - add task
async function enqueueJob(jobData) {
  await redis.lpush('jobs:queue', JSON.stringify({
    id: Date.now(),
    data: jobData,
    timestamp: new Date().toISOString()
  }));
}

// Consumer - process tasks
async function processJobs() {
  while (true) {
    const job = await redis.brpop('jobs:queue', 0);
    if (job) {
      const jobData = JSON.parse(job[1]);
      await handleJob(jobData);
    }
  }
}

Priority Queue

async function enqueueWithPriority(jobData, priority = 'normal') {
  const queue = `jobs:${priority}:queue`;
  await redis.lpush(queue, JSON.stringify(jobData));
}

async function processWithPriority() {
  const queues = [
    'jobs:high:queue',
    'jobs:normal:queue',
    'jobs:low:queue'
  ];
  
  while (true) {
    for (const queue of queues) {
      const job = await redis.rpop(queue);
      if (job) {
        await handleJob(JSON.parse(job));
        break;
      }
    }
    await sleep(100);
  }
}

Delayed Jobs

async function scheduleJob(jobData, delaySeconds) {
  const executeAt = Date.now() + (delaySeconds * 1000);
  await redis.zadd('jobs:delayed', executeAt, JSON.stringify(jobData));
}

async function processDelayedJobs() {
  while (true) {
    const now = Date.now();
    const jobs = await redis.zrangebyscore(
      'jobs:delayed', 0, now, 'LIMIT', 0, 10
    );
    
    for (const job of jobs) {
      await handleJob(JSON.parse(job));
      await redis.zrem('jobs:delayed', job);
    }
    await sleep(1000);
  }
}

Bull Queue - Professional Implementation

For production systems, we recommend Bull library:

const Queue = require('bull');

const emailQueue = new Queue('email', {
  redis: { host: 'localhost', port: 6379 }
});

await emailQueue.add('sendWelcome', {
  email: 'user@example.com',
  name: 'John Doe'
}, {
  attempts: 3,
  backoff: { type: 'exponential', delay: 2000 }
});

emailQueue.process('sendWelcome', async (job) => {
  const { email, name } = job.data;
  await sendEmail(email, `Welcome ${name}!`);
});

Redis in .NET

StackExchange.Redis Configuration

using StackExchange.Redis;

public class RedisService
{
    private readonly IDatabase _db;
    
    public RedisService(IConfiguration config)
    {
        var redis = ConnectionMultiplexer.Connect(
            config.GetConnectionString("Redis")
        );
        _db = redis.GetDatabase();
    }
    
    public async Task<T?> GetAsync<T>(string key)
    {
        var value = await _db.StringGetAsync(key);
        return value.HasValue 
            ? JsonSerializer.Deserialize<T>(value!) 
            : default;
    }
    
    public async Task SetAsync<T>(string key, T value, TimeSpan? expiry = null)
    {
        var json = JsonSerializer.Serialize(value);
        await _db.StringSetAsync(key, json, expiry);
    }
}

Best Practices

1. Use Appropriate Data Structures

// ❌ Bad - string for list
await redis.set('user:follows', JSON.stringify(follows));

// ✅ Good - Set
await redis.sadd('user:follows', ...follows);

2. Set TTL for All Keys

// ❌ Bad - no TTL
await redis.set('temp:data', value);

// ✅ Good - with TTL
await redis.setex('temp:data', 3600, value);

3. Use Pipeline for Multiple Operations

// ❌ Slow - individual operations
for (const item of items) {
  await redis.set(item.key, item.value);
}

// ✅ Fast - pipeline
const pipeline = redis.pipeline();
for (const item of items) {
  pipeline.set(item.key, item.value);
}
await pipeline.exec();

Implementation Checklist

Before deploying Redis, ensure:

[ ] Appropriate caching strategy selected
[ ] TTL set for all temporary keys
[ ] Cache invalidation implemented
[ ] Connection error handling
[ ] Monitoring and alerting configured
[ ] Backup and persistence enabled (if needed)
[ ] Connection pooling used
[ ] Memory limits set
[ ] Eviction policy configured
[ ] Cluster or replication (for HA)

Use Cases

E-commerce

Product and category caching
Shopping cart in session
Order processing queue
API rate limiting

Social Media

User posts and profiles caching
Real-time counters (likes, views)
Pub/Sub for notifications
User feed

SaaS

Application settings cache
Session storage
Feature flags
Analytics events queue

Summary

Redis is a versatile tool that:

Speeds up applications - cache reduces database load
Scales performance - handles millions of ops/s
Simplifies architecture - one tool for many tasks
Increases reliability - asynchronous processing

Proper Redis usage can boost application performance up to 100x and significantly improve user experience.

Need Support?

At MDS Software Solutions Group, we help with:

Redis caching implementation
Migration from Memcached to Redis
Application performance optimization
Queue systems implementation
Architecture audit