Redis 101

Let's learn about Redis

Reference: redis.io, https://brunch.co.kr/@jehovah/20

Intro to Redis

What is Redis?

• In-Memory Data Structure Store

  • A key-value based In-Memory data store

    • Since it is key-value based, results can be retrieved without needing separate queries

  • A memory-based data store, not disk-based

    • Since it processes data in memory rather than writing data to disk, the speed is remarkably fast

    • It does not rely solely on memory, so data is not lost even when the server shuts down

      • Uses a snapshot approach that saves to disk in between, and

      • Records all logs so data is recovered based on logs when the server restarts

• Open Source (BSD 3 License)

  • A proven open source solution used by many companies including Samsung Electronics, Naver, LINE, etc.

DataTypes

Redis supports various data types

• Strings

  • The most basic Redis value

  • Simple key-value mapping structure

• Lists

  • Array-style data structure

  • With Lists, inserting and removing data at the beginning and end is fast, but inserting data in the middle is difficult

• Sets

  • An unordered collection of String data

  • Does not allow duplicates

    • Duplicate data is treated as one

• Sorted sets

  • Same structure as Sets, but order can be determined through scores

  • Using Sorted sets makes it easy to implement features like Leaderboards

• Hashes

  • A structure well-suited for storing object types with multiple key-value pairs

Redis Use Cases

1. Caching

• As an In-Memory data structure store, it can serve as a cache

• Acts as a shield to prevent DB overload

2. Session Management

• Redis can be used as a session for user information management, etc.

3. Real-time Leaderboards

• Using Redis's sorted set data type, you can build leaderboards that are automatically sorted by desired values

4. Queues

• Queues can be easily implemented using Redis's list type

5. Chat

• Redis is widely used in actual chat services

• Since it natively supports the PUB/SUB standard, high-performance chat rooms, inter-server communication, etc. can be implemented

2. Managing Redis

• A key characteristic of Redis is that it is Single Threaded

  • That is, Redis can only execute one command at a time

    • If a Packet containing a command is larger than the MTU (Maximum Transmission Unit), the packet may arrive split

      • Redis combines the split commands and executes the command the moment it becomes a complete command

  • You might think Redis is slow because it is Single Threaded, but on average, Get/Set commands can process approximately 100,000 per second

    • However, an important note is that if a long-processing command is inserted in the middle, all subsequent commands must wait

      • ex) The Keys command that retrieves all keys takes a very long time to process; if the Keys command is executed in the middle, subsequent Get/Set commands may timeout and fail

• When operating a service with Redis, the server's memory may reach its limit. The memory limit can be set with the maxmemory value

  • When memory fills up to the maxmemory limit, Redis secures additional memory according to the Max memory policy

maxmemory-policy Settings

1. noeviction

2. allkeys-lru

3. volatile-lru

4. allkeys-random

5. volatile-random

6. volatile-ttl

7. allkeys-lfu

8. volatile-lfu

eviction

• When data is deleted due to maxmemory being exceeded, it is called eviction

• You can check whether eviction has occurred by connecting to Redis, running the INFO command, and checking the evicted_keys metric

  • The evicted_keys value is the count of evicted keys, so the larger it is, the more data has been deleted from memory

  • When using Amazon ElastiCache, you can check through the eviction graph in the monitoring tab

    • You can also receive alerts about eviction occurrences

used_memory_rss

• The used_memory_rss value is the actual memory Redis is using, including data

  • This value can be larger than the actual used_memory value

    • This occurs because the OS allocates memory in multiples of page size

      • ex) If the page size is 4096 and the requested memory size is 10, the OS allocates 4096 of memory

      • This is called the Fragmentation phenomenon, which is the cause of the difference between actual used memory and allocated memory

Redis Replication

• Among the ways to configure Redis, there is a Master/Slave structure for read distribution and data redundancy

  • The Master node performs both reads and writes,

  • The Slave node can only perform reads

    • For this to work, the slave must have all the data from the master

      • This is when Replication occurs

• Replication refers to the process of copying data from the master and moving it to the slave

Master-Slave Replication Process

1. Configure replicaof <Master IP> <Master Port> on the slave configuration side, or use the REPLICAOF command to request data sync from the master

2. The master proceeds with the process of creating an RDB file containing the current memory state in the background

   • At this point, the master copies memory through fork

     • Then, the forked process dumps the current memory information to disk

3. Simultaneously with step 2, the master stores incoming WRITE commands in a buffer

4. When the dump is complete, the master sends the RDB file to the slave, and the slave saves it to disk and then loads it into memory

5. The WRITE commands accumulated in step 3 are sent to the slave

Key Facts

• Replication is asynchronous

• Replication is non-blocking on the Master

• Replication can block operations on Replica

• Master can have any number of Replicas

• Generally Replicas are read-only

• Replica can be Master for other Replicas

Need for replication

• High availability

• Spread the read load across replicas

• Replication in lieu of Persistence

Types of Replication

1. Full Sync

• Entire data is transferred to Replica

• Dump not saved on Master in diskless mode

• Used when

  • Bootstrapping a new replica

  • Existing replica too far behind to do partial sync

2. Partial Sync

More to be added