Question:

I'm performing some analysis on a data stream and publishing the results on a Redis channel. Consumers subscribe to these channels and get real-time data feeds. All historical data analysis results are lost.

Now I want to add the ability to store historical data in Redis so that consumers can query this historical data (mainly by time). Since the analysis results are partitioned by time what would be the a good design to store the results in Redis?

Answer:

Use redis .

Sorted sets store data based on "scores", so in your case, just use a time stamp in millis; the data will be sorted automatically, allowing you to retrieve historical items using start/end date ranges, here's an example...

Add items to a sorted set...

zadd historical 
     
     
    

..add some sample data..

zadd historical 1 data1 zadd historical 2 data2 zadd historical 3 data3 zadd historical 4 data4 zadd historical 5 data5 zadd historical 6 data6 zadd historical 7 data7

..retrieve a subset of items using start/end range...

zrangebyscore historical 2 5

..returns...

1) "data2"2) "data3"3) "data4"4) "data5"

So, in your case, if you want to retrieve all historical items for the last day, just do this...

zrangebyscore historical 
     
     
    

Additional

(1) What if two data inserts in the same milisecond?

While members are unique in a sorted set, scores (time stamps in this case) may be repeated.

(2) What if each data value is not unique?

you can add some random string before or after your data to keep it not repeated.

I think in your last code example you'd want to reverse the range, i.e. zrangebyscore historical <currentTimeInMillis - 86400000> <currentTimeInMillis> since it looks like it expects the lower value first.

Redis command returns all the elements in the sorted set at key with a score between min and max (including elements with score equal to min or max). The elements are considered to be ordered from low to high scores. The elements having the same score are returned in lexicographical order (this follows from a property of the sorted set implementation in Redis and does not involve further computation).

Reference：