python使用pipeline批量读写redis的方法

1.插入数据

1. >>> import redis
2.  
3. >>> conn = redis.Redis(host='192.168.8.176',port=6379)
4.  
5. >>> pipe = conn.pipeline()
6.  
7. >>> pipe.hset("hash_key","leizhu900516",8)
8. Pipeline<ConnectionPool<Connection<host=192.168.8.176,port=6379,db=0>>>
9.  
10. >>> pipe.hset("hash_key","chenhuachao",9)
11. Pipeline<ConnectionPool<Connection<host=192.168.8.176,port=6379,db=0>>>
12.  
13. >>> pipe.hset("hash_key","wanger",10)
14. Pipeline<ConnectionPool<Connection<host=192.168.8.176,port=6379,db=0>>>
15.  
16. >>> pipe.execute()
17. [1L, 1L, 1L]
>>>

2.批量读取数据

1. >>> pipe.hget("hash_key","leizhu900516")
2. Pipeline<ConnectionPool<Connection<host=192.168.8.176,port=6379,db=0>>>
3.  
4. >>> pipe.hget("hash_key","chenhuachao")
5. Pipeline<ConnectionPool<Connection<host=192.168.8.176,port=6379,db=0>>>
6.  
7. >>> pipe.hget("hash_key","wanger")
8. Pipeline<ConnectionPool<Connection<host=192.168.8.176,port=6379,db=0>>>
9.  
10. >>> result = pipe.execute()
11.  
12. >>> print result
13. ['8', '9', '10'] #有序的列表
14. >>>

3.###生产环境批量读写
总结：redis的pipeline就是这么简单，实际生产环境，
根据需要去编写相应的代码,思路同理。如下:

1. redis_db = redis.Redis(host='127.0.0.1',port=6379)
2. data = ['zhangsan', 'lisi', 'wangwu']
3.  
4. with redis_db.pipeline(transaction=False) as pipe:
5.      for i in data:
6.      pipe.zscore(self.key, i)
7.  
8.      result = pipe.execute()
9.  
10. print result
11. # [100, 80, 78]

线上的redis一般都是集群模式，集群模式下使用pipeline的时候，在创建pipeline的对象时，需要指定

pipe =conn.pipeline(transaction=False)

经过线上实测，利用pipeline取值3500条数据，大约需要900ms，如果配合线程or协程来使用，每秒返回1W数据是没有问题的，基本能满足大部分业务。