performance improvement: add redis Connection pool

[dawnbreaks]

Hi, all
I think there is no need to make a tcp connection to redis server everytime you execute a udf function, a connection pool would be great for performance.

anybody could provide a patch for this enhancement? Or give me your opinions.

[ichiriac]

+1 i'm agree with you - opening and closing connexions on redis it's a perf killer for both mysql & redis

[jackeylu]

Well, as my opinion, I think MySQL2Redis is not a good choice to dump data from MySQL to Redis very frequently.

And, the hot pot of performance may be not in the redis server.

If we can prepare all the data firstly then dump them into redis with one or N (could be any number less than 1024) tcp connection from udp to redis, the performance could be improved a lot.

I am busy not for my job, but I can try to fix it in my part-time, thanks for your suggestion.

[dawnbreaks]

Hi, jackey
UDP maybe a better good choice, but that 's not best. An SQL may update thousands of rows in a table, in such a situation you need a queue to cache all the rows and a backend thread to poll the queue and update the redis asynchronously.

I have implemented a similar project based on this project. See https://github.com/dawnbreaks/mysql2redis

Thanks & Best Regards.

[jackeylu]

hi dawnbreaks, thank you for your information.

I am so glad that you do a better improvement for this project.

May I have a little suggestion? Can you provide a function that the data in a MySQL server can dump into more than one redis server?

The "More" may be 2 or 3, sometimes, we may have some backup of redis. but, people can sync data between redises with the mechanism of redis.

[dawnbreaks]

hi, jackey
Why you need a udf to dump data from mysql to redis? If you use redis as a cache server only , redis should only cache the hot data. If the dataset is small, you could clean the data in redis every midnight; if a large dataset, you need a salve server to backup the master redis.

Also you could consider abandon mysql and use redis as a main storage server.

Best Regards!

[jackeylu]

dawnbreaks, you are right, The solution of MySQL2Redis is odd, but it is really occurred. Why I created such a project? That is came from an odd requirement of one of my web site projects.

We was unable to change our architecture at that time, so MySQL2Redis such a temp solution came.