自动玩微信小游戏跳一跳

中文说明请点这里

最新修正了一下各种aspect ratio的小问题，如果还有不能正确运行的问题，请发issue.

如果修改sensitivity觉得效果不理想，一般是分辨率不是16:9，导致和resource匹配不准，这种情况建议试一试nn_play.py.

Requirements

• Python
• Opencv3
• Tensorflow (if using nn_play.py)

for Android

• Adb tools
• Android Phone

for IOS (Refer to this site for installation)

• iPhone
• Mac
• WebDriverAgent
• facebook-wda
• imobiledevice

Algorithms for Localization

• Multiscale-search
• CV based fast-search
• Convolutional Neural Network based coarse-to-fine model

For algorithm details, please go to https://zhuanlan.zhihu.com/p/32636329.

Notice: CV based fast-search only support Android for now

Run

It is recommended to run the following if have an android phone

python play.py --phone Android --sensitivity 2.045

If you have an iPhone, download the model following the link below, and run the following

python nn_play.py --phone IOS --sensitivity 2.045

• --phone has two options: Android or IOS.
• --sensitivity is the constant parameter that controls the pressing time.
• play.py using algorithm based on CV, support Android and IOS
• nn_play.py using algorithm based on Convolutional Neural Network, support Android and IOS, recommend for IOS

Performance

Our method can correctly detect the positions of the man (green dot) and the destination (red dot).

It is easy to reach the state of art as long as you like. But I choose to go die after 859 jumps for about 1.5 hours.

Demo Video

Here is a video demo. Excited!

Train Log & Data

CNN train log and train&validation data avaliable at

• Baidu Drive
• Google Drive

Training: download and untar data into any directory, and then modify self.data_dir in those files under cnn_coarse_to_fine/data_provider directory.

Inference: download and unzip train log dirs(train_logs_coarse and train_logs_fine) into resource directory.

How to Train CNN models by yourself?

0. Download and untar data into any directory, and then modify self.data_dir in those files under cnn_coarse_to_fine/data_provider directory.
1. base.large is model dir for coarse model, base.fine is model dir for fine model, other dirs under cnn_coarse_to_fine/config are models we don't use, but if you have interests, you can try train other models by yourself.
2. Run python3 train.py -g 0 to train your model, -g to specify GPU to use, if you don't have GPU, training model is not recommended because training speed with CPU is very slow.
3. After training, mv or copy .ckpt file to train log dirs(train_logs_coarse and train_logs_fine) for use.