-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
1457 lines (635 loc) · 79.3 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html class="theme-next muse use-motion" lang="zh-Hans">
<head>
<meta charset="UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1"/>
<meta name="theme-color" content="#222">
<meta http-equiv="Cache-Control" content="no-transform" />
<meta http-equiv="Cache-Control" content="no-siteapp" />
<link href="/lib/font-awesome/css/font-awesome.min.css?v=4.6.2" rel="stylesheet" type="text/css" />
<link href="/css/main.css?v=6.0.1" rel="stylesheet" type="text/css" />
<link rel="apple-touch-icon" sizes="180x180" href="/images/apple-touch-icon-next.png?v=6.0.1">
<link rel="icon" type="image/png" sizes="32x32" href="/images/favicon-32x32-next.png?v=6.0.1">
<link rel="icon" type="image/png" sizes="16x16" href="/images/favicon-16x16-next.png?v=6.0.1">
<link rel="mask-icon" href="/images/logo.svg?v=6.0.1" color="#222">
<meta name="keywords" content="Hexo, NexT" />
<meta property="og:type" content="website">
<meta property="og:title" content="分类">
<meta property="og:url" content="http://c4x.github.io/categories/index.html">
<meta property="og:site_name" content="Cfour's Blog">
<meta property="og:locale" content="zh-Hans">
<meta property="og:updated_time" content="2018-01-12T09:42:17.000Z">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="分类">
<script type="text/javascript" id="hexo.configurations">
var NexT = window.NexT || {};
var CONFIG = {
root: '/',
scheme: 'Muse',
version: '6.0.1',
sidebar: {"position":"left","display":"post","offset":12,"b2t":false,"scrollpercent":false,"onmobile":false},
fancybox: false,
fastclick: false,
lazyload: false,
tabs: true,
motion: {"enable":true,"async":false,"transition":{"post_block":"fadeIn","post_header":"slideDownIn","post_body":"slideDownIn","coll_header":"slideLeftIn","sidebar":"slideUpIn"}},
algolia: {
applicationID: '',
apiKey: '',
indexName: '',
hits: {"per_page":10},
labels: {"input_placeholder":"Search for Posts","hits_empty":"We didn't find any results for the search: ${query}","hits_stats":"${hits} results found in ${time} ms"}
}
};
</script>
<link rel="canonical" href="http://c4x.github.io/categories/"/>
<title>Cfour's Blog</title>
</head>
<body itemscope itemtype="http://schema.org/WebPage" lang="zh-Hans">
<div class="container sidebar-position-left
page-home">
<div class="headband"></div>
<header id="header" class="header" itemscope itemtype="http://schema.org/WPHeader">
<div class="header-inner"> <div class="site-brand-wrapper">
<div class="site-meta ">
<div class="custom-logo-site-title">
<a href="/" class="brand" rel="start">
<span class="logo-line-before"><i></i></span>
<span class="site-title">Cfour's Blog</span>
<span class="logo-line-after"><i></i></span>
</a>
</div>
<p class="site-subtitle">Cfour's Blog</p>
</div>
<div class="site-nav-toggle">
<button>
<span class="btn-bar"></span>
<span class="btn-bar"></span>
<span class="btn-bar"></span>
</button>
</div>
</div>
<nav class="site-nav">
<ul id="menu" class="menu">
<li class="menu-item menu-item-home">
<a href="/" rel="section">
<i class="menu-item-icon fa fa-fw fa-home"></i> <br />
首页</a>
</li>
<li class="menu-item menu-item-tags">
<a href="/tags/" rel="section">
<i class="menu-item-icon fa fa-fw fa-tags"></i> <br />
标签</a>
</li>
<li class="menu-item menu-item-categories">
<a href="/categories/" rel="section">
<i class="menu-item-icon fa fa-fw fa-th"></i> <br />
分类</a>
</li>
<li class="menu-item menu-item-archives">
<a href="/archives/" rel="section">
<i class="menu-item-icon fa fa-fw fa-archive"></i> <br />
归档</a>
</li>
</ul>
</nav>
</div>
</header>
<main id="main" class="main">
<div class="main-inner">
<div class="content-wrap">
<div id="content" class="content">
<section id="posts" class="posts-expand">
<article class="post post-type-normal" itemscope itemtype="http://schema.org/Article">
<div class="post-block">
<link itemprop="mainEntityOfPage" href="http://c4x.github.io/2018/08/14/Django_Rest_FrameWork/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="name" content="Cfour's Blog">
<meta itemprop="description" content="">
<meta itemprop="image" content="/default_avatar.png">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Cfour's Blog">
</span>
<header class="post-header">
<h1 class="post-title" itemprop="name headline">
<a class="post-title-link" href="/2018/08/14/Django_Rest_FrameWork/" itemprop="url">Django Rest FrameWork使用上的一些小技巧</a></h1>
<div class="post-meta">
<span class="post-time">
<span class="post-meta-item-icon">
<i class="fa fa-calendar-o"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建于" itemprop="dateCreated datePublished" datetime="2018-08-14T15:27:15+08:00">2018-08-14</time>
</span>
<span class="post-category" >
<span class="post-meta-divider">|</span>
<span class="post-meta-item-icon">
<i class="fa fa-folder-o"></i>
</span>
<span class="post-meta-item-text">分类于</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/Python/" itemprop="url" rel="index">
<span itemprop="name">Python</span>
</a>
</span>
，
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/Python/Django/" itemprop="url" rel="index">
<span itemprop="name">Django</span>
</a>
</span>
</span>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<p>不得不说Django Rest FrameWork使用还是很方便的，但是在日常中，经常遇到一些小问题让人头痛，索性专门开个文章，来一一记录日常遇到的一些问题和解决方法：</p>
<h2 id="serializer的字符串数组问题"><a href="#serializer的字符串数组问题" class="headerlink" title="serializer的字符串数组问题"></a>serializer的字符串数组问题</h2><p>在Django中，如果你的Model有一个ForeignKey的关联，并且你在Api做查询时将这个关联以prefetch_related形式全部查询出来了，最典型的就是图集：<br>例:<br>在model中定义：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ModelA</span><span class="params">(models.Model)</span>:</span></span><br><span class="line"> name = models.CharField(<span class="string">u'xxxx'</span>, max_length=<span class="number">10</span>)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ModelB</span><span class="params">(models.Model)</span>:</span></span><br><span class="line"> modela = models.ForeignKey(ModelA, verbose_name=<span class="string">u'oooo'</span>, related_name=<span class="string">'pics'</span>)</span><br><span class="line"> pic = models.ImageField(<span class="string">u'图'</span>, upload_to=<span class="string">''</span>, max_length=<span class="number">100</span>,blank=<span class="keyword">True</span>, null=<span class="keyword">True</span>)</span><br></pre></td></tr></table></figure>
<p>在Api中：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ModelAApiView</span><span class="params">(generics.RetrieveAPIView)</span>:</span></span><br><span class="line"> serializer_class = ModelASerializer</span><br><span class="line"> queryset = ModelA.objects.all().prefetch_related(<span class="string">"pics"</span>)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ModelBSerializer</span><span class="params">(serializers.ModelSerializer)</span>:</span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Meta</span>:</span></span><br><span class="line"> model = ModelB</span><br><span class="line"> fields = (<span class="string">'pic'</span>)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ModelASerializer</span><span class="params">(serializers.ModelSerializer)</span>:</span></span><br><span class="line"> pics = ModelBSerializer(many=<span class="keyword">True</span>)</span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Meta</span>:</span></span><br><span class="line"> model = ModelA</span><br><span class="line"> fields = (<span class="string">'name'</span>, <span class="string">'pics'</span>)</span><br></pre></td></tr></table></figure>
<p>这是通常的作坊，将ForeignKey的关联对象全部转换到主对象的json里面去，对应会生成的json就像这样：</p>
<figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> <span class="attr">"name"</span>:<span class="string">""</span>,</span><br><span class="line"> <span class="attr">"pics"</span>:[</span><br><span class="line"> {<span class="attr">"pic"</span>:<span class="string">"xxxx.jpg"</span>},{<span class="attr">"pic"</span>:<span class="string">"oooo.jpg"</span>}</span><br><span class="line"> ]</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>通常来说，这样就已经完结了，直到有一天，Android的小伙伴说，如果pic只是传一个图片地址的话，为什么不直接通过json的数组传递过来，还要通过数组-对象的形式传递呢。<br>后来，通过查找<a href="https://www.django-rest-framework.org/api-guide/relations/" target="_blank" rel="noopener">Rest FrameWork</a>，我找到了方法。可以改写ModelASerializer为</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ModelASerializer</span><span class="params">(serializers.ModelSerializer)</span>:</span></span><br><span class="line"> pics = serializers.StringRelatedField(many=<span class="keyword">True</span>)</span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Meta</span>:</span></span><br><span class="line"> model = ModelA</span><br><span class="line"> fields = (<span class="string">'name'</span>, <span class="string">'pics'</span>)</span><br></pre></td></tr></table></figure>
<p>这样，pics序列化出来就不是数组-对象而变成了数组，同时将使用对象的 <em><strong>unicode</strong></em> 方法。 之后json会变为：<br><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> <span class="attr">"name"</span>:<span class="string">""</span>,</span><br><span class="line"> <span class="attr">"pics"</span>:[</span><br><span class="line"> <span class="string">"xxxx.jpg"</span>,<span class="string">"oooo.jpg"</span></span><br><span class="line"> ]</span><br><span class="line">}</span><br></pre></td></tr></table></figure></p>
<h2 id="Token的失效问题"><a href="#Token的失效问题" class="headerlink" title="Token的失效问题"></a>Token的失效问题</h2><p>在Rest FrameWork中，TokenAuthentication是通过数据库表来存储的。 同时，它只记录了Token，Created， 如果，我们需要给一个Token设置过期时间，可以这样办：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> datetime <span class="keyword">import</span> datetime,timedelta</span><br><span class="line"><span class="keyword">from</span> rest_framework.authentication <span class="keyword">import</span> TokenAuthentication</span><br><span class="line"><span class="keyword">from</span> rest_framework <span class="keyword">import</span> exceptions</span><br><span class="line"><span class="keyword">from</span> django.conf <span class="keyword">import</span> settings</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ExpiringTokenAuthentication</span><span class="params">(TokenAuthentication)</span>:</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">authenticate_credentials</span><span class="params">(self, key)</span>:</span></span><br><span class="line"> model = self.get_model()</span><br><span class="line"> <span class="keyword">try</span>:</span><br><span class="line"> token = model.objects.get(key=key)</span><br><span class="line"> <span class="keyword">except</span> model.DoesNotExist:</span><br><span class="line"> <span class="keyword">raise</span> exceptions.AuthenticationFailed(<span class="string">'Invalid token.'</span>)</span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> token.user.is_active:</span><br><span class="line"> <span class="keyword">raise</span> exceptions.AuthenticationFailed(<span class="string">'User inactive or deleted.'</span>)</span><br><span class="line"> now = datetime.now()</span><br><span class="line"> <span class="keyword">if</span> token.created < now - timedelta(seconds=settings.SESSION_COOKIE_AGE):</span><br><span class="line"> <span class="keyword">raise</span> exceptions.AuthenticationFailed(<span class="string">'Token has expired'</span>)</span><br><span class="line"> <span class="keyword">if</span> token.created + timedelta(seconds=settings.SESSION_UPDATE_AGE) < now:</span><br><span class="line"> token.created = now</span><br><span class="line"> token.save(update_fields=[<span class="string">'created'</span>])</span><br><span class="line"> <span class="keyword">return</span> (token.user, token)</span><br></pre></td></tr></table></figure>
<p>同时，在setting中配置：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">REST_FRAMEWORK = {</span><br><span class="line"> <span class="string">'DEFAULT_AUTHENTICATION_CLASSES'</span>: (</span><br><span class="line"> <span class="string">'apps.common.authentication.ExpiringTokenAuthentication'</span>,</span><br><span class="line"> )</span><br><span class="line">}</span><br><span class="line">SESSION_COOKIE_AGE=<span class="number">3600</span></span><br></pre></td></tr></table></figure>
<p>这样，在每次请求过来之后，就能够去刷新token的时间，并保证过期的token不被认证。 当然， 以上代码也可以改造成使用redis的方式。</p>
<h2 id="prefetch-related的条件搜索"><a href="#prefetch-related的条件搜索" class="headerlink" title="prefetch_related的条件搜索"></a>prefetch_related的条件搜索</h2><p>当使用prefetch_related时，Django默认是进行了所有关联对象的取出操作，以一个相册为例：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">PhotoAlbum</span><span class="params">(models.Model)</span>:</span></span><br><span class="line"> title = models.CharField(max_length=<span class="number">128</span>)</span><br><span class="line"> author = models.CharField(max_length=<span class="number">128</span>)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Photo</span><span class="params">(models.Model)</span>:</span></span><br><span class="line"> album = models.ForeignKey(<span class="string">'PhotoAlbum'</span>)</span><br><span class="line"> status = models.BooleanField(default=<span class="keyword">False</span>)</span><br></pre></td></tr></table></figure>
<p>当需要筛选出author为Davey Jones的所有图片时，一般使用</p>
<blockquote>
<p>PhotoAlbum.objects.filter(author=”Davey Jones”).prefetch_related(“photo_set”)</p>
</blockquote>
<p>此时，如果需要筛选出所有有效的图片，一般不得不进行一次循环或者表推导。<br>当Django版本大于1.7时，有一个推荐的对象<a href="https://docs.djangoproject.com/en/2.1/ref/models/querysets/#django.db.models.Prefetch" target="_blank" rel="noopener">Prefetch</a>，进行如下搜索</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">PhotoAlbum.objects.filter(author=<span class="string">"Davey Jones"</span>).prefetch_related(</span><br><span class="line"> Prefetch(</span><br><span class="line"> <span class="string">"photo_set"</span>,</span><br><span class="line"> queryset=Photo.objects.filter(status=<span class="keyword">True</span>)</span><br><span class="line"> )</span><br><span class="line">)</span><br></pre></td></tr></table></figure>
<h2 id="APIView中进行分页"><a href="#APIView中进行分页" class="headerlink" title="APIView中进行分页"></a>APIView中进行分页</h2><p>如果使用generics.ListAPIView进行分页，详细很多人都应该知道直接实现对应的get_queryset方法以及实现对应的Serializer对象就可以了。但是如果要使用APIView呢？首先，实现一个类：</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">PaginationAPIView</span><span class="params">(APIView)</span>:</span></span><br><span class="line"> pagination_class = LimitOffsetPagination</span><br><span class="line"></span><br><span class="line"><span class="meta"> @property</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">paginator</span><span class="params">(self)</span>:</span></span><br><span class="line"> <span class="keyword">if</span> <span class="keyword">not</span> hasattr(self, <span class="string">'_paginator'</span>):</span><br><span class="line"> <span class="keyword">if</span> self.pagination_class <span class="keyword">is</span> <span class="keyword">None</span>:</span><br><span class="line"> self._paginator = <span class="keyword">None</span></span><br><span class="line"> <span class="keyword">else</span>:</span><br><span class="line"> self._paginator = self.pagination_class()</span><br><span class="line"> <span class="keyword">return</span> self._paginator</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">paginate_queryset</span><span class="params">(self, queryset)</span>:</span></span><br><span class="line"> <span class="keyword">if</span> self.paginator <span class="keyword">is</span> <span class="keyword">None</span>:</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">None</span></span><br><span class="line"> <span class="keyword">return</span> self.paginator.paginate_queryset(queryset, self.request, view=self)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">get_paginated_response</span><span class="params">(self, data)</span>:</span></span><br><span class="line"> <span class="keyword">assert</span> self.paginator <span class="keyword">is</span> <span class="keyword">not</span> <span class="keyword">None</span></span><br><span class="line"> <span class="keyword">return</span> self.paginator.get_paginated_response(data)</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">MyView</span><span class="params">(APIView)</span>:</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">get</span><span class="params">(self,request,*args,**kwargs)</span>:</span></span><br><span class="line"> queryset = ....</span><br><span class="line"> page = self.paginate_queryset(queryset)</span><br><span class="line"> serializer = Serializer(page, many=<span class="keyword">True</span>)</span><br><span class="line"> <span class="keyword">return</span> self.get_paginated_response(serializer.data)</span><br></pre></td></tr></table></figure>
<p>这样，就可以在使用APIView对结果进行分页处理</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</div>
</article>
<article class="post post-type-normal" itemscope itemtype="http://schema.org/Article">
<div class="post-block">
<link itemprop="mainEntityOfPage" href="http://c4x.github.io/2017/06/30/Nginx_Cut_image/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="name" content="Cfour's Blog">
<meta itemprop="description" content="">
<meta itemprop="image" content="/default_avatar.png">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Cfour's Blog">
</span>
<header class="post-header">
<h1 class="post-title" itemprop="name headline">
<a class="post-title-link" href="/2017/06/30/Nginx_Cut_image/" itemprop="url">Nginx剪裁图片</a></h1>
<div class="post-meta">
<span class="post-time">
<span class="post-meta-item-icon">
<i class="fa fa-calendar-o"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建于" itemprop="dateCreated datePublished" datetime="2017-06-30T20:27:15+08:00">2017-06-30</time>
</span>
<span class="post-category" >
<span class="post-meta-divider">|</span>
<span class="post-meta-item-icon">
<i class="fa fa-folder-o"></i>
</span>
<span class="post-meta-item-text">分类于</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/Linux/" itemprop="url" rel="index">
<span itemprop="name">Linux</span>
</a>
</span>
</span>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<p>在平时工作过程中，nginx常常用来做静态资源的代理服务器，例如： 图片、样式等。随着移动时代的到来，对于图片尺寸的要求也越来越多了，以往在图片上传后进行处理的方式已经很难保证日常的需要，很早以前就写过一个关于nginx自动剪裁图片尺寸的配置。</p>
<h2 id="前期准备"><a href="#前期准备" class="headerlink" title="前期准备"></a>前期准备</h2><ol>
<li>安装<a href="https://github.com/openresty/lua-nginx-module/tags" target="_blank" rel="noopener">lua-nginx-module</a></li>
<li>安装<a href="http://www.graphicsmagick.org/README.html" target="_blank" rel="noopener">GraphicsMagick</a></li>
</ol>
<h2 id="nginx配置文件的修改"><a href="#nginx配置文件的修改" class="headerlink" title="nginx配置文件的修改"></a>nginx配置文件的修改</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br></pre></td><td class="code"><pre><span class="line">server {</span><br><span class="line"> listen 80;</span><br><span class="line"> server_name 127.0.0.1;</span><br><span class="line"> root /home/www;</span><br><span class="line"></span><br><span class="line"> location ~* \.(txt|ico)$ {</span><br><span class="line"> root /home/www/static;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> location / {</span><br><span class="line"> alias /home/www/static/assets/;</span><br><span class="line"> expires 30d;</span><br><span class="line"> }</span><br><span class="line"> location ^~ /upload/ {</span><br><span class="line"> root /home/www/static/;</span><br><span class="line"> error_page 404 /upload/default.png;</span><br><span class="line"> if (!-f $file) {</span><br><span class="line"> rewrite "/upload/default.png" last;</span><br><span class="line"> }</span><br><span class="line"> expires max;</span><br><span class="line"> tcp_nodelay off;</span><br><span class="line"> tcp_nopush on;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> location /images/ {</span><br><span class="line"> set $image_root /home/www/static/upload;</span><br><span class="line"></span><br><span class="line"> if ($uri !~ "/images/([a-zA-Z\w]+)/([0-9]{4})/([0-9]{2})/([0-9]{2})/([0-9]+).(.*)!(.*)"){</span><br><span class="line"> rewrite "/images/(.+)" /upload/$1 last;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> if ($uri ~* "/images/([a-zA-Z\w]+)/([0-9]{4})/([0-9]{2})/([0-9]{2})/([0-9]+).(.*)!(.*)"){</span><br><span class="line"> set $filePath "/$1/$2/$3/$4/$5.$6";</span><br><span class="line"> set $reqPath "/$1/$2/$3/$4/$5.$7";</span><br><span class="line"> }</span><br><span class="line"> set $file "$image_root$reqPath";</span><br><span class="line"> if (-f $file) {</span><br><span class="line"> rewrite "/images/(.+)" /upload$reqPath last;</span><br><span class="line"> }</span><br><span class="line"> if (!-f $file) {</span><br><span class="line"> rewrite_by_lua '</span><br><span class="line"> local index = string.find(ngx.var.reqPath, "([0-9]+)x([0-9]+)");</span><br><span class="line"> local area = string.sub(ngx.var.reqPath, index);</span><br><span class="line"> index = string.find(area, "([.])");</span><br><span class="line"> size = string.sub(area, 0, index-1);</span><br><span class="line"> local err,command = pcall(function() return "gm convert -resize " .. size .. "! " .. ngx.var.image_root .. ngx.var.filePath .. " " .. ngx.var.file end);</span><br><span class="line"> if err then</span><br><span class="line"> result = os.execute(command);</span><br><span class="line"> if result then</span><br><span class="line"> ngx.redirect("/upload" .. ngx.var.reqPath);</span><br><span class="line"> else</span><br><span class="line"> ngx.redirect("/upload" .. ngx.var.filePath);</span><br><span class="line"> end;</span><br><span class="line"> else</span><br><span class="line"> ngx.redirect("/upload" .. ngx.var.filePath);</span><br><span class="line"> end;</span><br><span class="line"> ';</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>其实，图片的裁剪工作重头戏就在这个配置文件上，下面来解释一下配置文件。</p>
<h2 id="配置文件解读"><a href="#配置文件解读" class="headerlink" title="配置文件解读"></a>配置文件解读</h2><h3 id="location-upload"><a href="#location-upload" class="headerlink" title="location ^~ /upload/"></a>location ^~ /upload/</h3><p>此处配置只是单纯的在配置找不到的情况下，将一张默认的图片返回。</p>
<h3 id="location-images"><a href="#location-images" class="headerlink" title="location /images/"></a>location /images/</h3><p>此处的配置则是进行裁剪的关键</p>
<ol>
<li><p>首先，设置一个变量为图片的根目录</p>
</li>
<li><p>判断uri是否复合格式</p>
</li>
</ol>
<p>当前图片的访问路径为/images/models/years/month/day/picname.ext!picsize，例如：/images/user/2017/06/30/123456789.png!200x300.png</p>
<ol>
<li>将uri用正则表达式切分并赋值</li>
</ol>
<p>形成两个参数，以2部分为例：</p>
<blockquote>
<p>filePath: /user/2017/06/30/123456789.png<br>reqPath: /user/2017/06/30/123456789.200x300.png</p>
</blockquote>
<ol>
<li>对需要的尺寸图片全路径进行判断，是否有该尺寸图片</li>
</ol>
<p>使用上一步定义的两个参数，拼接成目标文件路径并赋值给file：</p>
<blockquote>
<p>file: $image_root$reqPath = /home/www/static/upload/user/2017/06/30/123456789.200x300.png</p>
</blockquote>
<p>判断文件是否存在，如果存在，直接返回对应文件。</p>
<ol>
<li>如果文件不存在，使用lua脚本，进行尺寸转换</li>
</ol>
<p>定义lua脚本，获取目标尺寸，并将原图片使用GraphicsMagick转换成目标大小， 也就是将第三步的filePath通过gm covert命令进行裁切，转换成reqPath中的文件。<br>同时，对command执行进行异常捕获，如果遇到异常，则使用原文件，这样可以防止突发情况导致nginx报错，从而没有图片的返回。</p>
<h2 id="后记"><a href="#后记" class="headerlink" title="后记"></a>后记</h2><p>使用这种方法，可以安全的转换图片到类似于webq等格式，同时，可以使图片尺寸按照需要的得以返回，从而减少了后台对图片进行裁切操作。缺陷在于，第一次访问时，图片的转换可能需要时间，但是，可以通过一些自动化脚本进行第一次图片的常见尺寸访问，从而加快速度。<br>同时，现在也可以使用云存储等手段达到目的。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</div>
</article>
<article class="post post-type-normal" itemscope itemtype="http://schema.org/Article">
<div class="post-block">
<link itemprop="mainEntityOfPage" href="http://c4x.github.io/2017/06/13/Java_Tips/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="name" content="Cfour's Blog">
<meta itemprop="description" content="">
<meta itemprop="image" content="/default_avatar.png">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Cfour's Blog">
</span>
<header class="post-header">
<h1 class="post-title" itemprop="name headline">
<a class="post-title-link" href="/2017/06/13/Java_Tips/" itemprop="url">Java中的一些注意事项收集</a></h1>
<div class="post-meta">
<span class="post-time">
<span class="post-meta-item-icon">
<i class="fa fa-calendar-o"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建于" itemprop="dateCreated datePublished" datetime="2017-06-13T15:27:15+08:00">2017-06-13</time>
</span>
<span class="post-category" >
<span class="post-meta-divider">|</span>
<span class="post-meta-item-icon">
<i class="fa fa-folder-o"></i>
</span>
<span class="post-meta-item-text">分类于</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/Java/" itemprop="url" rel="index">
<span itemprop="name">Java</span>
</a>
</span>
</span>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="Null-的过度使用"><a href="#Null-的过度使用" class="headerlink" title="Null 的过度使用"></a>Null 的过度使用</h2><p>避免过度使用 null 值是一个最佳实践。例如，更好的做法是让方法返回空的 array 或者 collection 而不是 null 值，因为这样可以防止程序抛出 NullPointerException。例如：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">List<String> pics = album.getPics(); </span><br><span class="line"><span class="keyword">for</span> (String pic : pics) { </span><br><span class="line"> processPic(pic);</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>如果此时album没有任何图片，在getPics()的时候将会返回一个null，程序也会抛出NullPointerException。因此需要加入空检查来解决这个问题。如果将返回的 null 值替换成一个空的 list，那么 NullPointerException 也不会出现。而且，因为我们不再需要对变量 accountId 做空检查，代码将变得更加简洁。</p>
<p>在Java8中，可以使用 Optional 类型，它既可以是一个空对象，也可以是一些值的封装。</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Optional<String> optionalString = Optional.ofNullable(nullableString); </span><br><span class="line"><span class="keyword">if</span>(optionalString.isPresent()) { </span><br><span class="line"> System.out.println(optionalString.get());</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<h2 id="并发修改"><a href="#并发修改" class="headerlink" title="并发修改"></a>并发修改</h2><p>如果要删除List中的某个元素应该怎么做？</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">List<IHat> hats = <span class="keyword">new</span> ArrayList<>();</span><br><span class="line"><span class="keyword">for</span> (IHat hat : hats) {</span><br><span class="line"> <span class="keyword">if</span> (hat.hasEarFlaps()) {</span><br><span class="line"> hats.remove(hat);</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>有的新人会选择这样做，实际上这样做会获得一个ConcurrentModificationException异常，因为代码在遍历这个集合的同时对其进行修改。<br>直接的解决方案是将要删除的元素放进一个 list，之后用另一个循环删除它。不过这需要一个额外的集合来存放将要被删除的元素。<br>另一种方法是使用Iterator.remove：</p>
<figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">Iterator<IHat> hatIterator = hats.iterator(); </span><br><span class="line"><span class="keyword">while</span> (hatIterator.hasNext()) { </span><br><span class="line"> IHat hat = hatIterator.next();</span><br><span class="line"> <span class="keyword">if</span> (hat.hasEarFlaps()) {</span><br><span class="line"> hatIterator.remove();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure>
<p>在Java8中，可以使用List.removeIf 方法。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</div>
</article>
<article class="post post-type-normal" itemscope itemtype="http://schema.org/Article">
<div class="post-block">
<link itemprop="mainEntityOfPage" href="http://c4x.github.io/2017/06/08/GitLab_CI_Docker/">
<span hidden itemprop="author" itemscope itemtype="http://schema.org/Person">
<meta itemprop="name" content="Cfour's Blog">
<meta itemprop="description" content="">
<meta itemprop="image" content="/default_avatar.png">
</span>
<span hidden itemprop="publisher" itemscope itemtype="http://schema.org/Organization">
<meta itemprop="name" content="Cfour's Blog">
</span>
<header class="post-header">
<h1 class="post-title" itemprop="name headline">
<a class="post-title-link" href="/2017/06/08/GitLab_CI_Docker/" itemprop="url">使用GitLab-ci做CI以及部署到Docker环境</a></h1>
<div class="post-meta">
<span class="post-time">
<span class="post-meta-item-icon">
<i class="fa fa-calendar-o"></i>
</span>
<span class="post-meta-item-text">发表于</span>
<time title="创建于" itemprop="dateCreated datePublished" datetime="2017-06-08T15:27:15+08:00">2017-06-08</time>
</span>
<span class="post-category" >
<span class="post-meta-divider">|</span>
<span class="post-meta-item-icon">
<i class="fa fa-folder-o"></i>
</span>
<span class="post-meta-item-text">分类于</span>
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/Linux/" itemprop="url" rel="index">
<span itemprop="name">Linux</span>
</a>
</span>
，
<span itemprop="about" itemscope itemtype="http://schema.org/Thing">
<a href="/categories/Linux/CI/" itemprop="url" rel="index">
<span itemprop="name">CI</span>
</a>
</span>
</span>
</div>
</header>
<div class="post-body" itemprop="articleBody">
<h2 id="使用GitLab-ci做CI以及部署到Docker环境"><a href="#使用GitLab-ci做CI以及部署到Docker环境" class="headerlink" title="使用GitLab-ci做CI以及部署到Docker环境"></a>使用GitLab-ci做CI以及部署到Docker环境</h2><p>最近在尝试通过gitlab-ci-runner进行项目的test，build。并将build之后的项目发布到一个docker容器中，作为测试环境。</p>
<p>不妨先定一个小目标，建立一个Django项目，不使用任何数据库，只输出一个“Hello， world。”在页面。我的目标就是，在项目提交到gitlab上的时候，gitlab-ci-runner将会执行Django的test，在test通过之后，形成一个待执行操作，如果选择该操作，这个项目将会被部署到docker的一个容器中去。</p>
<hr>
<h2 id="Gitlab-ci-runner"><a href="#Gitlab-ci-runner" class="headerlink" title="Gitlab-ci-runner"></a>Gitlab-ci-runner</h2><h3 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h3><p>先了解一下什么叫做<a href="https://docs.gitlab.com.cn/ce/ci/quick_start/README.html" target="_blank" rel="noopener">Runner</a></p>
<blockquote>
<p>GitLab提供可持续集成服务。只要在你的仓库根目录 [创建一个.gitlab-ci.yml 文件][yaml语法]， 并为该项目指派一个Runner，当有合并请求或者 push的时候就会触发CI pipeline。</p>
</blockquote>
<p>所以，将分成这样几个步骤：</p>
<pre><code>1. 首先应该安装一个Runner，并将它注册到Gitlab里面去。
2. 为项目增加一個.gitlab-ci.yml文件，然后push到项目里面。
</code></pre><p>安装Runner，可以看<a href="https://docs.gitlab.com.cn/runner/" target="_blank" rel="noopener">这里</a>。<br>对了，由于国内不可抗拒的原因，所以发一個<a href="https://mirror.tuna.tsinghua.edu.cn/help/gitlab-ce/" target="_blank" rel="noopener">清华大学开源软件镜像站</a>给大家。</p>
<p>現在，假设各位都安装好了Gitlab和Gitlab-ci-runner，並且已运行行好了Gitlab。</p>
<h3 id="注册"><a href="#注册" class="headerlink" title="注册"></a>注册</h3><p>现在，我们将runner注册到Gitlab中去：<br>在gitlab中，找到runner的token，</p>
<img src="/2017/06/08/GitLab_CI_Docker/runnerconfig.png" title="[runnerconfig]">
<p>register的时候需要使用到ci server，以及token，ci server在gitlab 8以上以及集成到了gitlab中，地址时 <a href="http://doman.com/ci" target="_blank" rel="noopener">http://doman.com/ci</a> ， 以本机为例就是 <a href="http://localhost/ci" target="_blank" rel="noopener">http://localhost/ci</a> ， 然后运行：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gitlab-ci-multi-runner register</span><br></pre></td></tr></table></figure>
<p>在这里，命令行会要求你提供ci server地址，token，以及为runner取一个tags。其次，还有诸如是否可以运行untagged等选择。这里，为了方便测试runner先将untagged选择为true。然后，选择runner executor，<a href="https://docs.gitlab.com/runner/executors/#selecting-the-executor" target="_blank" rel="noopener">executor</a>有很多种，这里主要说两种：</p>
<pre><code>1. Shell， shell表示在runner运行时，将会以当前机器的Base环境来运行build操作。
2. docker， docker表示在构建时，runner会按照。gitlab-ci.yml里面指定的images去创建一个docker容器，并运行构建。
</code></pre><p>这里，我选择的是shell。这样，一个基本的runner已经被注册到了gitlab中，当以这种方式注册runner时，runner被申明为Shared Runners，另外还有一个Specific Runners，Shared Runners可以理解为全局使用的runner，而另外一个是为了满足某些特殊的项目而启动的runner，能够在project->setting里面注册。</p>
<p>另外，注意一点，所有shell-runner执行的命令都是以gitlab-runner或者gitlab_ci_multi_runner来运行的。所以，为了使用docker，要将gitlab-runner加入到docker用户组中：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">usermod -aG docker gitlab-runner</span><br></pre></td></tr></table></figure>
<hr>
<h2 id="Docker"><a href="#Docker" class="headerlink" title="Docker"></a>Docker</h2><p>Docker的安装就不介绍了，为了方便介绍，只贴一下我build一个python3的Dockerfile吧：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">FROM ubuntu:16.04</span><br><span class="line">MAINTAINER cfour [email protected]</span><br><span class="line">RUN sed -i 's/http:\/\/archive\.ubuntu\.com\/ubuntu\//http:\/\/mirrors\.163\.com\/ubuntu\//g' /etc/apt/sources.list</span><br><span class="line">RUN apt-get -y update && apt-get install -y \</span><br><span class="line">git \</span><br><span class="line">python3 \</span><br><span class="line">python3-pip \</span><br><span class="line">unzip \</span><br><span class="line">fabric</span><br></pre></td></tr></table></figure>
<p>在docker上，只简单的安装了git，python3， pip3， fabric以及unzip。</p>
<hr>
<h2 id="gitlab-ci-yml-文件"><a href="#gitlab-ci-yml-文件" class="headerlink" title=".gitlab-ci.yml 文件"></a>.gitlab-ci.yml 文件</h2><p>现在，假设你的项目已经在gitlab上了，在项目里面添加一个.gitlab-ci.yml文件，请看<a href="https://docs.gitlab.com.cn/ce/ci/yaml/README.html" target="_blank" rel="noopener">.gitlab-ci.yml详解</a><br>我的配置是：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"># image: base:server</span><br><span class="line"></span><br><span class="line">stages:</span><br><span class="line"> - build</span><br><span class="line"> - test</span><br><span class="line"> - deploy</span><br><span class="line"></span><br><span class="line">test:</span><br><span class="line"> stage: test</span><br><span class="line"> script: </span><br><span class="line"> - /home/cfour/.pyenv/shims/python3.6 manage.py test</span><br><span class="line"></span><br><span class="line">build:</span><br><span class="line"> stage: build</span><br><span class="line"> script: </span><br><span class="line"> - /home/cfour/.pyenv/shims/pip3.6 install -r requirments.txt</span><br><span class="line"></span><br><span class="line">deploy-test:</span><br><span class="line"> stage: deploy</span><br><span class="line"> when: manual</span><br><span class="line"> script: </span><br><span class="line"> - /usr/bin/python /home/cfour/workspace/deploy/creat_docker.py $CI_PROJECT_NAME http://root:[email protected]/root/docker-ci-test.git</span><br></pre></td></tr></table></figure>
<p>来解读一下这个十分基础的配置文件。</p>
<table>
<thead>
<tr>
<th>关键词</th>
<th>描述</th>
</tr>
</thead>
<tbody>
<tr>
<td>image</td>
<td>在executor为docker是有效，指定生成容器的docker image</td>
</tr>
<tr>
<td>stages</td>
<td>定义builds阶段，默认即为build，test，deploy。</td>
</tr>
<tr>
<td>script</td>
<td>在当前阶段执行的脚本</td>
</tr>
</tbody>
</table>
<p>还有其他很多可以设置的方面。这里说说我遇到的问题，在executor使用docker时，填写本地build的image tag之后，runner会直接通过docker pull去获取镜像，而不是使用本地镜像。 这个可以通过在runner的配置文件config。toml里增加privileged = false来解决。</p>
<p>配置文件中的$CI_PROJECT_NAME为runner变量，相关变量可以<a href="https://docs.gitlab.com.cn/ce/ci/variables/README.html#variables" target="_blank" rel="noopener">查看</a>。</p>
<p>另外，在我的配置文件里面很多都写了全路径的原因是因为我使用了pyenv，并且项目环境为python3.6。</p>
<h3 id="Jobs"><a href="#Jobs" class="headerlink" title="Jobs"></a>Jobs</h3><p>在我的配置文件中，申明了 test， build， deploy-test， 这个叫做<a href="https://docs.gitlab.com.cn/ce/ci/yaml/README.html#jobs" target="_blank" rel="noopener">Jobs</a>，里面还有许多设置，列几个我觉得重要的：</p>
<table>
<thead>
<tr>
<th>关键词</th>
<th>描述</th>
</tr>
</thead>
<tbody>
<tr>
<td>script</td>
<td>需要执行的脚本，必须参数</td>
</tr>
<tr>
<td>stages</td>
<td>所属阶段，默认为test</td>
</tr>
<tr>
<td>only</td>
<td>在这个git refs上时才会被执行，例如：master</td>
</tr>
<tr>
<td>except</td>
<td>在这个git refs上时不会被执行，例如：develop</td>
</tr>
<tr>
<td>when</td>
<td>定义什么时候执行job，有on_success， on_failure， always， manual四个</td>
</tr>
</tbody>
</table>
<h3 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h3><p>直接push到gitlab上，在项目的piplines里面可以看到：</p>
<img src="/2017/06/08/GitLab_CI_Docker/piplines.png" title="[piplines]">
<p>已经执行了build和test，点击：</p>
<img src="/2017/06/08/GitLab_CI_Docker/piplines_build.png" title="[piplines_build]">
<p>进入可以看到：</p>
<img src="/2017/06/08/GitLab_CI_Docker/build.png" title="[build]">
<p>build的日志，相应的，test日志也可以去看。</p>
<h2 id="部署到docker"><a href="#部署到docker" class="headerlink" title="部署到docker"></a>部署到docker</h2><h3 id="creat-docker-py"><a href="#creat-docker-py" class="headerlink" title="creat_docker.py"></a>creat_docker.py</h3><p>在。gitlab-ci.yml文件里，deploy-test写了when: manual，这个将会生成一个可以手动控制的deploy按钮:</p>
<p>点击这个按钮，将按照script里面写的脚本，执行一个叫做creat_docker的python程序，create_docker内容如下：<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br></pre></td><td class="code"><pre><span class="line"># /usr/bin/python</span><br><span class="line"># encoding: utf-8</span><br><span class="line"># -*- coding: utf8 -*-</span><br><span class="line"></span><br><span class="line">import sys</span><br><span class="line">import docker</span><br><span class="line">import random</span><br><span class="line">import socket</span><br><span class="line">import re</span><br><span class="line">import os</span><br><span class="line"></span><br><span class="line">def is_open(port):</span><br><span class="line"> s = socke.socket(socket。AF_INET, socket.SOCK_STREAM)</span><br><span class="line"> try:</span><br><span class="line"> s.connect(("127.0.0.1", int(port)))</span><br><span class="line"> s.shutdown(2)</span><br><span class="line"> return True</span><br><span class="line"> except:</span><br><span class="line"> return False</span><br><span class="line"></span><br><span class="line">def deploy_project(container， repo_url， c_port):</span><br><span class="line"> cmd = 'fab -f /root/deploy/pyCapistran.py deploy:repo_url={}, port={}.format(repo_url, c_port)</span><br><span class="line"> container。exec_run(cmd, stream=True)</span><br><span class="line"></span><br><span class="line">if __name__ == '__main__':</span><br><span class="line"> name = sys.argv[1]</span><br><span class="line"> repo_url = sys.argv[2]</span><br><span class="line"> client = docker.from_env()</span><br><span class="line"> try:</span><br><span class="line"> container = client.containers.get(name)</span><br><span class="line"> #if have container get port</span><br><span class="line"> port_bind = os.popen("docker port %s" % name).read()</span><br><span class="line"> ports = re.findall(r"\d{4}", port_bind)</span><br><span class="line"> c_port, l_port = ports[0], ports[1]</span><br><span class="line"> except:</span><br><span class="line"> l_port = random.randint(9000, 9999)</span><br><span class="line"> while is_open(l_port):</span><br><span class="line"> l_port = random.randint(9000, 9999)</span><br><span class="line"> c_port = random.randint(9000, 9999)</span><br><span class="line"> container = client.containers.run("base:server", command = "tail -f /dev/null", name=name, ports={c_port:l_port}, </span><br><span class="line"> volumes={'/home/cfour/.pyenv/versions/3.6.0/lib/python3.6/site-packages': {'bind': '/usr/local/lib/python3.5/dist-packages', 'mode': 'rw'},</span><br><span class="line"> '/home/cfour/workspace/deploy': {'bind': '/root/deploy', 'mode': 'rw'}}, detach=True)</span><br><span class="line"> deploy_project(container, repo_url, c_port)</span><br><span class="line"> print "start server port at:%s" % l_port</span><br></pre></td></tr></table></figure></p>
<p>可以看到脚本中接收一个name参数，一个repo_url参数，name代表创建的容器名称，而repo_url则是项目的git地址，在收到参数后，脚本会尝试按照名称获取容器，如果能获取到，则获得当前容器的端口映射。如果不能获取，尝试生成随机端口，创建容器，绑定端口。然后让容器执行一个叫pyCapistrano的脚本。</p>
<h3 id="pyCapistrano-py"><a href="#pyCapistrano-py" class="headerlink" title="pyCapistrano.py"></a>pyCapistrano.py</h3><p>这个部署脚本来源于<a href="http://dgd2010.blog.51cto.com/1539422/1845307" target="_blank" rel="noopener">urey_pp 的BLOG</a>， 我做了一些修改：<br>去掉了原来的一些config，将部署目录直接定位在了deploy下。另外将repo目录通过git archive到releases目录，这样，就形成了按时间存储的版本管理。然后，形成部署的目录结构后，以在creat_docker.py中获取到的随机端口运行项目中的run.sh：<br><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"># /bin/bash</span><br><span class="line">basepath=$(cd `dirname $0`; pwd)</span><br><span class="line">cd $basepath</span><br><span class="line">export PYTHONPATH=/usr/local/lib/python3。5/dist-packages</span><br><span class="line">pip3 install -r requirments.txt</span><br><span class="line">echo "init over!!!!"</span><br><span class="line">ps -ef | grep python | awk '{print $2 }' | xargs -r kill</span><br><span class="line">nohup /usr/bin/python3 manage.py runserver 0.0.0.0:$1 >> nohup.log 2>&1 &</span><br><span class="line">echo "start server"</span><br></pre></td></tr></table></figure></p>
<p>run。sh就是简单的跑了一下django项目。对应的<a href="">pyCapistrano.py</a></p>
<p>看看效果吧：</p>
<img src="/2017/06/08/GitLab_CI_Docker/deploy.png" title="[deploy]">
<img src="/2017/06/08/GitLab_CI_Docker/result.png" title="[result]">
<h2 id="后记"><a href="#后记" class="headerlink" title="后记"></a>后记</h2><p>至此，一个自动的CI/CD pipline已经完成了。当然这不是唯一的方法，gitlab的webhook一样可以做到。但是，本着操心的原则，最终还是选择了这种方式，失败了很多次，还好结果是好的，但是也深深的感觉到了还有更多的事等着完成。包括脚本的完善和重构。当然，这两个脚本可以用来做java的部署，只是需要一些修改，例如:在ci进行build的时候生成jar或者war，然后脚本不再需要通过git clone去下载整个项目，只需要获得war包，并部署到服务器中。后续我会再放出java版本。</p>
</div>
<footer class="post-footer">
<div class="post-eof"></div>
</footer>
</div>
</article>
<article class="post post-type-normal" itemscope itemtype="http://schema.org/Article">
<div class="post-block">
<link itemprop="mainEntityOfPage" href="http://c4x.github.io/2017/05/26/Spring_Transaction/">