|
| 1 | +# Quick Sort |
| 2 | + |
| 3 | +In essence, quick sort is an application of `divide and conquer` strategy. There are usually three steps: |
| 4 | + |
| 5 | +Step1. Pick a pivot -- a random element. |
| 6 | +Step2. Partition -- put the elements smaller than pivot to its left and greater ones to its right. |
| 7 | +Step3. Recurse -- apply above steps until the whole sequence is sorted. |
| 8 | + |
| 9 | +## out-in-place implementation |
| 10 | + |
| 11 | +Recursive implementation is easy to understand and code. Python `list comprehension` looks even nicer: |
| 12 | + |
| 13 | +```python |
| 14 | +#!/usr/bin/env python |
| 15 | + |
| 16 | + |
| 17 | +def qsort1(alist): |
| 18 | + print(alist) |
| 19 | + if len(alist) <= 1: |
| 20 | + return alist |
| 21 | + else: |
| 22 | + pivot = alist[0] |
| 23 | + return qsort1([x for x in alist[1:] if x < pivot]) + \ |
| 24 | + [pivot] + \ |
| 25 | + qsort1([x for x in alist[1:] if x >= pivot]) |
| 26 | + |
| 27 | +unsortedArray = [6, 5, 3, 1, 8, 7, 2, 4] |
| 28 | +print(qsort1(unsortedArray)) |
| 29 | +``` |
| 30 | + |
| 31 | +The output: |
| 32 | + |
| 33 | +``` |
| 34 | +[6, 5, 3, 1, 8, 7, 2, 4] |
| 35 | +[5, 3, 1, 2, 4] |
| 36 | +[3, 1, 2, 4] |
| 37 | +[1, 2] |
| 38 | +[] |
| 39 | +[2] |
| 40 | +[4] |
| 41 | +[] |
| 42 | +[8, 7] |
| 43 | +[7] |
| 44 | +[] |
| 45 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 46 | +``` |
| 47 | + |
| 48 | +Despite of its simplicity, above quick sort code is not that 'quick': recursive calls keep creating new arrays which results in high space complexity. So `list comprehension` is not proper for quick sort implementation. |
| 49 | + |
| 50 | +### Complexity |
| 51 | + |
| 52 | +Take a quantized look at how much space it actually cost. |
| 53 | + |
| 54 | +In the best case, the pivot happens to be the **median** value, and quick sort partition divides the sequence almost equally, so the recursions' depth is $$\log n$$ . As to the space complexity of each level (depth), it is worth some discussion. |
| 55 | + |
| 56 | +A common mistake can be: each level contains $$n$$ elements, then the space complexity is surely $$O(n)$$ . The answer is right, while the approach is not. As we know, space complexity is usually measured by memory consumption of a running program. Take above out-in-place implementation as example, **in the best case, each level costs half as much memory as its upper level does** . Sums up to be: |
| 57 | + |
| 58 | + $$\sum _{i=0} ^{} \frac {n}{2^i} = 2n$$ . |
| 59 | + |
| 60 | +For more detail, refer to the picture below as well as above python code. The first level of recursion saves 8 values, the second 4, and so on so forth. |
| 61 | + |
| 62 | +In the worst case, it will take $$i - 1$$ times of swap on level $$i$$. Sums up to be: |
| 63 | + |
| 64 | +$$\sum_{i=0}^n (n-i+1) = O(n^2)$$ |
| 65 | + |
| 66 | + |
| 67 | + |
| 68 | +## in-place implementation |
| 69 | + |
| 70 | +### one index for partition |
| 71 | + |
| 72 | +One in-place implementation of quick sort is to use one index for partition, as the following image illustrates. Take example of `[6, 5, 3, 1, 8, 7, 2, 4]` again, $$l$$ and $$u$$ stand for the lower bound and upper bound of index respectively. $$i$$ traverses and $$m$$ maintains index of partition which varies with $$i$$. $$target$$ is the pivot. |
| 73 | + |
| 74 | + |
| 75 | + |
| 76 | +For each specific value of $$i$$, $$x[i]$$ will take one of the follwing cases: if $$x[i] \geq t$$ , $$i$$ increases and goes on traversing; else if $$x[i] < t$$ , $$x[i]$$ will be swapped to the left part, as statement `swap(x[++m], x[i])` does. Partition is done when `i == u`, and then we apply quick sort to the left and right parts, recursively. Under what circumstance does recursion terminate? Yes, `l >= u`. |
| 77 | + |
| 78 | +### Python |
| 79 | + |
| 80 | +```python |
| 81 | +#!/usr/bin/env python |
| 82 | + |
| 83 | + |
| 84 | +def qsort2(alist, l, u): |
| 85 | + print(alist) |
| 86 | + if l >= u: |
| 87 | + return |
| 88 | + |
| 89 | + m = l |
| 90 | + for i in xrange(l + 1, u + 1): |
| 91 | + if alist[i] < alist[l]: |
| 92 | + m += 1 |
| 93 | + alist[m], alist[i] = alist[i], alist[m] |
| 94 | + # swap between m and l after partition, important! |
| 95 | + alist[m], alist[l] = alist[l], alist[m] |
| 96 | + qsort2(alist, l, m - 1) |
| 97 | + qsort2(alist, m + 1, u) |
| 98 | + |
| 99 | +unsortedArray = [6, 5, 3, 1, 8, 7, 2, 4] |
| 100 | +print(qsort2(unsortedArray, 0, len(unsortedArray) - 1)) |
| 101 | +``` |
| 102 | + |
| 103 | +### Java |
| 104 | + |
| 105 | +```java |
| 106 | +public class Sort { |
| 107 | + public static void main(String[] args) { |
| 108 | + int unsortedArray[] = new int[]{6, 5, 3, 1, 8, 7, 2, 4}; |
| 109 | + quickSort(unsortedArray); |
| 110 | + System.out.println("After sort: "); |
| 111 | + for (int item : unsortedArray) { |
| 112 | + System.out.print(item + " "); |
| 113 | + } |
| 114 | + } |
| 115 | + |
| 116 | + public static void quickSort1(int[] array, int l, int u) { |
| 117 | + for (int item : array) { |
| 118 | + System.out.print(item + " "); |
| 119 | + } |
| 120 | + System.out.println(); |
| 121 | + |
| 122 | + if (l >= u) return; |
| 123 | + int m = l; |
| 124 | + for (int i = l + 1; i <= u; i++) { |
| 125 | + if (array[i] < array[l]) { |
| 126 | + m += 1; |
| 127 | + int temp = array[m]; |
| 128 | + array[m] = array[i]; |
| 129 | + array[i] = temp; |
| 130 | + } |
| 131 | + } |
| 132 | + // swap between array[m] and array[l] |
| 133 | + // put pivot in the mid |
| 134 | + int temp = array[m]; |
| 135 | + array[m] = array[l]; |
| 136 | + array[l] = temp; |
| 137 | + |
| 138 | + quickSort1(array, l, m - 1); |
| 139 | + quickSort1(array, m + 1, u); |
| 140 | + } |
| 141 | + |
| 142 | + public static void quickSort(int[] array) { |
| 143 | + quickSort1(array, 0, array.length - 1); |
| 144 | + } |
| 145 | +} |
| 146 | +``` |
| 147 | + |
| 148 | +The swap of $$x[i]$$ and $$x[m]$$ should not be left out. |
| 149 | + |
| 150 | +The output: |
| 151 | + |
| 152 | +``` |
| 153 | +[6, 5, 3, 1, 8, 7, 2, 4] |
| 154 | +[4, 5, 3, 1, 2, 6, 8, 7] |
| 155 | +[2, 3, 1, 4, 5, 6, 8, 7] |
| 156 | +[1, 2, 3, 4, 5, 6, 8, 7] |
| 157 | +[1, 2, 3, 4, 5, 6, 8, 7] |
| 158 | +[1, 2, 3, 4, 5, 6, 8, 7] |
| 159 | +[1, 2, 3, 4, 5, 6, 8, 7] |
| 160 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 161 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 162 | +``` |
| 163 | + |
| 164 | +### Two-way partitioning |
| 165 | + |
| 166 | +Another implementation is to use two indexes for partition. It speeds up the partition by working two-way simultaneously, both from lower bound toward right and from upper bound toward left, instead of traversing one-way through the sequence. |
| 167 | + |
| 168 | +The gif below shows the complete process on `[6, 5, 3, 1, 8, 7, 2, 4]`. |
| 169 | + |
| 170 | + |
| 171 | + |
| 172 | +1. Take `3` as the pivot. |
| 173 | +2. Let pointer `lo` start with number `6` and pointer `hi` start with number `4`. Keep increasing `lo` until it comes to an element ≥ the pivot, and decreasing `hi` until it comes to an element < the pivot. Then swap these two elements. |
| 174 | +3. Increase `lo` and decrease `hi` (both by 1), and repeat step 2 so that `lo` comes to `5` and `hi` comes to `1`. Swap again. |
| 175 | +4. Increase `lo` and decrease `hi` (both by 1) until they meet (at `3`). The partition for pivot `3` ends. Apply the same operations on the left and right part of pivot `3`. |
| 176 | + |
| 177 | +A more general interpretation: |
| 178 | + |
| 179 | +1. Init $$i$$ and $$j$$ to be at the two ends of given array. |
| 180 | +2. Take the first element as the pivot. |
| 181 | +3. Perform partition, which is a loop with two inner-loops: |
| 182 | + - One that increases $$i$$, until it comes to an element ≥ pivot. |
| 183 | + - The other that decreases $$j$$, until it comes to an element < pivot. |
| 184 | +4. Check whether $$i$$ and $$j$$ meet or overlap. If so, swap the elements. |
| 185 | + |
| 186 | +Think of a sequence whose elements are *all equal*. In such case, each partition will return the middle element, thus recursion will happen $$\log n$$ times. For each level of recursion, it takes $$n$$ times of comparison. The total comparison is $$n \log n$$ then. [^programming_pearls] |
| 187 | + |
| 188 | +### Python |
| 189 | + |
| 190 | +```python |
| 191 | +#!/usr/bin/env python |
| 192 | + |
| 193 | +def qsort3(alist, lower, upper): |
| 194 | + print(alist) |
| 195 | + if lower >= upper: |
| 196 | + return |
| 197 | + |
| 198 | + pivot = alist[lower] |
| 199 | + left, right = lower + 1, upper |
| 200 | + while left <= right: |
| 201 | + while left <= right and alist[left] < pivot: |
| 202 | + left += 1 |
| 203 | + while left <= right and alist[right] >= pivot: |
| 204 | + right -= 1 |
| 205 | + if left > right: |
| 206 | + break |
| 207 | + # swap while left <= right |
| 208 | + alist[left], alist[right] = alist[right], alist[left] |
| 209 | + # swap the smaller with pivot |
| 210 | + alist[lower], alist[right] = alist[right], alist[lower] |
| 211 | + |
| 212 | + qsort3(alist, lower, right - 1) |
| 213 | + qsort3(alist, right + 1, upper) |
| 214 | + |
| 215 | +unsortedArray = [6, 5, 3, 1, 8, 7, 2, 4] |
| 216 | +print(qsort3(unsortedArray, 0, len(unsortedArray) - 1)) |
| 217 | +``` |
| 218 | + |
| 219 | +### Java |
| 220 | + |
| 221 | +```java |
| 222 | +public class Sort { |
| 223 | + public static void main(String[] args) { |
| 224 | + int unsortedArray[] = new int[]{6, 5, 3, 1, 8, 7, 2, 4}; |
| 225 | + quickSort(unsortedArray); |
| 226 | + System.out.println("After sort: "); |
| 227 | + for (int item : unsortedArray) { |
| 228 | + System.out.print(item + " "); |
| 229 | + } |
| 230 | + } |
| 231 | + |
| 232 | + public static void quickSort2(int[] array, int l, int u) { |
| 233 | + for (int item : array) { |
| 234 | + System.out.print(item + " "); |
| 235 | + } |
| 236 | + System.out.println(); |
| 237 | + |
| 238 | + if (l >= u) return; |
| 239 | + int pivot = array[l]; |
| 240 | + int left = l + 1; |
| 241 | + int right = u; |
| 242 | + while (left <= right) { |
| 243 | + while (left <= right && array[left] < pivot) { |
| 244 | + left++; |
| 245 | + } |
| 246 | + while (left <= right && array[right] >= pivot) { |
| 247 | + right--; |
| 248 | + } |
| 249 | + if (left > right) break; |
| 250 | + // swap array[left] with array[right] while left <= right |
| 251 | + int temp = array[left]; |
| 252 | + array[left] = array[right]; |
| 253 | + array[right] = temp; |
| 254 | + } |
| 255 | + /* swap the smaller with pivot */ |
| 256 | + int temp = array[right]; |
| 257 | + array[right] = array[l]; |
| 258 | + array[l] = temp; |
| 259 | + |
| 260 | + quickSort2(array, l, right - 1); |
| 261 | + quickSort2(array, right + 1, u); |
| 262 | + } |
| 263 | + |
| 264 | + public static void quickSort(int[] array) { |
| 265 | + quickSort2(array, 0, array.length - 1); |
| 266 | + } |
| 267 | +} |
| 268 | +``` |
| 269 | + |
| 270 | +The output: |
| 271 | + |
| 272 | +``` |
| 273 | +[6, 5, 3, 1, 8, 7, 2, 4] |
| 274 | +[2, 5, 3, 1, 4, 6, 7, 8] |
| 275 | +[1, 2, 3, 5, 4, 6, 7, 8] |
| 276 | +[1, 2, 3, 5, 4, 6, 7, 8] |
| 277 | +[1, 2, 3, 5, 4, 6, 7, 8] |
| 278 | +[1, 2, 3, 5, 4, 6, 7, 8] |
| 279 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 280 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 281 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 282 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 283 | +[1, 2, 3, 4, 5, 6, 7, 8] |
| 284 | +``` |
| 285 | + |
| 286 | +Having analyzed three implementations of quick sort, we may grasp one key difference between *quick sort* and *merge sort* : |
| 287 | + |
| 288 | +1. Merge sort divides the original array into two sub-arrays, and merges the sorted sub-arrays to form a totally ordered one. In this case, recursion happens before processing(merging) the whole array. |
| 289 | +2. Quick sort divides the original array into two sub-arrays, and then sort them. The whole array is ordered as soon as the sub-arrays get sorted. In this case, recursion happens after processing(partition) the whole array. |
| 290 | + |
| 291 | +Robert Sedgewick's presentation on [quick sort](http://algs4.cs.princeton.edu/23quicksort/) is strongly recommended. |
| 292 | + |
| 293 | +## Reference |
| 294 | + |
| 295 | +- [Quicksort - wikepedia](https://en.wikipedia.org/wiki/Quicksort) |
| 296 | +- [Quicksort | Robert Sedgewick](http://algs4.cs.princeton.edu/23quicksort/) |
| 297 | +- Programming Pearls Column 11 Sorting - gives an in-depth discussion on insertion sort and quick sort |
| 298 | +- [Quicksort Analysis](http://7xojrx.com1.z0.glb.clouddn.com/docs/algorithm-exercise/docs/quicksort_analysis.pdf) |
| 299 | +- [^programming_pearls]: Programming Pearls |
0 commit comments