Heap Sort

Heap sort is the first of the linearithmic sorts we will study. It is far more complicated than the previous sorts. It has two stages:

Quickly rearrange the elements into a heap structure, in which each element is greater or equal than its children.
Exploit the heap structure to quickly find the largest element and put it at the end, and then recover the heap structure (excluding the one that was just swapped).

Here it is in action:

So what is a heap? It’s really just a special ordering of an array, that can be obtained very quickly. It requires us to pretend that the array is a binary tree, with each element having “children” and a “parent”. However, it’s still just an array—there isn’t really a tree data structure. Element \(0\) is considered the root of the tree. We define the parent-child relationship as follows:

The left child of element \(i\) is element \(2i + 1\).
The right child of element \(i\) is element \(2i + 2\).
The parent of element \(i\) is element \((i-1)/2\).

Here is an array of 12 random elements, displayed as both array and tree:

Note that this tree is complete. That means that it gets filled in ascending order of the base array. (The indices are beneath the tree nodes.) Therefore, the root is added first. After that, the leftmost element of any row must be added first, and only after the previous row is completely filled in. Each subsequent element in a row can only be added when every element to its left is already added. (Other trees we’ll see in the future will not be complete, and won’t obey these rule.)

The first stage in heap sort is to make this tree into a heap. A heap is a complete tree that obeys the “heap property”: each element must be greater than or equal to its children. To do this, we start with the last parent (element \((n-2)/2\) (where \(n\) is the length of the array) and we sink it. Sinking it means comparing it to its greatest child, and swapping it if it’s smaller. Keep sinking the value until it reaches the point where it’s either not smaller than its children, or it’s on the bottom row. Then work backward one element at a time to the root, until you have sinked every parent. Now the tree is a heap. You can see this by clicking on the step button above.

The second stage is to actually sort the array. The root element must be the largest element of the tree, so swap it to the end. It’s now “retired”, and cannot be moved again. It’s no longer considered to be part of the tree. However, the tree is now probably not a heap, because the new root is likely smaller than its children. So you need to sink the new root in order to recover the heap. Once this is done you can swap the root to the end again, retire it, and so on. Keep going until the array is sorted.

Here is heap sort on a larger data set. (Remember, the tree structure doesn’t really exist. It’s just a useful way to think about the array.) Note the two stages: Stage I forms the heap, and Stage II sorts. You can tell when it’s moved to Stage II, because you’ll see it begin to swap large elements to the end and retire them.

Heap sort is absolutely not adaptive, since it destroys any initial order by creating a heap. Watch what happens when we give it this mostly sorted array:

The heapification process also destroys any relations between tied elements. It is not stable at all.

Why is heap sort linearithmic? It comes down to the sink operation. The sink operation is logarithmic: the amount of time it takes is (at worst) proportional to the number of rows of the tree, and that is \(\log n\). Further, the number of sinks we do is linear with respect to \(n\). Stage does \(n/2\) sinks, and Stage II does \(n\) sinks. Multiplying these together, we see that the overall sort is linearithmic. (It turns out that you can prove that Stage I is linear, but the math is a little harder. Stage II is definitely logarithmic, and in a long array Stage II will dominate.)

You might be tempted to do the sink operation recursively. However a loop is better: it takes less memory. When you do this operation recursively, the number of recursive calls is logarithmic, and you need to store local variables for each one. Therefore, the space complexity is \(\O(\log n)\). When doing it with a loop, the space complexity is constant: \(\O(1)\).

Heap sort does suffer from one critical failure, that isn't evident here. On modern machines, data that is near to each other (or at least has been inspected in the near past) is quicker to access. This is because of a phenomenon called caching: recently looked at data is stored in a kind of memory cache, for quick reinspection. Because heap sort compares many elements that are far away from each other, it tends not to do as well on real, modern computers as other linearithmic sorts we will see.

Here's everything so far:

On to merge sort...