Organizational Research By

Surprising Reserch Topic

what is the reason behind processing a sorted array faster than an unsorted array?



asked Sep 13, 2013 in C++ by rajesh
edited Sep 12, 2013
0 votes
35 views



Related Hot Questions

2 Answers

0 votes
 the culprit is this if-statement:
 
if (data[c] >= 128)
    sum += data[c];
Notice that the data is evenly distributed between 0 and 255. When the data is sorted, roughly the first half of the iterations will not enter the if-statement. After that, they will all enter the if-statement.
 
This is very friendly to the branch predictor since the branch consecutively goes the same direction many times. Even a simple saturating counter will correctly predict the branch except for the few iterations after it switches direction.
 
Quick visualization:
 
T = branch taken
N = branch not taken
 
data[] = 0, 1, 2, 3, 4, ... 126, 127, 128, 129, 130, ... 250, 251, 252, ...
branch = N  N  N  N  N  ...   N    N    T    T    T  ...   T    T    T  ...
 
       = NNNNNNNNNNNN ... NNNNNNNTTTTTTTTT ... TTTTTTTTTT  (easy to predict)
However, when the data is completely random, the branch predictor is rendered useless because it can't predict random data. Thus there will probably be around 50% misprediction. (no better than random guessing)
 
data[] = 226, 185, 125, 158, 198, 144, 217, 79, 202, 118,  14, 150, 177, 182, 133, ...
branch =   T,   T,   N,   T,   T,   T,   T,  N,   T,   N,   N,   T,   T,   T,   N  ...
 
       = TTNTTTTNTNNTTTN ...   (completely random - hard to predict)
So what can be done?
 
If the compiler isn't able to optimize the branch into a conditional move, you can try some hacks if you are willing to sacrifice readability for performance.
 
Replace:
 
if (data[c] >= 128)
    sum += data[c];
with:
 
int t = (data[c] - 128) >> 31;
sum += ~t & data[c];
This eliminates the branch and replaces it with some bitwise operations.
 
(Note that this hack is not strictly equivalent to the original if-statement. But in this case, it's valid for all the input values of data[].)
 
Benchmarks: Core i7 920 @ 3.5 GHz
 
C++ - Visual Studio 2010 - x64 Release
 
//  Branch - Random
seconds = 11.777
 
//  Branch - Sorted
seconds = 2.352
 
//  Branchless - Random
seconds = 2.564
 
//  Branchless - Sorted
seconds = 2.587
Java - Netbeans 7.1.1 JDK 7 - x64
 
//  Branch - Random
seconds = 10.93293813
 
//  Branch - Sorted
seconds = 5.643797077
 
//  Branchless - Random
seconds = 3.113581453
 
//  Branchless - Sorted
seconds = 3.186068823
Observations:
 
With the Branch: There is a huge difference between the sorted and unsorted data.
With the Hack: There is no difference between sorted and unsorted data.
In the C++ case, the hack is actually a tad slower than with the branch when the data is sorted.
answered Sep 13, 2013 by rajesh
edited Sep 12, 2013 by rajesh
0 votes
#include
#include
#include
 
int main()
{
    // Generate data
    const unsigned arraySize = 32768;
    int data[arraySize];
 
    for (unsigned c = 0; c < arraySize; ++c)
        data[c] = std::rand() % 256;
 
    // !!! With this, the next loop runs faster
    std::sort(data, data + arraySize);
 
    // Test
    clock_t start = clock();
    long long sum = 0;
 
    for (unsigned i = 0; i < 100000; ++i)
    {
        // Primary loop
        for (unsigned c = 0; c < arraySize; ++c)
        {
            if (data[c] >= 128)
                sum += data[c];
        }
    }
 
    double elapsedTime = static_cast(clock() - start) / CLOCKS_PER_SEC;
 
    std::cout << elapsedTime << std::endl;
    std::cout << "sum = " << sum << std::endl;
}
 
 
Without std::sort(data, data + arraySize);, the code runs in 11.54 seconds.
With the sorted data, the code runs in 1.93 seconds.
Initially I thought this might be just a language or compiler anomaly. So I tried it in Java:
 
 
import java.util.Arrays;
import java.util.Random;
 
public class Main
{
    public static void main(String[] args)
    {
        // Generate data
        int arraySize = 32768;
        int data[] = new int[arraySize];
 
        Random rnd = new Random(0);
        for (int c = 0; c < arraySize; ++c)
            data[c] = rnd.nextInt() % 256;
 
        // !!! With this, the next loop runs faster
        Arrays.sort(data);
 
        // Test
        long start = System.nanoTime();
        long sum = 0;
 
        for (int i = 0; i < 100000; ++i)
        {
            // Primary loop
            for (int c = 0; c < arraySize; ++c)
            {
                if (data[c] >= 128)
                    sum += data[c];
            }
        }
 
        System.out.println((System.nanoTime() - start) / 1000000000.0);
        System.out.println("sum = " + sum);
    }
}

 

answered Sep 13, 2013 by rajesh
edited Sep 12, 2013 by rajesh

...