Like the backtracking algorithm for solving N queens, the BFS solution *also* makes use of a promising function, otherwise we’d be simply doing an iterative enumeration brute force approach to the problem, which while this might lead to a result (eventually) using depth first search, the memory requirements to do such breadth first would be completely unfeasible.

Traditionally, the N queens problem is framed as a constraint satisfaction problem. And viewed from the angle of combinatorial optimization back tracking is a natural solution. When distilled down to its most basic components we find that backtracking is reliant on two techniques to arrive at its solution: a “promising” function to determine if the current node being examined is worth being expanded, and depth first search which is used to traverse the nodes as they are expanded. If were to remove the promising function from the picture, we’d have the brute force solution – it would find the answer but it would take a long time doing it. Still, this is important because we know that the N queens problem at its heart reduces to a depth first search, and if we can find a simple path with depth first search, than we can find a *shortest* path with breadth first search. If we change vour depth first search to a breadth first search and keep the promising function we can speed up our breadth first search in the same way we sped up our depth first search!

I will be using a simple 2d vector to represent the chessboard, though more compact representations are not only possible, but highly recommended in order to help alleviate the aforementioned high memory usage.

typedef vector<vector<int>> ChessBoard;

void init(ChessBoard& board, int n) {

board = vector<vector<int>>(n, vector<int>(n, 0));

}

The promising function is used to determine if the move we are considering taking is worth taking. It checks to see if placing a queen at that position would put her under attack from any queens that have been placed previously. This means we have to check if any queens have already been placed in the same column or row, and if any queens can attack her from the diagonal positions. We can approach this task two ways: row wise or column wise. They proceed identically, except the moves are rotated 90 degrees.

The algorithms and description below assume we are proceeding in row-wise order (top to bottom).

When a queen is placed in a position, if there is another queen in any of the places marked with an ‘X’, than the node is not promising and we can’t place our queen there.

| X | | | | X | | | | X | |

| | X | | | X | | | X | | |

| | | X | | X | | X | | | |

| | | | X | X | X | | | | |

| X | X | X | X | Q | X | X | X | X | X |

| | | | X | X | X | | | | |

| | | X | | X | | X | | | |

| | X | | | X | | | X | | |

| X | | | | X | | | | X | |

| | | | | X | | | | | X |

If you’re looking at the above diagram and thinking that it’s pretty ridiculous if we have to scan what seems to be half of the positions on the board for every move, you can breath a sigh of relief. Thankfully It’s not as bad as it first appears: we don’t actually need to check each spot. We can observe that some spots couldn’t possibly be occupied yet, and as such we don’t need to bother checking those spots. Proceeding row wise we are placing a queen in a fresh row at each step, and so we don’t have to check if any other queens are in our row as that would be impossible. Like wise for checking if any queens have been placed in the same column, we only need to check the column up to the last queen placed. So for checking the row and columns, out of these positions:

| | | | | X | | | | | |

| | | | | X | | | | | |

| | | | | X | | | | | |

| | | | | X | | | | | |

| X | X | X | X | Q | X | X | X | X | X |

| | | | | X | | | | | |

| | | | | X | | | | | |

| | | | | X | | | | | |

| | | | | X | | | | | |

| | | | | X | | | | | |

The only positions we *actually* need to check are these positions:

| | | | | X | | | |

| | | | | X | | | |

| | | | | X | | | |

| | | | | X | | | |

| | | | | Q | | | |

| | | | | | | | |

| | | | | | | | |

| | | | | | | | |

The same observation holds true for the diagonal checks as well! We only need to check the diagonals that could have been placed so far:

| X | | | | | | | |

| | X | | | | | | X |

| | | X | | | | X | |

| | | | X | | X | | |

| | | | | Q | | | |

| | | | | | | | |

| | | | | | | | |

| | | | | | | | |

This gives us the following O(q) algorithm with q being the number of queens placed thus far to serve as our promising function:

bool promising(ChessBoard& cb, int row, int col) {

//column

for (int k = 0; k < row; k++)

if (cb[k][col] == 1 && k != row)

return false;

//right diagonal

int dx = row-1, dy = col-1, dz = col+1;

while ((dx >= 0 && dy >= 0))

if (cb[dx--][dy--] == 1)

return false;

//left diagonal

dx = row-1;

while (dx >= 0 && dz < cb.size())

if(cb[dx--][dz++] == 1)

return false;

return true;

}

Before placing a queen and pushing the node onto the queue, we check it with the promising function, and if it comes back false, we don’t bother enqueueing the node. In this we avoid filling up the queue with nodes that couldn’t possibly lead to a solution.

With our promising function set up we can start putting our search together. Breadth First Search of a tree uses a FIFO queue to completely process one level of the tree before processing that levels children and so on, which is why when applied to a tree, breadth first search is sometimes called *level order* traversal. For our search tree, every time we place a queen, that is a new node in the tree. If when we pop the next node off of the queue during our traversal, and we have already placed N queens, than that node is a valid solution. Otherwise we try to place a queen in the next row and continue our search.

At every iteration we have to know which queens have already been placed where, and which row we are trying to place a queen on for the current iteration. To ensure this we use the following data structure to represent a Node in our queue. This way we when we pop a Node of the queue, we have that information available to us immediately. This to start the search, we enqueue a blank board, and row 0 for our node, as we are placing the first queen.

typedef pair<int,ChessBoard> Node;

bool solveBFS(ChessBoard& cb, int row) {

bool found = false;

queue<Node> fq;

fq.push(make_pair(row, cb));

We then continue until there are no more nodes to explore, popping a node from the queue, checking which of its child nodes are promising, and enqueueing those that can possibly lead to a valid solution. By only expanding promising nodes we can have confidence that when N queens have been placed, we have a valid solution.

while (!fq.empty()) {

Node cr = fq.front(); fq.pop();

int row = cr.first;

ChessBoard cb = cr.second;

if (row == cb.size()) {

if (validate(cb)) {

printBoard(cb);

soln++;

found = true;

}

} else {

for (int i = 0; i < cb.size(); i++) {

if (promising(cb, row, i)) {

cb[row][i] = 1;

fq.push(make_pair(row+1, cb));

cb[row][i] = 0;

}

}

}

}

return found;

}

It’s actually quite similar to the inner loop of the backtracking solution, and with good reason. Those with enquiring minds may find it interesting to know that we can swap the FIFO queue out for a LIFO stack data structure to arrive back at an iterative version of the backtracking algorithm, albeit one which explores the tree in a slightly different order than the recursive version. Something to keep in mind however, is that when are using the recursive backtracking algorithm, we are making and then undoing changes to one, single instance of the board which is passed around by reference. This is not possible when using Breadth First Search, or even the iterative version of the back tracking algorithm. The reason for this is because we need to enqueue the current state of the board, and if we enqueued references instead of *copies* of the board, we’d have a queue or stack full of references to identical boards!

Solution 92

| | | | | | | | Q |

| | | | Q | | | | |

| Q | | | | | | | |

| | | Q | | | | | |

| | | | | | Q | | |

| | Q | | | | | | |

| | | | | | | Q | |

| | | | | Q | | | |

So, now we know that we can find our solution using breadth first search, and we know why the back tracking version is preferable from a memory usage point of view. But is there something to suggest using BFS instead? Is it any faster at the cost of all that memory? Err… Well… no. Timing both algorithms to find all solutions to the 8 queens problem shows the back tracking algorithm to be about 2.5x faster than breadth first search.*

N-Queens BFS: 19.4765ms, Solutions: 92

N-Queens DFS: 7.38989ms. Solutions: 92

* However, when you compile using gcc’s -O3 level of optimization, BFS speeds up *considerably* while backtracking stays fairly stable. One can speculate that should somebody focus on optimizing the board allocator, BFS could potentially be faster than backtracking.

Backtracking is an adaptation of a preorder depth first search which incorporates a *promising function.* A promising function determines if the branch being explored is worth continuing or not, and depending on the quality of this promising function has the potential to __greatly__ reduce the number of nodes which need to be explored to find a solution. This is called pruning the search space tree, and can lead to significant speed ups in search times compared to a true exhaustive search. For a problem such a the 8-queens problem, this leads to an efficient solution which returns our answer with ease. The knights tour on the other hand, while being played on the same sized board as the N queens problem is noticeably slower to arrive at an answer.

Our “chess board” will be a 2d integer array, with each position initialized to -1, to symbolize that it has not been occupied yet. Depending on where on the board the knight has the potential to move to one of up to 8 places. We store offsets as arrays, so that we can quickly compute possible next moves, though we must be careful to check that a computed next position is *in-bounds*.

#include <iostream>

#include <iomanip>

#include <vector>

#include <queue>

using namespace std;

int coloffset[8] = { 2, 1, -1, -2, -2, -1, 1, 2 };

int rowoffset[8] = { 1, 2, 2, 1, -1, -2, -2, -1 };

typedef vector<vector<int>> ChessBoard;

void init(ChessBoard& board, int n) {

board = vector<vector<int>>(n, vector<int>(n, -1));

}

bool inBounds(ChessBoard& cb, int col, int row) {

return (row >= 0 && row < cb.size() && col >= 0 && col < cb.size());

}

Backtracking algorithms all follow the same general pattern, which as mentioned above is based on a preorder depth first search of our search space. Since our goal is to occupy every square, once we have made 64 successful moves we are done, so that is our base case. Until reaching our goal, each recursive call will place the knight in one square – or none of backtracking.

bool promising(ChessBoard& cb, int col, int row) {

return inBounds(cb, col, row) && (cb[row][col] == -1);

}

bool solveBackTracking(ChessBoard cb, int c, int r, int placed) {

tries++;

if (placed == cb.size() * cb.size()) {

printBoard(cb);

return true;

} else {

for (int i = 0; i < 8; i++) {

int nc = c + coloffset[i];

int nr = r + rowoffset[i];

if (promising(cb, nc, nr)) {

cb[nr][nc] = placed;

if (solveBackTracking(cb, nc, nr, placed+1))

return true;

cb[nr][nc] = -1;

}

}

}

return false;

}

Our promising function is exceptionally simple: we simply check that the move keeps us on the board, and that it has not been occupied yet. Unfortunately, because of the constraints we’re working with there isn’t much more information for us to build a better promising function from. This plays significantly into why backtracking is particularly slow for this problem. The other major part is how big our search tree is: 8 possible next steps, and we need to find 64 of them. It’s still better than a true blind brute force search though, and it does lead to a correct answer so this is what you see as *the* way to solve the knights tour.

max@MaxGorenLaptop:~/algs$ ./kt

Nodes: 8250733

Solution 1

| 0| 37| 58| 35| 42| 47| 56| 51|

| 59| 34| 1| 48| 57| 50| 43| 46|

| 38| 31| 36| 41| 2| 45| 52| 55|

| 33| 60| 39| 26| 49| 54| 3| 44|

| 30| 9| 32| 61| 40| 25| 22| 53|

| 17| 62| 27| 10| 23| 20| 13| 4|

| 8| 29| 18| 15| 6| 11| 24| 21|

| 63| 16| 7| 28| 19| 14| 5| 12|

max@MaxGorenLaptop:~/algs$

Yup you read that right. Eight and one quarter *million* nodes explored before finding a valid solution with backtracking. That’s roughly 130k nodes *per move*. While it certainly isn’t good, it’s not… *catastrophic*. But man, wouldn’t it be so cool to if we could find it on the first try?

max@MaxGorenLaptop:~/algs$ ./kt

Nodes: 64

Solution 1

| 0| 3| 56| 19| 40| 5| 42| 21|

| 33| 18| 1| 4| 57| 20| 39| 6|

| 2| 55| 34| 59| 36| 41| 22| 43|

| 17| 32| 47| 52| 45| 58| 7| 38|

| 48| 13| 54| 35| 60| 37| 44| 23|

| 31| 16| 51| 46| 53| 26| 61| 8|

| 12| 49| 14| 29| 10| 63| 24| 27|

| 15| 30| 11| 50| 25| 28| 9| 62|

max@MaxGorenLaptop:~/algs$

Hah, yeah that would be cool.

What? Why are you looking at me like that?

Warnsdorf’s heuristic is a method for choosing nodes during a knights tour. At each step, instead of simply trying all of the possible next moves sequentially, select the move which brings you to a square from which you have the *least* possible next moves from. In this way we assign a score to each possible next move, and try them in the order of *best* possible next move. We will still use backtracking, but we will transform our search from depth first search to *best first search.* To perform a best first search we will use a min priority queue to pick the next move once we have scored them.

typedef pair<int,int> ScoredMove;

typedef priority_queue<ScoredMove, vector<ScoredMove>, greater<ScoredMove>> _priorityQueue;

class PriorityQueue {

private:

_priorityQueue _pq;

public:

PriorityQueue() { }

bool empty() {

return _pq.empty();

}

void push(int score, int move) {

_pq.push(make_pair(score, move));

}

int pop() {

int move = _pq.top().second;

_pq.pop();

return move;

}

};

To score our moves according to Wanrsdorf’s rule, we’ll still use our old promising function, because we still need to know the information it provides, by “looking ahead” from a promising node we augment the available information giving our initially weak promising function a *significant* boost in “guessing” power.

int countMovesFrom(ChessBoard& cb, int c, int r) {

int valid_moves = 0;

for (int i = 0; i < 8; i++) {

int nc = c + coloffset[i];

int nr = r + rowoffset[i];

if (promising(cb, nc, nr))

valid_moves++;

}

return valid_moves;

}

void scoreNextMoves(ChessBoard& cb, int c, int r, int paced, PriorityQueue& pq) {

for (int i = 0; i < 8; i++) {

int nc = c + coloffset[i];

int nr = r + rowoffset[i];

if (promising(cb, nc, nr)) {

int cm = countMovesFrom(cb, nc, nr);

if (cm > 0 || placed == 63)

pq.push(cm, i);

}

}

}

bool tryMove(ChessBoard& cb, int c, int r, int move, int placed) {

int nc = c + coloffset[move];

int nr = r + rowoffset[move];

cb[nr][nc] = placed;

if (solveBestFirst(cb, nc, nr, placed+1))

return true;

cb[nr][nc] = -1;

return false;

}

bool solveBestFirst(ChessBoard cb, int c, int r, int placed) {

tries++;

if (placed == cb.size() * cb.size()) {

printBoard(cb);

return true;

} else {

PriorityQueue pq;

scoreNextMoves(cb, c, r, placed, pq);

while (!pq.empty()) {

int next = pq.pop();

if (tryMove(cb, c, r, next, placed))

return true;

}

}

return false;

}

Using the above Warndorfs heuristic to perform a best first search resulted in finding *64* solutions faster than the back tracking solution could find one.

All in a day’s work. Until next time, Happy Hacking!

]]>The exact cover problem is a special type of set cover problem. Even if you don’t exactly know what a set cover problem is, you’re probably unknowingly already familiar with them. If you’ve ever played Sudoku, that’s a set cover problem. The 8 queens puzzle is a set cover *and* exact cover problem. And there are many, many other real world examples. Needles to say, set cover problems are important, and efficient algorithms to solve them are equally so. The exact cover problem is known to be NP complete, but there *are* algorithms that solve at least smaller instances of them efficiently, such as algorithm x.

They lay mans explanation of what an exact cover is, if you have a list of lists, and exact cover is the subset of that list that contains every item in the set with no overlap. The formal definition of exact cover is this: Given a set, S which is comprised of subsets of set X, the exact cover is the subset of S, S* whose members collectively contain every member of X, and whos set intersections are mutually 0.

To look at it in another way suppose we have the following:

Set X = {1,2,3,4,5,6,7}

Set S = {A,B,C,D,E,F}

A = {1,4,7}

B = {1,4}

C = {4,5,7}

D = {3,5,6}

E = {2,3,6,7}

F = {2, 7}

Exact Cover of X = {B,D,F}

B = {1, 4, }

D = { 3, 5, 6, }

F = { 2, 7}

X = {1, 2, 3, 4, 5, 6, 7}

The above example is borrowed from the wikipedia page for exact cover problems, and is well worth a read for a more detailed overview, including an in-depth work through of the above example.

The simplest way to model our problem for processing algorithmically is as an incidence matrix. An incidence matrix is a 2d boolean array, with the columns indexed by X and the rows indexed by S. As such, the above data set is transformed into the following:

1,2,3,4,5,6,7

---------------

A: 1,0,0,1,0,0,1

B: 1,0,0,1,0,0,0

C: 0,0,0,1,1,0,1

D: 0,0,1,0,1,1,0

E: 0,1,1,0,0,1,1

F: 0,1,0,0,0,0,1

It is this representation that the description of algorithm x is described as working on. Knuth describes algorithm X as follows:

Algorithm X(matrix A, set partial_solution):

If matrix A has no columns (is empty):

the partial_solution is a valid solution;

terminate successfully.

Else:

step 1) choose a column c (deterministically).

step 2) Choose a row r such that A[r][c] = 1 (nondeterministically).

step 3) Include row r in partial_solution.

step 4)

For each column j such that A[r][j] = 1,

for each row i such that A[i][j] = 1,

delete row i from matrix A.

delete column j from matrix A.

step 5) Repeat this algorithm recursively on the reduced matrix A.

All right, seems pretty straight forward… but what’s this stuff about selecting columns deterministically and rows non deterministically? Is there any pseudocode for that? Well, not exactly, but Knuth is a nice guy and he gave us some suggestions. First, he recommends when selecting a column, to chose the *first* column with the lowest number of 1’s from the matrix. That means if two or more columns tie, choose the column with the lowest index. Ok, not bad we can do that like this:

Integer FindColumn(matrix A):

Integer min = INF, minCol = 0

for each col in matrix A:

count = occurence of 1's in this column

if (count < min):

min = count, minCol = col;

return minCol

Next up is non deterministically choose a row from the matrix, where A[row][col] == 1. I’ll let you in on a secret. When you read “non deterministically” what they mean is try the first, and if that doesn’t work, try the next one. This gives us two bits of information moving forward: one, we need to find all the rows where A[row][col] == 1, and more importantly: this is the condition that we will be back tracking on. First, lets select the rows to try.

List<Integer> findRows(matrix A, int col):

List<Integer> rowsToTry;

for every row in matrix A:

if (A[row][col] == 1)

add row to rowsToTry

return rowsToTry

With those two helpers sketched out, along with knuths pseudocode we have what we need to start implementing. Lets start with our matrix.

#include <iostream>

#include <vector>

#include <set>

using namespace std;

typedef vector<pair<char, vector<int>>> IncidenceMatrix;

void printMatrix(IncidenceMatrix& p) {

cout<<"\nCurrent Matrix: "<<endl;

for (auto m : p) {

cout<<m.first<<": ";

for (int c : m.second) {

cout<<c<<" ";

}

cout<<endl;

}

cout<<"-----------------\n"<<endl;

}

int main() {

IncidenceMatrix problem = {

{'A',{1,0,0,1,0,0,1}},

{'B',{1,0,0,1,0,0,0}},

{'C',{0,0,0,1,1,0,1}},

{'D',{0,0,1,0,1,1,0}},

{'E',{0,1,1,0,0,1,1}},

{'F',{0,1,0,0,0,0,1}}

};

algX(problem, set<char>());

}

I’ve “tagged’ each row of the matrix with it’s* set* name, because its numerical index will change as we reduce the matrix, but the set it represents *must* stay constant, otherwise all of this is for naught. Next is the algorithms for choosing a column and selecting rows.

int findColumn(IncidenceMatrix& p) {

int min = p.size(), rc = 0;

for (int col = 0; col < p[0].second.size(); col++) {

int ones = 0;

for (int row = 0; row < p.size(); row++) {

if (p[row].second[col] == 1)

ones++;

}

if (ones < min) {

min = ones;

rc = col;

}

}

return rc;

}

vector<int> findRows(IncidenceMatrix& p, int col) {

vector<int> rows;

for (int i = 0; i < p.size(); i++) {

if (p[i].second[col] == 1) {

rows.push_back(i);

}

}

return rows;

}

Allright, now we can start writing the depth first part of the algorithm. We know our base case is when we hit an empty matrix. When that happens, the set of rows present in our partial solution set is our exact cover, and so we use that as our terminating condition.

bool algX(IncidenceMatrix p, set<char> solnSet) {

if (p.empty()) {

cout<<"Exact Cover Set: ";

for (char r : solnSet) {

cout<<r<<" ";

}

cout<<endl;

return true;

}

//continues...

Next, we choose our column, and select our rows. This is when we start our possible backtracking. As try each row selected in turn, we add its label to our partial solution set. With each row we run our matrix reduction as described in the pseudocode, and then call algX on the reduced matrix. If that recursive call does not lead to a solution, we remove the row from the solution set (back track) and try the next row.

int col = findColumn(p);

printMatrix(p);

for (int row : findRows(p, col)) {

solnSet.insert(p[row].first);

char save = p[row].first;

if (!algX(reduceMatrix(p, row) , solnSet)) {

solnSet.erase(save);

}

}

return false;

}

If after trying all of our selected rows we have not found a solution, we return false and exit. But first, lets talk about reducing the matrix. Knuth described the process as this: “The nondeterministic choice of r means that the algorithm essentially clones itself into independent sub-algorithms; each sub-algorithm inherits the current matrix A, but reduces it with respect to a different row r.”[1] Well, we’re not going to parallelize the program, but we can do the same process linearly with a loop.

We know that we will be removing rows and columns, both of which have the potential to be *very* costly operations to do *in-place* on a 2D array structure. Instead, I recommend matrix reduction by re-writing. This also aids in our back tracking, as if the reduced matrix doesnt lead to a solution, its no big deal to throw it away since we still have our original matrix – no need to *un-reduce* the matrix we just reduced such as had we done the reduction in-place. We first want to analyze our matrix, and gather a list of rows, as well as a list of columns that will not be included in the next iteration. Creating the next matrix is then a simple matter of copying the current matrix, checking each row if we should skip it before adding it, and likewise for the columns”

IncidenceMatrix reduceMatrix(IncidenceMatrix& p, int row) {

set<int> rowsToRemove, colsToRemove;

analyzeCurrentMatrix(p, row, rowsToRemove, colsToRemove);

return buildNextMatrix(p, rowsToRemove, colsToRemove);

}

void analyzeCurrentMatrix(IncidenceMatrix& p, int row, set<int>& rowsToRemove, set<int>& colsToRemove) {

for (int col = 0; col < p[row].second.size(); col++) {

if (p[row].second[col] == 1) {

for (int i = 0; i < p.size(); i++) {

if (p[i].second[col] == 1) {

rowsToRemove.insert(i);

}

}

colsToRemove.insert(col);

}

}

}

IncidenceMatrix buildNextMatrix(IncidenceMatrix& p, set<int>& rowsToRemove, set<int>& colsToRemove) {

IncidenceMatrix nextMatrix;

for (int r = 0; r < p.size(); r++) {

if (rowsToRemove.find(r) == rowsToRemove.end()) {

vector<int> nrow;

for (int c = 0; c < p[r].second.size(); c++) {

if (colsToRemove.find(c) == colsToRemove.end()) {

nrow.push_back(p[r].second[c]);

}

}

nextMatrix.push_back(make_pair(p[r].first, nrow));

}

}

return nextMatrix;

}

Once we have it all put together, we can run it on the example from above, by adding diagnostic messages at important points in the algorithm, we get the following annotated output:

max@MaxGorenLaptop:~/dsa$ ./loudx

(1) Selected col: 0

Current Matrix:

A: 1 0 0 1 0 0 1

B: 1 0 0 1 0 0 0

C: 0 0 0 1 1 0 1

D: 0 0 1 0 1 1 0

E: 0 1 1 0 0 1 1

F: 0 1 0 0 0 0 1

(1) Selected row: A

(1) Removing row: A from column 0

(1) Removing row: B from column 0

(1) Removing Column: 0

(1) Removing row: A from column 3

(1) Removing row: B from column 3

(1) Removing row: C from column 3

(1) Removing Column: 3

(1) Removing row: A from column 6

(1) Removing row: C from column 6

(1) Removing row: E from column 6

(1) Removing row: F from column 6

(1) Removing Column: 6

Adding A to partial solution set.

(2) Selected col: 0

Current Matrix:

D: 0 1 1 1

(2) No eligible rows, Backtracking.

Removing: A from partial solution set.

(1) Selected row: B

(1) Removing row: A from column 0

(1) Removing row: B from column 0

(1) Removing Column: 0

(1) Removing row: A from column 3

(1) Removing row: B from column 3

(1) Removing row: C from column 3

(1) Removing Column: 3

Adding B to partial solution set.

(2) Selected col: 2

Current Matrix:

D: 0 1 1 1 0

E: 1 1 0 1 1

F: 1 0 0 0 1

(2) Selected row: D

(2) Removing row: D from column 1

(2) Removing row: E from column 1

(2) Removing Column: 1

(2) Removing row: D from column 2

(2) Removing Column: 2

(2) Removing row: D from column 3

(2) Removing row: E from column 3

(2) Removing Column: 3

Adding D to partial solution set.

(3) Selected col: 0

Current Matrix:

F: 1 1

(3) Selected row: F

(3) Removing row: F from column 0

(3) Removing Column: 0

(3) Removing row: F from column 1

(3) Removing Column: 1

Adding F to partial solution set.

matrix is empty. Exact Cover Found: { B, D, F }

And so there you have it, Algorithm X for the Exact Cover problem, sans the dancing links technique. Set covering, of which exact cover is only one type can be used to solve many types of combinatory problems, as such algorithms and techniques to find them are an important area of research. They’re pretty interesting too. Until next time, Happy Hacking.

[1] Knuth, Donald E,. “Dancing Links”, Stanford University, https://arxiv.org/pdf/cs/0011047

]]>The template method pattern is a software design pattern created for exactly the scenario at hand. By extracting the decision logic to separate, overloadable methods we can create *one* base class, and derive classes from that which do the assigned task, drastically reducing the amount of code needed to be written for each algorithm.

Lets take a look at the main loop of our merge() from my previous post to get a clearer picture of what I’m talking about.

while (lit != lhs.end() && rit != rhs.end()) {

if (*lit < *rit) {

lit++;

} else if (*rit < *lit) {

rit++;

} else {

result.insert(*lit);

lit++;

rit++;

}

}

Inside the loop we have conditional branches for three cases. Lets call them predicate A, predicate B, and predicate C. Predicate A checks whether the left item is less than the right item, predicate B checks if the right item is less than left item, and predicate C checks if the two items are equal. By replacing the code blocks inside of the if statements with procedure calls, we end up with the following refactored code:

while (lit != lhs.end() && rit != rhs.end()) {

if (*lit < *rit) {

predicateA();

} else if (*rit < *lit) {

predicateB();

} else {

predicateC();

}

}

Not only is it more compact, but easier to read as well. Of more importance, is the fine grained control we now have over the merge process by employing template methods. This allows us to very easily implement all four set algorithms using the same merge() algorithm. All we need to do is implement the proper predicate() methods for the task at hand. Because of this, the template method design pattern has the added bonus of greatly reducing the number of areas where bugs can potentially sneak in to our code.

Our base class should contain our templated merge method, along with the methods needed by our predicate implementations to interface with the sets, takeFromLeft() and takeFromRight() which retrieve the current item, and advance the iterator to the “left” and “right’ set respectively. I chose to implement these algorithms as Function Objects, and thus the only public interface is operator().

template <class T, typename comp = std::less<T>>

class SetMerger {

private:

virtual void predicateA(Set<T>& result) = 0;

virtual void predicateB(Set<T>& result) = 0;

virtual void predicateC(Set<T>& result) = 0;

bool leftHasMore() {

return lit != set_a.end();

}

bool rightHasMore() {

return rit != set_b.end();

}

void init(Set<T>& a, Set<T>& b) {

set_a = a; set_b = b;

lit = set_a.begin(); rit = set_b.begin();

}

protected:

T takeFromLeft() {

return *lit++;

}

T takeFromRight() {

return *rit++;

}

void merge(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

init(lhs, rhs);

while (leftHasMore() && rightHasMore()) {

if (comp()(*lit , *rit)) {

predicateA(result);

} else if (comp()(*rit, *lit)) {

predicateB(result);

} else {

predicateC(result);

}

}

while (leftHasMore()) result.insert(takeFromLeft());

while (rightHasMore()) result.insert(takeFromRight());

}

typename Set<T>::iterator lit;

typename Set<T>::iterator rit;

Set<T> set_a;

Set<T> set_b;

};

The SetMerger class is not meant to be instantized, as evidences both by the lack of constructor nor any public methods. In addition, the predicate methods are declared as purely virtual. With our base class completed all we need to do to implement our set algorithms is derive a class from the SetMerger class and implement the appropriate predicate methods.

Our classes are being implemented as function objects. As such, the only public interface is the constructor, which simply calls our merge method on the given arguments. set intersection, which only adds items to the result set if they are in both sets (predicate C) is now implemented as follows:

template <class T>

class Intersection : public SetMerger<T> {

private:

void predicateA(Set<T>& result) {

this->takeFromLeft();

}

void predicateB(Set<T>& result) {

this->takeFromRight();

}

void predicateC(Set<T>& result) {

result.insert(this->takeFromLeft());

this->takeFromRight();

}

public:

Intersection(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

this->merge(lhs, rhs, result);

}

};

Had we chosen a concrete implementation for all of the algorithms every other part of the code would have been duplicated. Instead, we only implement those parts which are different. With that in mind, implementing the remaining 3 set algorithms proceeds the same as it did for Intersection.

template <class T>

class Difference : public SetMerger<T> {

private:

void predicateA(Set<T>& result) {

result.insert(this->takeFromLeft());

}

void predicateB(Set<T>& result) {

this->takeFromRight();

}

void predicateC(Set<T>& result) {

this->takeFromLeft();

this->takeFromRight();

}

public:

Difference(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

this->merge(lhs, rhs, result);

}

};

template <class T>

class SymmetricDifference : public SetMerger<T> {

private:

void predicateA(Set<T>& result) {

result.insert(this->takeFromLeft());

}

void predicateB(Set<T>& result) {

result.insert(this->takeFromRight());

}

void predicateC(Set<T>& result) {

this->takeFromLeft();

this->takeFromRight();

}

public:

SymmetricDifference(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

this->merge(lhs, rhs, result);

}

};

template <class T>

class Union : public SetMerger<T> {

private:

void predicateA(Set<T>& result) {

result.insert(this->takeFromRight());

}

void predicateB(Set<T>& result) {

result.insert(this->takeFromLeft());

}

void predicateC(Set<T>& result) {

result.insert(this->takeFromLeft());

this->takeFromRight();

}

public:

Union(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

this->merge(lhs, rhs, result);

}

};

So remember kids: Don’t Repeat Yourself! Until next time, Happy Hacking!

]]>The single most important property of sets is that __the keys must be unique__. This property should be enforced by whatever underlying data structure is being used to implement the set. The three most common data structures used are linked lists, hash tables, and balanced search trees. Which data structure is most appropriate to use is a case-by-case consideration that shouldn’t be taken lightly, as it can have a significant impact on performance.

In todays post I’m going to discuss algorithms for performing the following operations or ordered sets:

- union, which produces a set from combining two sets.
- difference – If comparing set A and set B, the set difference is the set containing the elements present in Set B which are
__not__in Set A. - symmetric difference – similar to difference, but will produce a set containing the elements which are present in B but not in A, as well as the elements that are present in A that are not present in B.
- intersection – produces a set of elements that are present in both sets.

In the above descriptions I say “both” and will be showing examples using 2 sets, but these algorithms can be (and frequently are) scaled up to work with multiple sets at a time.

For the sake of simplicity, I will be using linked list, kept in sorted order as the underlying data structure. I will also be using dummy head and tail nodes, as it allows for a cleaner implementation allowing us to dereference pointers without having to worry about checking for null pointers.

The Set ADT __must__ support the following operations at the minimum:

- insert(T item) – add an item to the set, only if the item is not already present
- remove(T item) – delete an item from the set
- find(T item) – check if an item is in the set, and retrieve it if so.
- size() – return the number of elements in the set

The above list is the bare minimum required for a proper set ADT. Many implementations offer other operations, such as a check for emptiness, range searches, and swapping, just to name a few.

template <typename T>

struct listnode {

T info;

listnode* next;

listnode(T i = T(), listnode* n = nullptr) : info(i), next(n) { }

listnode(T i) : info(i), next(nullptr) { }

};

template <class T>

class Set {

private:

using node = listnode<T>;

using link = node*;

using iterator = ForwardIterator<T>;

link head, z;

int count;

void init() {

head = new node(std::numeric_limits<T>::min());

z = new node(std::numeric_limits<T>::max());

z->next = z;

head->next = z;

count = 0;

}

public:

Set() {

init();

}

Set(const Set& set) {

init();

link c = head;

for (link it = set.head->next; it != set.z; it = it->next) {

c->next = new node(it->info, z);

c = c->next;

count++;

}

}

~Set() {

while (head != z) {

link x = head;

head = head->next;

delete x;

}

}

bool empty() {

return head == z;

}

int size() {

return count;

}

void insert(T key) {

link t = new node(key);

link x = head, p = x;

while (x != z && key > x->info) {

p = x;

x = x->next;

}

if (key != x->info) {

t->next = p->next;

p->next = t;

count++;

}

}

iterator find(T key) {

for (link it = head->next; it != z; it = it->next) {

if (key == it->info) {

return iterator(it);

}

}

return end();

}

iterator begin() {

return iterator(head->next);

}

iterator end() {

return iterator(z);

}

Set operator=(const Set& set) {

if (this == &set)

return *this;

init();

link c = head;

for (link it = set.head->next; it != set.z; it = it->next) {

c->next = new node(it->info, z);

c = c->next;

count++;

}

return *this;

}

};

Anyone who reads my posts knows I’m a fan of the iterator pattern. Set algorithm’s are a place where the iterator pattern proves its merit once again, if the underlying data structure is changed to say, a binary search tree, so long as the BST has iterators implemented the algorithms shown will still work. I use iterators which conform to the STL interface so they can be used with C++’s enhanced for loop. As an added bonus, these algorithms also work with std::set.

template <typename T>

class ForwardIterator {

private:

listnode<T>* curr;

public:

ForwardIterator(listnode<T>* pos) {

curr = pos;

}

T& operator*() {

return curr->info;

}

ForwardIterator operator++() {

curr = curr->next;

return *this;

}

ForwardIterator operator++(int) {

ForwardIterator t = *this;

++*this;

return t;

}

bool operator==(const ForwardIterator& it) const {

return curr == it.curr;

}

bool operator!=(const ForwardIterator& it) const {

return !(*this == it);

}

};

Set operations can be found at the heart of many data intensive applications and queries. SQL statements for example, often reduce to set operations – think table joins. This means that these algorithms are often processing not insignificant quantities of data. Clearly, efficient algorithms are required, it is for this reason we implement *ordered* sets, as they allows us to exploit certain properties of ordered data that are unavailable to us if the collection is unsorted.

In an unordered collection, these algorithms reduce to a search problem, and a rather inefficient one at that, regardless of how the data is structured. The reason for this is simple to understand, and is best illustrated through code. Lets begin with the set intersection operation. Set intersection is the act of creating one set from two or more sets, comprised only of the items which are present in all of them. A naiive approach, and what is essentially the *only* approach open to us if our data is unsorted would look something like this:

template <class T>

void set_intersection(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

for (auto it : lhs)

if (rhs.find(it) != rhs.end())

result.insert(it);

for (auto it : rhs)

if (lhs.find(it) != lhs.end())

result.insert(it);

}

All of the algorithms I will discuss – difference, symmetric difference, intersection, and union – would look similar to the above. because the only recourse we have for checking if the particular invariant we want holds, is to first check that it is valid in list a, and then also check whether it is valid in list b. This requires us to completely traverse __both__ sets O(N) times, *for every element in both sets*. Ouch. Thankfully, we’re not totally stuck: by choosing to keep our data sorted we can do better.

The gist of what it is we’re trying to do is this: take two ordered sets, and using the elements from both sets, create one set based on a given predicate. If *that* doesn’t remind you of something, ill try being a bit more obvious: We are taking two sorted sets and – ahem – *merging* – elements from either set into one.

By taking the merge routine from merge sort and tweaking the way items to be included in the result set are selected we can perform all four set operations in linear time. I’m pretty sure it goes without saying that that’s a whole heck of a lot a better than the quadratic complexity of the naive implementation discusses above. Let’s take the same set intersection problem and see how it is accomplished via merging.

template <class T>

void set_intersection(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

auto lit = lhs.begin(), rit = rhs.begin();

while (lit != lhs.end() && rit != rhs.end()) {

if (*lit < *rit) {

lit++;

} else if (*rit < *lit) {

rit++;

} else {

result.insert(*lit);

lit++;

rit++;

}

}

}

Let’s take a moment to dissect what’s going on here and why. Our goal, as stated above is to find the items that only occur in *both* sets – thus we initialize our iterators and walk the sets simultaneously. Because the lists are in sorted order, we know that if the item from the left set is less than the item in the right set, they cant be equal, and thus cant be in both sets. We check the same but in reverse for the right side. On each iteration, only one of the three cases can be true, for the two negative cases, we advance that sides iterator, leaving the other as it was for the next comparison. Eventually, if an item is in *both* sets, they will have to be compared to each other, otherwise we will consume both sets without adding any items to the result set, as none of them are present in both sets.

Set difference is even closer in structure to merge sort, as once we exit the main loop, we must add the remaining unconsumed items (if they exist) to the result set. This time we went to favor the result of the first comparison for adding to the list, for the same reason described above: as the lists are in sorted order, we know that if the item from the left set is less than the item in the right set, they cant be equal, and thus cannot be present in both sets,

template <typename T>

void set_difference(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

auto a = lhs.begin(), b = rhs.begin();

while (a != lhs.end() && b != rhs.end()) {

if (*a < *b) {

result.insert(*a);

a++;

} else if (*b < *a) {

b++;

} else {

a++;

b++;

}

}

while (a != lhs.end()) result.insert(*a++);

while (b != rhs.end()) result.insert(*b++);

}

Symmetric difference is implemented the same as above, but we insert the result of *either* inequality into the result set:

template <typename T>

void symmetric_difference(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

auto a = lhs.begin(), b = rhs.begin();

while (a != lhs.end() && b != rhs.end()) {

if (*a < *b) {

result.insert(*a);

a++;

} else if (*b < *a) {

result.insert(*b);

b++;

} else {

a++;

b++;

}

}

while (a != lhs.end()) result.insert(*a++);

while (b != rhs.end()) result.insert(*b++);

}

And last but not least: set union, which happens also be an unaltered merge() algorithm borrowed directly from merge sort:

template <typename T>

void set_union(Set<T>& lhs, Set<T>& rhs, Set<T>& result) {

auto a = lhs.begin(), b = rhs.begin();

while (a != lhs.end() && b != rhs.end()) {

if (*b > *a) {

result.insert(*a);

a++;

} else {

result.insert(*b);

b++;

}

}

while (a != lhs.end()) result.insert(*a++);

while (b != rhs.end()) result.insert(*b++);

}

That’s all I’ve got for you today, but stay tuned for an upcoming post on how we can exploit the shared structure of the above algorithms, combined with C++’s template system to create a *framework* for set algorithms, without the obvious repetition of implementing the above algorithms. Until then, Happy Hacking!

AVL Tree’s are the oldest self balancing binary search tree, and the main contender to the Red/Black Tree. Like Red/Black trees, AVL trees maintain balance by performing rotations. The decision of when to rotate and which nodes is made by obtaining the *balance factor* of a given node, and if it exceeds acceptable tolerances, we rotate. For a more detailed discussion of implementing AVL Trees, check my earlier posts on the subject.

There are two properties which we need to check to determine whether a tree is an AVL tree or not: The BST property, and of course, the AVL property. This is done via tree traversal and takes O(N) time. I use recursive preorder traversal for simplicity, but you could also use breadth first traversal, or in order traversal, or which ever tree traversal you like so long as you can compare a node to its two children.

The binary search tree property holds that at any given node, the key at the root is greater than all of the keys in its right subtree. Every AVL tree is a binary search tree, and so if the tree being examined is not a binary search tree, it can’t be an AVL tree either. Determining whether a given tree is a binary search tree is a straightforward affair, keeping the rules for a BST in mind, the following recursive procedure practically writes its self:

bool validateBST(Node* node) {

if (node == nullptr)

return true; //an empty tree is a valid tree.

if (node->left != nullptr && node->key < node->left->key)

return false; //In a BST, the left key should be less then the root key.

if (node->right != nullptr && node->key > node->right->key)

return false; //In a BST, the right key should be greater than the root key.

//continue checking the rest of the tree

return validateBST(node->left) && validateBST(node->right);

}

With that part handled, we can now focus on the real reason we’re testing our tree.

The AVL property holds that at any given node, the difference in height between its left and right subtree can be no greater than one. This is done using the same checks used during insertion to determine when to rotate, except instead of rotating, we mark the tree as invalid, as a balanced AVL tree should require no rotations. It should also be noted that we don’t need to bother testing the double rotation cases, as they would still fail for the single rotation test. Thus, testing if a tree is a valid AVL tree is as simple as checking is the balance factor is greater than 1 or less then -1:

bool validateAVL(Node* node) {

if (node == nullptr)

return true; //An empty tree is balanced.

if (balfactor(node) > 1 || balfactor(node) < -1) {

return false;

}

return validateAVL(node->left) && validateAVL(node->right);

}

And of course, there is no need to traverse the tree twice to validate it, as with red black trees we combine the routines so the checks are done simultaneously:

bool validateAVLBST(Node* node) {

if (node == nullptr)

return true;

if (node->left != nullptr && node->key < node->left->key)

return false;

if (node->right != nullptr && node->key > node->right->key)

return false;

if (balfactor(node) > 1 || balfactor(node) < -1)

return false;

return validateAVLBST(node->left) && validateAVLBST(node->right);

}

I wasn’t kidding when I said it was a straight forward affair!

]]>In a previous article I covered bottom up insertion in B+ tree’s. While obviously a member of the family of B Trees, they are a quite different data structure with different properties and use cases than the “classical” B Tree I will cover in todays post. Unlike B+ trees whose upper levels serve solely as an index with data held only in their leaf nodes, traditional B Trees store data in both leaf and internal nodes. This is actually more desirable for an in-memory structure as keys in the tree are uniquely paired to their in-node associated value, saving overall space. Red/Black Trees, a data structure I have covered in detail, is its self an abstraction of “classical” B Trees implemented as binary trees.

The nodes of a B Tree must have room to store M child pointers, and M-1 Key/Value Pairs. I use a KVPair structure to hold they key/value pairs, but you can also use parallel arrays. The key/value pairs are stored in sorted order which works to maintain the overall search tree property. The child pointers point to nodes whos keys fall in the range between successive keys. One advantage to using the KVPair class, is that when combined with a sentinel dummy node it provides us with a means for gracefully returning from failed search operations in a manner similar to the iterator pattern.

template <class K, class V>

class KVPair;

template <class K, class V>

struct bnode {

KVPair<K,V> data[M-1];

bnode* next[M];

int n;

bool isleaf;

bnode(bool leaf) {

isleaf = leaf;

for (int i = 0; i < M; i++)

next[i] = nullptr;

}

};

template <class K, class V>

class KVPair {

private:

K _key;

V _value;

bool _empty;

public:

KVPair(K k, V v) {

_key = k;

_value = v;

_empty = false;

}

KVPair() { _empty = true; }

void clear() {

_empty = true;

_key = K();

_value = V();

}

bool operator==(const KVPair& o) const {

if (_empty && o._empty)

return true;

return _key == o._key;

}

bool operator!=(const KVPair& o) const {

return !(*this==o);

}

const K& key() const { return _key; }

V& value() { return _value; }

bool empty() { return _empty; }

void setValue(V val) {

_value = val;

}

};

Another design choice to consider is how to implement the nodes: plain old structs, or classes. In my post on B+ trees I discussed the utility of the “page” abstraction, where each node is essentially a self contained sorted array based symbol table with the B+ tree class chaining them together into a tree structure. It really boils down to whether you want to take a more procedural vs OOP way of implementing the tree and it’s associated structures, and in this case I have opted for the more procedural style. The reason for this is the split operation. In traditional B trees the split method is more complicated than for B+ tree’s, and I found it more organizationally consistent to keep *all* of the algorithmic logic in one class.

B Trees – the entire family of B Trees – are based on a deceptively simple concept which can be viewed from several different levels of abstraction. At the heart of all of it are two fundamental properties: the sorted array, and the search tree property. B Tree’s are balanced m-way search tree’s, who’s nodes contain m-1 sorted arrays. They maintain perfect balance, having all of their leaf nodes on the same level while growing *laterally and* *upwards*. This is accomplished by a carefully choreographed series of node splits and key promotions. Based on how and when one chooses to perform these node splits leads to the various different B Tree variants.

template <class K, class V>

class BTree {

private:

using link = bnode<K,V>*;

link root;

int count;

int levels;

KVPair<K,V> nullInfo;

void distributeKeys(link child, link nn, int index);

void addNodeToParent(link node, link child, int index);

void addKVPairToParent(link node, KVPair<K,V> data, int index);

void split(link node, int index);

void traverse(link node);

link search(link node, K key);

void addKVPair(link node, K key, V value);

void insert(link x, K key, V value);

public:

BTree();

void insert(K key, V value);

bool contains(K key);

KVPair<K,V>& get(K key);

int height();

void show();

};

To introduce the organizational structure of B Trees, I’ll start with traversal and search. As I previously mentioned, an important property of B Trees is that they maintain the search tree property, which posits that an in-order traversal of the tree should yield the key’s in sorted order. An In order traversal is performed by first recursively traversing a nodes child at index *i,* before printing the key at index *i.* Once we have traversed all of the keys in a node, we traverse down the index at size(node.keys) + 1. This is because of our nodes containing M-1 keys and M children.

void traverse(link node) {

if (node != nullptr) {

int i = 0;

while (i < node->n) {

if (node->next[i] != nullptr)

traverse(node->next[i]);

cout<<node->data[i].key()<<" ";

i++;

}

if (node->next[i] != nullptr)

traverse(node->next[i]);

}

}

void show() {

traverse(root);

cout<<endl;

}

Since the keys are in sorted order we can use binary search to quickly guide us to the correct node when searching. Depending on the degree of your nodes, linear search can also be used and there’s some contention as to which my truly be faster on modern architecture. Considering B Tree’s should be sized to keep an entire node in a single cache line, binary search should still be faster. Having implemented them both ways I didn’t notice much of a difference in performance. What I did fine interesting was that you end up with differently shaped tree’s depending on which algorithm you use, as they can select slightly different but still valid partitioning keys.

int findIndex(link node, K key) {

int l = 0, r = node->n-1;

while (l <= r) {

int m = (l+r)/2;

if (key < node->data[m].key()) r = m - 1;

else if (key > node->data[m].key()) l = m + 1;

else return m;

}

return l;

}

KVPair<K,V>& search(link node, K key) {

if (node == nullptr)

return nullInfo;

int index = findIndex(node, key);

if (key == node->data[index].key())

return node->data[index];

return search(node->next[index], key);

}

KVPair<K,V>& get(K key) {

return search(root, key);

}

Like most search trees, B Tree’s can be constructed either top down or bottom up. We will be implementing an iterative top down insertion algorithm, for the reasons outlined above. I chose to user iteration solely for convenience. Thanks to their normally large branching factor and perfect balance, even truly HUGE B Trees are still fairly short, and thus one can implement the algorithm using recursion without worrying about overflowing the stack.

We will be using a technique called “pre-emptive splitting” where we check the size of the next node as we traverse down the tree towards the insertion location. If the node is full we immediately split the node before even traversing to it. This ensures that every node we move to will have room for an insertion, without the need to do any balancing on the way back up. The only special case is when the root node needs splitting, as this is also the *only* time the tree grows in __height__.

void add(K key, V value) {

if (root->n == M-1) {

link t = root;

root = new bnode<K,V>(false);

root->next[0] = t;

split(root, 0);

levels++;

}

insert(root, key, value);

}

After checking the root and handling it’s splitting if required, we search down the tree to find the appropriate leaf with which to insert our data. If during the search we encounter a node with the same key we are inserting we simply update its associated value and exit. B Tree’s *can* handle multiple identical keys just fine, but you will need a more complicated search procedure. If the next node we need to go to is full, we split the node, and then adjust our index pointer incase the node to insert to has changed to the new node created during the split. Upon reaching a leaf node, we insert our KVPair with insertion sort.

void addKVPair(link node, K key, V value) {

int j = node->n;

while (j > 0 && node->data[j-1].key() > key) {

node->data[j] = node->data[j - 1];

j--;

}

node->data[j] = KVPair<K,V>(key, value);

node->n++;

}

void insert(link x, K key, V value) {

link node = x;

while (!isLeaf(node)) {

int i = findIndex(node, key);

if (key == node->data[i].key()) {

node->data[i].setValue(value);

return;

}

if (isFull(node->next[i])) {

split(node, i);

if (node->data[i].key() < key)

i++;

}

node = node->next[i];

}

addKVPair(node, key, value);

count++;

}

The splitting routines are really the beating heart of the B Tree. Depending on how and when one chooses to split nodes, different variants of the B Tree result, and as it turns out the “traditional” B Tree split routine is actually much more involved than the one used for B+ trees. The reason for this is because the upper levels of B Tree’s serve as more than just an index, and thus it is unique Key Value pairs that get promoted to the parent, and subsequently removed from the child. This second step is not necessary in B+ trees, where the upper levels are only used as an index to the correct leaf node where ALL data is stored.

void split(link node, int index) {

link child = node->next[index];

link nn = new bnode<K,V>(child->isleaf);

distributeKeys(child, nn, index);

addNodeToParent(node, nn, index);

addKVPairToParent(node, child->data[(M/2)-1], index);

}

The first step in splitting a node is to create the new node, and distribute the KVPairs and child pointers between them, save for the pair that will be promoted. After that, the newly created node is inserted into the parent of the node it was split from. Finally, the KVPair used as the pivot for partitioning the node is also inserted into the parent node.

void distributeKeys(link child, link nn, int index) {

for (int j = 0; j < M/2 - 1; j++)

nn->data[j] = child->data[j + (M/2)];

if (!child->isleaf) {

for (int j = 0; j < M/2; j++) {

nn->next[j] = child->next[j+(M/2)];

}

}

nn->n = (M/2) - 1;

child->n = (M/2) - 1;

}

void addNodeToParent(link node, link child, int index) {

for (int j = node->n+1; j > index; j--)

node->next[j] = node->next[j-1];

node->next[index+1] = child;

}

void addKVPairToParent(link node, KVPair<K,V> kvpair, int index) {

for (int j = node->n; j > index; j--)

node->data[j] = node->data[j-1];

node->data[index] = kvpair;

node->n++;

}

We now have the necessary algorithms for inserting and retrieving items from an in memory B Tree. To test the B Tree, I used the sample word files available from here and here to calculate the frequencies of occurrences of words in the text files.

void btree_test(string filename) {

BTree<string,int> st;

ifstream in_file(filename);

if (!in_file.is_open()) {

cout<<"Couldn't open "<<filename<<" for reading."<<endl;

return;

}

string input;

auto timestamp1 = std::chrono::steady_clock::now();

while (!in_file.eof()) {

in_file>>input;

auto t = st.get(input);

if (t.empty()) {

st.add(input, 1);

} else {

st.get(input).setValue(t.value() + 1);

}

}

auto timestamp2 = std::chrono::steady_clock::now();

auto timing = std::chrono::duration<double, std::milli>(timestamp2 - timestamp1);

in_file.close();

cout<<"B Tree"<<endl;

cout<<"Items: "<<st.size()<<endl;

cout<<"Height: "<<st.height()<<endl;

cout<<"key: government, value: "<<st.get("government").value()<<endl;

cout<<"key: business, value: "<<st.get("business").value()<<endl;

cout<<"Built: "<<timing.count()<<"ms."<<endl;

}

int main(int argc, char** argv) {

btree_test(argv[1]);

return 0;

}

max@MaxGorenLaptop:~/dsa/btree$ ./tbt words10k.txt

B Tree

Items: 10679

Height: 4

key: government, value: 7

key: business, value: 122

Built: 70.1517ms.

max@MaxGorenLaptop:~/dsa/btree$

max@MaxGorenLaptop:~/dsa/btree$ ./tbt words100k.txt

B Tree

Items: 144256

Height: 5

key: government, value: 2549

key: business, value: 1018

Built: 1878.68ms.

max@MaxGorenLaptop:~/dsa/btree$

And that’s it! It *is* simple, but one must also be careful with promotion of KVPairs to the parent, as it can be a bit tricky to select the correct pivoting key. For example, the above tree actually only works with values of M that are even. The algorithm is unsuitable for implementing a 2-3 tree – the smallest occuring valid B Tree. It *can* (and does) implement a 2-3-4 tree however, which can be directly compared to a RedBlack Tree to see if the tree is working as expected (it does). Anyway, that’s all I’ve got for today, so until next time, happy hacking!

//If this cant be done as an O(1) operation, then the algorithm

//wont run in O(nlogn) time.

template <class T>

void appendFirst(SortableList<T>& from, SortableList<T>& too) {

int first = from.getFirst();

from.pop();

too.append(first);

}

template <class T>

void merge(SortableList<T>& front, SortableList<T>& back, SortableList<T>& result) {

while (!front.empty() && !back.empty()) {

if (front.getFirst() < back.getFirst()) {

appendFirst(front, result);

} else {

appendFirst(back, result);

}

}

while (!front.empty()) appendFirst(front, result);

while (!back.empty()) appendFirst(back, result);

}

void mergesort(SortableList<T>& sq) {

if (sq.size() <= 1)

return;

LinkedList<int> front, back;

int e = sq.size()/2;

while (sq.size() > e) appendFirst(sq, front);

while (!sq.empty()) appendFirst(sq, back);

mergesort(front);

mergesort(back);

merge(front, back, sq);

}

The answer is adaptability. The above algorithm will sort *any* container which implements the methods defined in the following SortableList interface:

template <class T>

class SortableList {

public:

virtual bool empty() = 0;

virtual int size() = 0;

virtual void append(T info) = 0;

virtual void pop() = 0;

virtual T getFirst() = 0;

};

Now it doesn’t matter if you’re underlying structure is an array, or a linked list, the same implementation of mergesort can sort either one.

template <class T>

class ArrayList : public SortableList<T> {

private:

T *a;

int max;

int front;

int back;

public:

ArrayList() {

front = 0;

back = 0;

max = 512;

a = new T[max];

}

void append(T info) {

a[back++] = info;

if (back > max) back = 0;

}

void pop() {

front++;

if (front > max) front = 0;

}

T getFirst() {

return a[front];

}

int size() {

return back - front;

}

bool empty() {

return size() == 0;

}

void print() {

for (int i = front; i < back; i++)

cout<<a[i]<<" ";

cout<<endl;

}

};

template <class T>

class LinkedList : public SortableList<T> {

private:

struct node {

T info;

node* next;

node(T i, node* n) {

info = i;

next = n;

}

};

node* head;

node* tail;

int n;

public:

LinkedList() {

head = nullptr;

tail = nullptr;

n = 0;

}

int size() {

return n;

}

bool empty() {

return head == nullptr;

}

void push(T info) {

head = new node(info, head);

n++;

}

void append(T info) {

node* t = new node(info, nullptr);

if (empty()) {

head = t;

} else {

tail->next = t;

}

tail = t;

n++;

}

void pop() {

node* t = head;

head = head->next;

t->next = nullptr;

n--;

delete t;

}

T getFirst() {

return head->info;

}

void print() {

for (auto t = head; t != nullptr; t = t->next)

cout<<t->info<<" ";

cout<<endl;

}

};

This is what makes OOP and generic programming such a wonderful combination. It makes following the “Don’t repeat yourself” advice a breeze!

int main() {]]>

LinkedList<int> ll;

ArrayList<int> al;

for (int i = 0; i < 10; i++) {

ll.push(rand() % 99);

al.append(rand() % 99);

}

cout<<"Linked List: "<<endl;

ll.print();

mergesort(ll);

ll.print();

cout<<"Array List: "<<endl;

al.print();

mergesort(al);

al.print();

return 0;

}

max@MaxGorenLaptop:/mnt/c/Users/mgoren$ ./mergesort

Linked List:

79 42 95 5 41 69 55 23 72 28

5 23 28 41 42 55 69 72 79 95

Array List:

43 79 70 39 1 40 25 4 54 55

1 4 25 39 40 43 54 55 70 79

max@MaxGorenLaptop:/mnt/c/Users/mgoren$

There has been a truly astonishing amount of research performed over the years in order to determine the optimal pivot selection method, and while the jury is still out on the absolute best, practical application of the various *single pivot* methods for quicksort has generally agreed on three “generally good choices” – four if you count “just picking one” (which actually tends to work pretty well, until it doesn’t). The three most popular methods of choosing a pivot for quicksort are:

- Using a Randomized Pivot – This works by choosing a random value in the bounds of the array being sorted and use that as the pivot value.
- Choosing the Median of Three – The middle value is chosen from three values taken from the beginning, middle, and end of the array slice being sorted.
- Choosing the ‘Ninther’ – The median of the result of 3 Median of 3’s, this is only applicable to larger arrays, but works on the same principle as Median of Three

While the randomized algorithm does work well in practice, the algorithm is at the mercy of the random number generator. Because of this non-deterministic approach, it is unsuitable for some applications. This turns out to not be much of a big deal because Median of three works just as well, without resorting to randomness.

The median of three method is very popular, both for its ease of implementation and its simplicity. An array, and its bounds are supplied as input, and a middle value is calculate from the supplied bounds. The low bound, high bound and middle value are then sorted to obtain the median. In most implementations the median value is also placed into the pivot position, though I have left the three values sorted. The algorithm is then easily implemented as described:

int medianOf3(int a[], int l, int r) {

int m = (l+r)/2;

if (a[l] > a[m])

exch(a, l, m);

if (a[l] > a[r])

exch(a, l, r);

if (a[m] > a[r])

exch(a, m, r);

return m;

}

The implementation of introspective sort supplied in Go’s standard library implemented the median of three algorithm for it’s pivot selection, as well as many other library quicksort implementations.

The ninether method is recursive application of median of 3, where the supplied bounds are divided into three sections, and the median of each section is compared to obtain the median of 9. In this method it is important to leave the median of 3 result in its sorted order and not to perform the swap of the median into the pivot position until you have exited the procedure. You also must be careful to choose *unique* values: remember that the bounds are inclusive.

Unlike the median of 3 method, this method is recursive. The base case should be obvious: returning the median of three.

int medOf9(int a[], int l, int r) {

if (r - l < 9) {

int m = (l+r)/2;

if (a[l] > a[m])

exch(a, l, m);

if (a[l] > a[r])

exch(a, l, r);

if (a[m] > a[r])

exch(a, m, r);

return m;

}

int width = (r-l)/3;

int lo = medOf9(a, l, l+width-1);

int mid = medOf9(a, l+width, r-width-1);

int end = medOf9(a, r-width, r);

if (a[lo] > a[mid])

exch(a, lo, mid);

if (a[lo] > a[end])

exch(a, lo, end);

if (a[mid] > a[end])

exch(a, mid, end);

return mid;

}

The median of 3 approach leaves the array relatively un changed. The algorithm shown above for median of 9 can end up making *significant* changes to the input array, and as it turns out this is in our favor. If we count the inversions in an array before and after a call to medOf9(), the number goes down – sometimes by quite a bit. Seeing as the number of inversions in an array is a measure of its sorted-ness, anything we can do to reduce the number of inversions speeds up our sort. As an example, the following is a trace of *one* call to medOf9(0, n-1):

int* copyArr(int a[], int n) {

int *copy = new int[n];

for (int i = 0; i < n; i++)

copy[i] = a[i];

return copy;

}

int countInversions(int a[], int aux[], int l, int r) {

if (r - l <= 1)

return 0;

int m = (l+r) / 2;

int fc = countInversions(a, aux, l, m);

int rc = countInversions(a, aux, m, r);

int ic = 0;

for (int k = l; k < r; k++) aux[k] = a[k];

int i = l, j = m, k = l;

while (i < m && j < r) {

if (aux[i] < aux[j]) {

a[k++] = aux[i++];

} else {

a[k++] = aux[j++];

ic = ic + (m - i); // <- inversion detected.

}

}

while (i < m) a[k++] = aux[i++];

while (j < r) a[k++] = aux[j++];

return fc + rc + ic;

}

int inversionCount(int a[], int n) {

int *aux = new int[n];

for (int i = 0; i < n; i++) {

aux[i] = a[i];

}

int ic = countInversions(a, aux, 0, n);

delete [] aux;

return ic;

}

int main() {

int n = 30;

int a[n];

for (int i = 0; i < n; i++) {

a[i] = rand() % 100;

}

print(a, n);

cout<<"Inversions: "<<inversionCount(copyArr(a, n), n)<<endl;

medOf3(a, 0, n-1);

cout<<"Inversions: "<<inversionCount(copyArr(a, n), n)<<endl;

print(a, n);

}

83 86 77 15 93 35 86 92 49 21 62 27 90 59 63 26 40 26 72 36 11 68 67 29 82 30 62 23 67 35

Inversions: 241

Median of 9 for bounds: 0 - 29, slice size: 9

83 86 77 15 93 35 86 92 49 21 62 27 90 59 63 26 40 26 72 36 11 68 67 29 82 30 62 23 67 35

Median of 9 for bounds: 20 - 29, slice size: 3

11 68 67 29 82 30 62 23 67 35

62 < 23 < 35 ->

Swap: 62 <-> 23

Swap: 62 <-> 35

Result: 23 < 35 < 62

Median of 3: 35

11 < 68 < 67 ->

Swap: 68 <-> 67

Result: 11 < 67 < 68

Median of 3: 67

29 < 82 < 30 ->

Swap: 82 <-> 30

Result: 29 < 30 < 82

Median of 3: 30

Returned Medians: 67 < 30 < 35

Result: 30 < 35 < 67

Final Median: 35

83 < 93 < 49 ->

Swap: 83 <-> 49

Swap: 93 <-> 83

Result: 49 < 83 < 93

Median of 3: 83

Median of 9 for bounds: 9 - 19, slice size: 3

21 62 27 90 59 63 26 40 26 72 36

40 < 26 < 36 ->

Swap: 40 <-> 26

Swap: 40 <-> 36

Result: 26 < 36 < 40

Median of 3: 36

21 < 62 < 27 ->

Swap: 62 <-> 27

Result: 21 < 27 < 62

Median of 3: 27

90 < 59 < 26 ->

Swap: 90 <-> 59

Swap: 59 <-> 26

Swap: 90 <-> 59

Result: 26 < 59 < 90

Median of 3: 59

Returned Medians: 27 < 59 < 36

Result: 27 < 36 < 59

Final Median: 36

Returned Medians: 83 < 36 < 35

Result: 35 < 36 < 83

Final Median: 36

49 86 77 15 35 35 86 92 93 21 27 62 26 36 63 90 26 59 72 40 11 30 68 29 83 82 23 67 67 62

Inversions: 227

Anyway, today’s post is a short one, as that’s all I’ve got for today. Until next time, Happy Hacking!

]]>There are a number of famous algorithms that have been developed for computing minimum spanning tree’s. Prim’s algorithm – sometimes called Jarnik’s Algorithm – is one such algorithm that was also re-discovered by Edsgar Dijkstra. That essentially the same algorithm was discovered *at least* 3 separate times speaks to it’s fundamental nature. It is an adaptation of priority first search, and I have covered it in a previous post. In this post I want to discuss what I personally find to be one of the more elegant of the general graph algorithms: Kruskal’s algorithm.

Kruskal’s algorithm is an interesting exercise for computer science students as it leaves many implementation decisions open ended. The outcome of those decisions if not carefully considered can dominate the run time performance of the entire algorithm. The two major design decisions are the choice of sorting method for the graph’s edges, as Kruskal’s algorithm requires the graph’s edges be sorted by edge weight, and the choice of data structure for implementing disjoint sets. In todays post I will go over a particularly elegant implementation that is known to be computationally optimal. The code used in the examples discussed below was also used directly to generate the images of the graph and minimum spanning tree shown above.

There has been *alot* of research done into the analysis of various disjoint set data structures, and in all reality they deserve a post of their own. For this post I’m going to cover the abridged version. Disjoint sets play a central role in the “union/find” algorithm, which is used for all manner of things, of particular relevance to us, is their use for determining *dynamic connectivity.* The central idea is that we maintain a list of sets, usually as a parent pointer tree, but I’ve also seen linked versions. This list of sets begins as a list of singleton sets.

Set membership is determined through the use of the equal, merge, and find operations.

class DisjointSet {

public:

bool equal(int p, int q);

int find(int p);

void merge(int p, int q);

};

There is a *very* simple way to implement this in just a handful of lines of code. There are also a handful of approaches that make one of either union or find in constant or logarithmic time, these are called quickfind and quickunion respectively. Below is an example of the “simple” approach:

bool equal(int p, int q) {

return find(p) == find(q);

}

int find(int p) {

if (p != par[p])

p = find(u[p]);

return p;

}

void merge(int p, int q) {

int proot = find(p);

int qroot = find(q);

par[qroot] = proot;

}

While certainly simple, there is a fairly big problem with this implementation. During the merge operation we blindly attach the tree rooted at q to the tree rooted at p, allowing for the potential of the tree to grow incredibly large. Similar to an unbalanced binary search tree, this effects the performance of the find operation, as it now has to traverse longer pointer chains to find the root of a given tree. There are two other methods of similar performance and complexity. “Union by size, with find by path halving” and “Union by rank with path splitting”. Because they are of equivalent performance and complexity, I will only cover the union by size approach, as it’s a little easier to understand. From here on I will refer to this algorithm as “The Union Find(UF) Algorithm”. The UF algorithm allows to perform all operations in logarithmic time. What’s more, its implementation is not much more complicated than the simple version shown above.

The UF algorithm requires two arrays, one for the parent pointer tree, and another to hold the size of the tree rooted at that index. The parent array is initialized with all entries pointing to themselves as their parent, with an associated size of 1. This is our initial set of singletons.

class DisjointSet {

private:

int *par;

int *size;

int N;

void makeSet(int i) {

par[i] = i;

size[i] = 1;

}

public:

DisjointSet(int numStart = 7) {

numSets = numStart;

N = numSets;

par = new int[N];

size = new int[N];

for (int i = 0; i < N; i++) {

makeSet(i);

}

}

~DisjointSet() {

delete [] par;

delete [] size;

}

bool equal(int p, int q) {

return find(p) == find(q);

}

Our new implementation of find() make’s use of a technique called “path halving”. Path halving works by traversing the tree and replacing the pointer to the current nodes parent, with a pointer to the current nodes *grand*parent. This reduces the height of the tree so that the find() operation will complete in logarithmic time.

int find(int p) {

while (p != par[p]) {

par[p] = par[par[p]];

p = par[p];

}

return p;

}

The refactored merge() implementation is where we make use of the size array. Previously, we had been merging set’s without regard to the result besides being the same set. In this version, we we merge the sets with the intention of keeping the resultant tree as “flat” as possible. Just like with a binary search tree, the flatter the tree means a shorter the path from/to the root to node, resulting in quicker access times.

void merge(int p, int q) {

int rp = find(p);

int rq = find(q);

if (rp != rq) {

if (size[rp] < size[rq])

swap(rp, rq);

par[rq] = rp;

size[rp] += size[rq];

}

}

Now that we have a suitable disjoint set data structure, we can start to tackle Kruskal’s algorithm.

The high level overview of kruskals algorithm is beautiful for its simplicity. To compute the minimum spanning tree, you begin with a list of the graph’s edges, sorted by weight. From this list of edges, you take the edge with the shortest edge weight and check if adding it to the tree would cause a cycle. If adding the edge to the MST forms a cycle, then it is *not* part of the MST and can be removed, otherwise leave the edge in place. You proceed this way until you have the V-1 edges that connect the V vertices of the graph with the lowest overall edge weight.

We’re now ready for the next big design decision: how – and to a lesser extent, *when* – to sort the edges of the graph. No matter whether you have chosen to use an adjacency list or an adjacency matrix as you’re graph representation, we’ve still got some work to do in preparing the edges, as kruskal’s algorithm doesn’t operate directly on either. We need a different kind of graph representation: on *edge list.* An edge list can be stored explicitly alongside you’re graph as useful metadata, or can be generated “on demand” in a straightforward way from either representation as both store the required information implicitly.

For my Graph data structure I used an adjacency list representation, with the nodes of of the adjacency list being comprised of the following WeightedEdge structure:

struct WeightedEdge {

int v;

int u;

int wt;

WeightedEdge* next;

WeightedEdge(int from = 0, int to = 0,int weight = 0) {

v = from;

u = to;

wt = weight;

next = nullptr;

}

bool operator==(const WeightedEdge& we) const {

return (v == we.v && u == we.u) || (v == we.u && u == we.v);

}

bool operator!=(const WeightedEdge& we) const {

return !(*this==we);

}

};

I use this edge structure for the adjacency list because it can also be used unmodified for an edge list. This also simplifies things for undirected graphs by not adding back edges to the edge list – something which is much harder to do *efficiently* when generating the edge list from an adjacency list or matrix. This is stored as an unordered singly linked list so adding an entry can be performed in constant time. The graph structure supplies a public method edges(), which returns an iterator to this list. *For dense graphs this should be avoided, seeing as this problem doesn’t arise when using an adjacency matrix*.

If you’re storing the edge list explicitly, you may be tempted store this list in sorted order. Keep in mind however that this adds an additional O(n) step to every call of addEdge() to the graph – 2 of them, as the graph is undirected. When you consider this would require the algorithm to explore approximately half of the edges in the entire graph – *twice* – on *every insertion,* this quickly proves not to be a wise decision. Instead, merge sort is well suited to sorting the list efficiently if the edges are kept in a linked list. Seeing as our edge structure contains a next pointer for use in adjacency lists, it would be silly not to utilize this pointer for the edge list as well. Alternatively, we could store the edge list in an array and use one of any number of O(nlogn) sorting algorithms such as quicksort.

WeightedEdge* Kruskal::merge(WeightedEdge* a, WeightedEdge* b) {

WeightedEdge d; WeightedEdge* c = &d;

while (a != nullptr && b != nullptr) {

if (b->wt > a->wt) {

c->next = a; a = a->next; c = c->next;

} else {

c->next = b; b = b->next; c = c->next;

}

}

c->next = (a == nullptr) ? b:a;

return d.next;

}

WeightedEdge* Kruskal::mergesort(WeightedEdge* h) {

if (h == nullptr || h->next == nullptr)

return h;

WeightedEdge* fast = h->next->next;

WeightedEdge* slow = h;

while (fast != nullptr && fast->next != nullptr) {

slow = slow->next;

fast = fast->next->next;

}

WeightedEdge* front = h;

WeightedEdge* back = slow->next;

slow->next = nullptr;

return merge(mergesort(front), mergesort(back));

}

WeightedEdge* Kruskal::sortEdges(Graph& G) {

WeightedEdge* toSort = G.edges().get();

return mergesort(toSort);

}

As you can se from the above code, the properties of our graph data structure made it well suited to integrating with merge sort. And many real world implementations of kruskal’s algorithm make use of an explicitly sorted edge list. But here’s the rub: If you remember from the graph at the beginning of this post, even a small graph of 15 vertices had 105 edges – with the potential still for much more. At the same time, the minimum spanning tree of __any__ graph has V – 1 edges. That’s a whole lot of sorted edges that we potentially may not end up even needing to examine. What If I told you there was a way we could efficiently get the edges we need for the MST, deferring much of the sorting work until it is needed? There happens to be a data structure which does this *extraordinarily* well: the min-heap. Instead of explicitly sorting the entire edge list, we will place all of the edges into a priority queue!

struct CompEdge {

bool operator()(WeightedEdge* lhs, WeightedEdge* rhs) {

return lhs->wt > rhs->wt;

}

};

class Kruskal {

private:

priority_queue<WeightedEdge*, vector<WeightedEdge*>, CompEdge> pq;

vector<WeightedEdge*> mstEdges;

void sortEdges(Graph& G) {

for (adjIterator it = G.edges(); !it.end(); it = it.next()) {

pq.push(it.get());

}

}

If you’re still wondering how the disjoint set figures into all of this you can relax, as it is union/find’s time to shine. Now that we have access to the list of edges in sorted order, we can use them to greedily build the minimum spanning tree. We use disjoint sets in order to maintain the set of edges that are in the in the MST, from the sets of edges *not* in the MST in order to determine if adding an edge would form a cycle. This might sound complicated, but its actually the *exact* kind of problem union/find was designed for! If the two vertices that make up the edge being examined are already in the same set, than adding that edge to the MST would form a cycle. Using this knowledge we can easily determine which edges to add to the MST and which edges we can skip. When we find an edge that belongs in the MST we perform a set union on the sets which contain the vertices that make up that edge in addition to adding the edge to the MST.

As you can see from the code below whether using a priority queue or sorted list, the the algorithm emerges rather naturally:

//Using priority queue

vector<WeightedEdge>& compute(Graph& G) {

sortEdges(G);

DisjointSet djs(G.V());

while (!pq.empty()) {

int from = pq.top()->v;

int to = pq.top()->u;

if (!djs.equal(from, to)) {

djs.merge(from, to);

mstEdges.push_back(pq.top());

}

pq.pop();

}

return mstEdges;

}

//using sorted list

vecotr<WeightedEdge>& compute(Graph& G) {

DisjointSet djs(G.V());

for (WeightedEdge* edge = sortEdges(G); edge != nullptr; edge = edge->next) {

if (!djs.equal(edge->v, edge->u)) {

djs.merge(edge->v, edge->u);

mstEdges.push_back(edge);

if (mstEdges.size() == G.V() - 1) {

return mstEdges;

}

}

}

return mstEdges;

}

The above algorithm – both implementations – will compute the minimum spanning tree of a Graph in worst case O(E log E) where E is the number of edges. If you happen to be one of those people who are fascinated by mathematical proofs, Tarjan’s proof of union find using the inverse ackerman function is considered by many to be an excellent example of such.

The other classical algorithm for computing the MST of a graph is Prim’s algorithm. Prim’s will compute the MST in O(E log V), which depending on the density of the graph may be considerably faster than Kruskal’s O(E log E). Like Kruskal’s algorithm, Prim’s algorithm is also a greedy algorithm – though it performs it’s task completely differently. Unlike Kruskal’s algorithm, Prim’s algorithm explores the graph similar to other more “traditional” graph search algorithms like BFS, and Dijkstra’s algorithm in that it uses the Graph’s adjacency API. I also find it to be a less interesting algorithm but that is a purely personal opinion. It certainly *is* good to have a completely different algorithm to perform the same task so as to be able to compare results from unknown data sets. As such, here is a basic implementation of Prim’s algorithm.

class Prim {

private:

priority_queue<pair<int, int>, vector<pair<int,int>>, greater<pair<int,int>>> pq;

bool *seen;

int *cost;

int *pred;

vector<pair<int,int>> _MST;

public:

Prim(Graph& G) {

seen = new bool[G.V()];

cost = new int[G.V()];

pred = new int[G.V()];

for (int i = 0; i < G.V(); i++) {

seen[i] = false;

cost[i] = std::numeric_limits<int>::max();

pred[i] = -1;

}

pq.push(make_pair(0, 0));

while (!pq.empty()) {

int curr = pq.top().second;

pq.pop();

if (!seen[curr]) {

seen[curr] = true;

if (pred[curr] != -1)

_MST.push_back(make_pair(pred[curr], curr));

for (auto it = G.adj(curr); !it.end(); it.next()) {

if (!seen[it.get()->u] && it.get()->wt < cost[it.get()->u]) {

cost[it.get()->u] = it.get()->wt;

pq.push(make_pair(cost[it.get()->u], it.get()->u));

pred[it.get()->u] = curr;

}

}

}

}

}

vector<pair<int,int>>& MST() {

return _MST;

}

};

Anyway, that’s all I’ve got for today, If you want the code for todays post, including that for creating the graphics of the MST and Graph, they are available on my Github.

Until next time, Happy Hacking!

]]>