Four years ago I remade a university project using C++ as a learning project (See original post). Since then I have a lot more experience with C++, working with it for Unreal in a professional context. I wanted to revisit this project again, this time using some more of the skills I have learned along the way.
Checkout the new project here: https://github.com/Ezra-Mason/XOR-gate-neural-network/
Improvements
One of the first issues I found with the old code was I didn’t make any reference in how to compile or run the code. At the time I was using g++ and VS Code and the Code Runner plugin. This time for build system I opted to for a Visual Studio project which can handle that, since its only really for windows this seems reasonable rather than using setting up CMake.
The next thing I noticed was the uses of arrays and std::vectors. None of the code expects any of the lists to be dynamic, so this time I’m using arrays to simply allocate known size arrays at compile time. Another note about the vectors is the truth table. Originally I made a 2D array for easier formatting when writing it out but this can just be a single array. So instead of table[i][j] and can just call table[i*4 + j]. The truth table, along with a lot of other variables in the code, can be made const since we dont ever expect their values to change.
One major thing present in the methods of the older code was no method took parameters passed as references or pointers, instead there was a lot of copying of the parameters.
The algorithm itself hasn’t changed except for making explicit the copying of values from the new weights to current weights using std::copy(). Additionally, the threshold function didn’t need to assign a local variable for the steepness of the function, instead it can just return the calculated value.
The final major change to the algorithm itself is removing the repeated, and hard coded, calculation of the hidden nodes output
double H4 = Sigmoid(
weights[0]+
weights[4]*truth_table[i][0]+
weights[5]*truth_table[i][1]+
weights[6]*truth_table[i][2]);
The interesting thing about the argument of these methods is that it is essentially calculating the dot(weights, input). With this realisation we can then make the whole feed forward calculation much smoother and loop over the three hidden nodes instead of manually.
for (size_t j = 0; j < HiddenNodeCount; j++)
{
const int StartIndex = HiddenNodeCount * (j + 1) + 1;
const int EndIndex = StartIndex + HiddenNodeCount;
HiddenOutputs.emplace_back(SigmoidThreshold(
Weights[j] +
std::inner_product(
Weights + StartIndex,
Weights + EndIndex,
Inputs.begin(),
0.f)));
}
I chose not to include the bias weight within the dot product for two reasons. One its not placed contiguously with the other weights related to the given hidden node and would therefore require allocating new lists for each hidden node and allocating a new inputs array with a dummy 1.f in to maintaining the calculation. Two, it’s weight represents the bias of the hidden node not a connection into the node, explicitly leaving it form the calculation reminds me that it is separate from the other weights.
Conclusion
Overall it was a nice way to spend a few hours reviewing some older code and writing a smaller project. I hope to add to the project more with an implementation of back-propagation and compare the results of the two methods!
Ez Mason 