Building a Neural Network from scratch in Go

17 Jun 2026 · Stone Liu

Neural Networks

Neural Networks have always been quite mysterious to me, even after learning about them in school I could not fully grasped how the inner machinery of the network learned arbitrary tasks such as image classification. So like any normal person sought to do, I decided I would take on the task of building it completely from scratch myself using this book.

Data Modeling

I first started out by modeling a perceptron, its a type of neuron that takes as input several bits and produces a binary output or . We can take each of the inputs to our neuron and add a threshold which determines if the neuron activates or not. The weight to each input represents the importance of the input. This can be formalized as the following rule:

In code, I represented my perceptron as such:

type Perceptron struct {
  threshold float64
  // Number of inputs the perceptron is allowed to take.
  inputs    uint
}

func CreatePerceptron(threshold float64, inputSize uint) *Perceptron {
  return &Perceptron{
    threshold: threshold,
    inputs:    inputSize,
  }

}

// Input is a single binary input together with its weight.
type Input struct {
  X uint8
  W float64
}

Whats interesting about perceptrons is that it can serve as a universal model for computation. For instance you can create a NAND gate by taking two binary inputs each with a weight of and a bias of .

func NAND(x1 uint8, x2 uint8) uint8 {
  // 3 is the bias and 2 is the number of inputs
  p1 := CreatePerceptron(3, 2)
  return p1.Forward(Input{X: x1, W: -2}, Input{X: x2, W: -2})
}

You can also model other functions such as XOR by building a network of NAND perceptrons. However the problem with perceptrons is that it can never truly learn, the activations are all linear and thus can only linear relationships and nothing more complex.

The Sigmoid Neuron

The sigmoid neuron is the defacto standard when first learning artificial neural networks. It solves the problem of the Perceptron, since the activations there were merely a step function and composing a network of perceptrons could only truly learn linear relationships.

TBD (To Be Continued)…