28th JANUARY, 2026
Table of Contents:
Like any technical field, machine learning has its share of intimidating terminology. Backpropagation, gradient descent, activation functions - the list goes on. But at the heart of it all is something called a neuron. The name comes from neuroscience, but what does it actually mean in this context?
To put it simply: it's a function that takes in input, do some calculations and outputs a single number. Yup. That's exactly what it is. Let's have fun exploring it.
Let's zoom in on an individual neuron. We immediately see that the signals (inputs) are nothing else than single numbers (there can be many more inputs!). Then something happens inside the neuron (function) that outputs a single number. So, many inputs, single output. But what exactly happens inside?
We must go deeper to unveil the math that produces the output.
WOW! At this zoom level we can clearly see the math! There are even some values that the neuron stores - weights and bias - these values are the result of training. And when the model is trained, these values are essentially frozen. Let's write the math down.
Here is the set of calulcations each neuron performs:
Now, let's address the elephant in the room: What is sigmoid? Sigmoid is what's called an activation function. Think of it as a translator that takes any number (positive or negative, large or small) and squashes it into a range between 0 and 1. In our example, we calculated 1.01, and sigmoid transformed it into 0.73.
Why do we need this? Without activation functions, neurons would just be performing linear algebra - multiplying and adding numbers. The sigmoid adds non-linearity, which allows neural networks to learn complex patterns. It's like the difference between drawing only straight lines versus being able to draw curves.
There are other activation functions too (ReLU, tanh, etc.), but sigmoid is a classic choice that's easy to understand and visualize.
Now that we understand what happens inside a single neuron, let's zoom back out. In practice, neurons are rarely used alone - they're organized into layers that work together.
A typical neural network has three types of layers:
Input layer - This is where data enters the network. Each neuron in this layer holds one piece of your input data (like a pixel value in an image, or a word in a sentence).
Hidden layers - These are the layers between input and output where the "magic" happens. Each neuron takes outputs from the previous layer, performs its calculations (weights, bias, activation function), and passes results forward. Networks can have many hidden layers stacked together - that's where "deep" in "deep learning" comes from.
Output layer - This final layer produces the network's answer. For a cat-vs-dog classifier, it might have two neurons (one for "cat probability", one for "dog probability"). For generating text, it might have thousands of neurons representing different possible next words.
The real power emerges when these layers connect: the output of each neuron becomes the input for neurons in the next layer, creating a cascade of calculations that can learn remarkably complex patterns.