In machine learning, every data point lives in space. Each feature is an axis. When we ask how similar two points are, we are really asking how far apart they are, and choosing a ruler for that distance.
Same points. Different rulers.
L1 and L2 measure the same space differently. L1 adds absolute differences. L2 squares them, sums them, and takes the square root. That one choice changes what counts as close.
L1 (Manhattan): |x₁ − x₂| + |y₁ − y₂| + ...
L2 (Euclidean): √((x₁ − x₂)² + (y₁ − y₂)² + ...)
What the diamond and circle mean
Pick a point as your center. Ask which points are exactly distance one away. Draw all of them. The shape is the geometry of the metric.
The circle reaches equally far in every direction. The diamond reaches farther along the axes than along the diagonals. So with the same physical arrangement of points, the order in which neighbors are included can change.
- L2 / circle: an axis point at (1, 0) and a diagonal point at (1/√2, 1/√2) are both one unit from the origin.
- L1 / diamond: those same points have distances 1 and √2. The axis point is reached first because a diagonal move spends the distance budget on two axes.
Same data. Different distance rules, and that changes which points count as nearest.
Normalize first.
Before arguing about L1 versus L2, make sure the features speak comparable numerical languages. Imagine comparing two people using age and income. A one-year age difference and a thousand-unit income difference are not naturally commensurable just because both are numbers.
- Raw features: age = 30, income = 50,000. Without scaling, income can dominate the distance simply because it is written in larger units.
- After normalization: each feature is mapped to a comparable scale. Only then does the metric choice describe the behavior you actually want. Fit any scaling on training data, not on held-out data.
L1 does not fix different feature scales. Normalize the inputs first. Then choose how differences should be penalized.
How L1 and L2 differ after that.
Once features are on a shared scale, the useful distinction is how deviations combine. L2 squares the coordinate differences before summing and taking a square root, giving a large coordinate difference more influence than L1 does. L1 adds the magnitudes linearly.
Neither distance is universally better. The geometry should match the structure of the problem. Choosing an L1 distance does not, by itself, make a model sparse; that is a property associated with L1 regularization on model parameters.
Rotation changes the story.
L2 is rotationally symmetric: rotate the coordinate system and the distances do not change. L1 is tied to the axes. Rotate the same data and the diamond changes its relationship to the points. This is why the choice is also a statement about which directions in the feature space matter.
Losses and regularization: related, but distinct
In regression, absolute-error loss grows linearly while squared-error loss grows quadratically, so a large residual has more influence under squared error. Absolute error can therefore be less sensitive to large outliers in the target. Squared-error loss is the squared L2 norm of the residual vector, not the L2 distance itself.
In regularization, the penalty acts on model parameters rather than residuals. L1 penalties can drive some coefficients to zero, while squared L2 penalties usually shrink coefficients smoothly. Keep distance, loss, and regularization separate even though their geometry is related.
So which one should you use?
- For distances: choose L1 when adding axis-wise differences fits the problem; choose L2 when rotational symmetry and Euclidean proximity are appropriate.
- For regression losses: consider absolute error when reducing the influence of large residuals matters; consider squared error when large errors should receive more weight.
- For regularization: consider L1 when a sparse coefficient vector is useful, and L2 when you want smooth shrinkage across coefficients.
The important order is not “memorize the table.” It is: understand the scale of the inputs, picture the geometry, then choose the behavior that fits the decision.
The metric is a modeling choice about what “close” should mean, not a footnote after the model is already built.