OCDevel AI Podcast

Logistic Regression and What Classification Really Is: Sigmoid, Log-Odds, Cross-Entropy, the Linear Decision Boundary, and Building One Neuron from Scratch

Logistic Regression and What Classification Really Is

Phase 1 continues. Last episode fit a line for regression; this one turns that same linear score into a classifier. Classification predicts a discrete category (usually with a probability), not a continuous quantity.

The machine: keep the linear score z = w·x + b, then squash it with the sigmoid σ(z) = 1/(1+e^−z) into a probability p = P(y=1|x). Invert it and you get the logit: log(p/(1−p)) = w·x + b, so logistic regression is linear in log-odds. Each weight is an odds ratio via e^(w_j), why banks and epidemiology still use it for interpretable reason codes.

Why not squared error: MSE on a sigmoid is non-convex with vanishing gradients on the worst mistakes. The right loss is log loss / binary cross-entropy, which is the negative log-likelihood of a Bernoulli model, classification's version of MSE-as-Gaussian-MLE. Its gradient collapses to the clean (p − y)·x, the same form as linear regression.

Also covered: the linear decision boundary (a hyperplane) and why logistic regression can't solve XOR; the from-scratch NumPy rebuild (swap identity→sigmoid, MSE→cross-entropy, keep the same update); L2/L1/elastic-net regularization and the perfect-separation blow-up; multiclass via one-vs-rest vs softmax; a light look at accuracy's trap on imbalanced data (full metrics next episode); history from Verhulst's 1838 growth curve to Berkson's "logit" (1944) and Cox (1958); and the punchline, logistic regression is one neuron.

Worked example: scikit-learn 1.9.0 LogisticRegression on Breast Cancer Wisconsin, with StandardScaler, predict_proba, decision_function, and coefficient reading.

In the news (June 28–July 5, 2026)

  • Anthropic Claude Sonnet 5, new default for Free/Pro users; 1M context, intro pricing $2/$10 per M tokens; reported ~63.2% SWE-bench Pro.
  • Meituan LongCat-2.0 (VentureBeat, SiliconANGLE), 1.6T-param MoE coding model, MIT license, reportedly trained and served on non-Nvidia Chinese chips.
  • Google Gemini image models GA, Gemini 3 Pro Image ("Nano Banana Pro"), strong in-image text, SynthID watermark.
  • Anthropic-led jailbreak severity framework + safety classifier, reported false positives (a precision/recall story).
  • Ai2 MolmoMotion, open vision-language motion model, weights + dataset on Hugging Face (allenai).