OCDevel AI Podcast

NumPy and pandas: The Working Tools of Machine Learning

Phase 1 foundations: the two libraries every later machine learning episode assumes you already know. This is array and dataframe fluency, not modeling yet.

NumPy. Why Python lists are slow for numbers (boxed objects, pointer-chasing) and why the ndarray is fast (contiguous typed memory, vectorized C loops, BLAS/LAPACK). Anatomy of an array: dtype, shape, strides, and why slicing returns a view. Creating arrays (arange, linspace, zeros, random.default_rng), ufuncs and vectorization, broadcasting rules and the (3,) vs (3,1) pitfall, views vs copies (slices are views, fancy/boolean indexing copies), the axis argument, and linear algebra (@ vs *, np.linalg.solve for the normal equations). Pitfalls: silent integer overflow, NaN propagation, float equality (np.isclose), and ragged arrays.

pandas. What it adds over NumPy: labeled axes, heterogeneous columns, missing data, time series, and relational ops. Reading data with read_csv (parse_dates, dtype, na_values), .loc vs .iloc, boolean filtering, the historical SettingWithCopyWarning and why Copy-on-Write replaced it, missing data handling, split-apply-combine with groupby, merge/join/concat and many-to-many explosions, vectorized .str and .dt, reshaping and tidy data, categoricals, method chaining, and the PyArrow backend. Crossing back to ML with df.to_numpy().

Version notes: NumPy 2.0 (June 2024, NEP-50 promotion, StringDType), current line 2.4.x, and pandas 3.0 (January 2026: Copy-on-Write is the only mode, SettingWithCopyWarning removed, string dtype by default, non-nanosecond datetimes). See What's new in pandas 3.0 and InfoQ's coverage.

News roundup. SpaceX agreed to acquire Cursor parent Anysphere for $60B all-stock (CNBC, Fortune). Tencent reportedly backed ex-Qwen lead Junyang Lin's new world-models lab (Techmeme). AI leaders met G7 at Evian on voluntary commitments (CNBC). OpenAI published Deployment Simulation. And Gemini API preview image/video models are slated to shut down late June 2026.