Module 11 // Predictive Layer

Machine Learning

> The full ML lifecycle — from framing a problem and preparing features to training, evaluating, and deploying predictive models with scikit-learn.

What is Machine Learning?

Machine Learning (ML) is the practice of building algorithms that learn patterns from historical data to make predictions or decisions on new, unseen data — without being explicitly programmed for every rule.

Supervised Learning — labeled data, predict a target (regression / classification).
Unsupervised Learning — no labels, find structure (clustering, dimensionality reduction).
Reinforcement Learning — agent learns by interacting with an environment to maximize reward.

The ML Workflow

Frame the problem & define success metrics
Collect & explore the data (EDA)
Clean & engineer features
Split into train / validation / test sets
Train candidate models & tune hyperparameters
Evaluate on held-out data
Deploy & monitor in production

Train / Test Split

Always evaluate on data the model has never seen — otherwise scores lie.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,
    random_state=42,
    stratify=y,           class=class="str">"com"># keep class balance for classification
)

Linear Regression

Predict a continuous target as a weighted sum of input features.

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
import numpy as np

model = LinearRegression()
model.fit(X_train, y_train)

preds = model.predict(X_test)
rmse = np.sqrt(mean_squared_error(y_test, preds))
r2   = r2_score(y_test, preds)

print(fclass="str">"RMSE: {rmse:.2f}")
print(fclass="str">"R²  : {r2:.3f}")
print(class="str">"Coefficients:", dict(zip(X.columns, model.coef_)))

Logistic Regression (Classification)

Despite the name, logistic regression is a classifier — it outputs the probability of a class.

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix

clf = LogisticRegression(max_iter=1000, class_weight=class="str">"balanced")
clf.fit(X_train, y_train)

y_pred = clf.predict(X_test)
y_prob = clf.predict_proba(X_test)[:, 1]

print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))

Decision Trees & Random Forests

Tree-based models capture non-linear interactions without feature scaling. Random Forests average many trees to reduce variance.

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(
    n_estimators=300,
    max_depth=None,
    min_samples_leaf=2,
    n_jobs=-1,
    random_state=42,
)
rf.fit(X_train, y_train)

class=class="str">"com"># Feature importance — which signals drove the decisions?
importances = sorted(
    zip(X.columns, rf.feature_importances_),
    key=lambda x: x[1], reverse=True,
)
for name, score in importances[:10]:
    print(fclass="str">"{name:<25} {score:.4f}")

Gradient Boosting (XGBoost / LightGBM)

Boosted trees are the default winning approach on tabular data — they build trees sequentially, each correcting the errors of the previous.

from xgboost import XGBClassifier

xgb = XGBClassifier(
    n_estimators=500,
    learning_rate=0.05,
    max_depth=6,
    subsample=0.8,
    colsample_bytree=0.8,
    eval_metric=class="str">"logloss",
)
xgb.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)

Unsupervised: K-Means Clustering

Group similar rows when you have no labels — segmentation, anomaly detection.

from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler

X_scaled = StandardScaler().fit_transform(X)

km = KMeans(n_clusters=4, n_init=10, random_state=42)
labels = km.fit_predict(X_scaled)

class=class="str">"com"># Elbow method: pick k where inertia stops dropping
inertias = [KMeans(n_clusters=k, n_init=10).fit(X_scaled).inertia_
            for k in range(1, 10)]

Feature Engineering & Pipelines

Wrap preprocessing + model into a single Pipeline so the same transformations apply at train and inference time — no leakage.

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer

numeric = [class="str">"age", class="str">"income", class="str">"tenure"]
categorical = [class="str">"country", class="str">"plan"]

preprocess = ColumnTransformer([
    (class="str">"num", Pipeline([
        (class="str">"impute", SimpleImputer(strategy=class="str">"median")),
        (class="str">"scale", StandardScaler()),
    ]), numeric),
    (class="str">"cat", OneHotEncoder(handle_unknown=class="str">"ignore"), categorical),
])

pipe = Pipeline([
    (class="str">"prep",  preprocess),
    (class="str">"model", LogisticRegression(max_iter=1000)),
])

pipe.fit(X_train, y_train)
pipe.score(X_test, y_test)

Cross-Validation & Hyperparameter Tuning

from sklearn.model_selection import GridSearchCV

param_grid = {
    class="str">"model__C": [0.01, 0.1, 1, 10],
    class="str">"model__penalty": [class="str">"l1", class="str">"l2"],
}

grid = GridSearchCV(
    pipe, param_grid,
    cv=5, scoring=class="str">"roc_auc", n_jobs=-1,
)
grid.fit(X_train, y_train)

print(class="str">"Best params:", grid.best_params_)
print(class="str">"Best AUC   :", grid.best_score_)

Evaluation Metrics

Regression — MAE, RMSE, R², MAPE
Classification — Accuracy, Precision, Recall, F1, ROC-AUC, PR-AUC
Clustering — Silhouette score, Davies–Bouldin index
Ranking — NDCG, MAP@k

Bias–Variance & Overfitting

Underfit models miss real patterns (high bias). Overfit models memorize noise (high variance). Cure overfitting with: more data, regularization, simpler models, cross-validation, or early stopping.

Saving & Serving Models

import joblib

class=class="str">"com"># Persist the fitted pipeline (preprocessing + model)
joblib.dump(pipe, class="str">"model.joblib")

class=class="str">"com"># Load it later — in an API, batch job, or notebook
loaded = joblib.load(class="str">"model.joblib")
prediction = loaded.predict(new_data)

Next Steps

Deep Learning with PyTorch or TensorFlow for unstructured data (images, text)
MLOps: experiment tracking (MLflow), model registries, CI/CD for models
Feature stores & online inference (Feast, Redis)
Explainability: SHAP, LIME, partial dependence plots