Hoofdstuk 31 — Modelselectie en evaluatie: cross‑validation, metrics en GridSearchCV

Let op: De tekst is in het Nederlands; alle Python codevoorbeelden zijn in het Engels.

Doelen:

• Cross‑validation begrijpen en toepassen (KFold, StratifiedKFold, GroupKFold).
• Juiste metriek kiezen en meerdere metriekken rapporteren.
• Hyperparameter tuning met GridSearchCV en RandomizedSearchCV.

Installatie:

pip install scikit-learn numpy pandas

31.1 Cross‑validation basics

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.datasets import load_breast_cancer

X, y = load_breast_cancer(return_X_y=True)
pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression(max_iter=2000))
])
scores = cross_val_score(pipe, X, y, cv=5, scoring='accuracy')
print('Accuracy 5‑fold:', scores.mean(), '+/-', scores.std())

31.2 StratifiedKFold en meerdere metrics

from sklearn.model_selection import StratifiedKFold, cross_validate
from sklearn.metrics import make_scorer, f1_score, roc_auc_score

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
res = cross_validate(
    pipe, X, y, cv=cv,
    scoring={'acc':'accuracy', 'f1':make_scorer(f1_score), 'auc':'roc_auc'},
    return_train_score=False
)
print({k: res[f'test_{k}'].mean() for k in ['acc','f1','auc']})

31.3 GridSearchCV

from sklearn.model_selection import GridSearchCV

param_grid = {
    'clf__C': [0.1, 1.0, 10.0],
    'clf__penalty': ['l2'],
    'clf__solver': ['lbfgs', 'liblinear']
}

grid = GridSearchCV(pipe, param_grid=param_grid, cv=5, scoring='roc_auc', n_jobs=-1)
grid.fit(X, y)
print('Best params:', grid.best_params_)
print('Best score:', grid.best_score_)

31.4 RandomizedSearchCV en distributions

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import loguniform

param_dist = {
    'clf__C': loguniform(1e-3, 1e2)
}
rand = RandomizedSearchCV(pipe, param_distributions=param_dist, n_iter=20, cv=5, scoring='roc_auc', random_state=42)
rand.fit(X, y)
print('Best params:', rand.best_params_)

31.5 Groeps‑CV (kort)

from sklearn.model_selection import GroupKFold
import numpy as np

X_small = X[:200]
y_small = y[:200]
# Stel dat elke 20 rijen tot dezelfde groep behoren
groups = np.repeat(np.arange(10), 20)
cv = GroupKFold(n_splits=5)
for train_idx, test_idx in cv.split(X_small, y_small, groups=groups):
    pass  # normal fit/eval flow

31.6 Samenvatting

• Gebruik CV om variantie te reduceren en robuuste schattingen te krijgen.
• Rapporteer meerdere relevante metriekken.
• Stem hyperparameters af met GridSearchCV of RandomizedSearchCV; houd data‑lekken tegen met pipelines.