Get startedCompare models on real Iris flowers

Static notebook preview · outputs are cleared. Open in Colab or download the notebook to run the cells. Colab setup

python

!pip install "welt-client[parquet]"

Compare models on real Iris flowers

Run the installation cell first, connect your workspace, then run the remaining cells in order. These are real hosted requests: the evaluation and the final fit each use workspace credits and retain a private training dataset. No extra helper file is needed. Outputs are deliberately not saved in this source.

python

from welt import Client

with Client() as client:
    client.connect()

We follow seven steps: get data → inspect → hold out test rows → cross-validate models on the training rows → read the leaderboard → fit the chosen model → check it on the held-out rows. Iris has 150 rows, four measurements and three encoded classes. Seed9 stratified120/30 split.

python

from pathlib import Path
from tempfile import mkdtemp
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
from welt import Client

# 1. Get a real table from Welt's research inventory.
dataset_id = "asset-d839644b6de36fc5cafdda18c5daeca3"
path = Path(mkdtemp(prefix="welt-iris-")) / "iris.parquet"
with Client() as client:
    info = client.research_dataset(dataset_id)
    print("Dataset:", info["name"])
    client.download_research_dataset(dataset_id, path)
data = pd.read_parquet(path)
target = "target"  # The outcome column in this table.

python

# 2. Look at the rows before modelling.
print(data.head())
print("Rows and columns:", data.shape)
print(data[target].value_counts())

python

# 3. Set aside test rows first. The evaluation only sees the training rows,
#    so the test rows stay an unbiased check of the model you choose.
train, test = train_test_split(
    data, test_size=0.2, random_state=9, stratify=data[target])
query = test.drop(columns=target)
y_test = test[target]
print("Training rows:", len(train), "— Test rows:", len(test))

Cross-validation splits the training rows into five folds. Each model predicts every fold from the other four, so no row is predicted by a model that has seen it. With models left empty, Welt evaluates every available catalogue model whose limits admit the table.

python

# 4. Cross-validate models on the training rows.
client = Client()
evaluation = client.evaluate(train, target, task="classification", folds=5, seed=9)
print("Evaluation:", evaluation.id, evaluation.status)
print("Evaluated:", [m["model_id"] for m in evaluation.models])
for model in evaluation.skipped:
    print("Skipped:", model["model_id"], model["reason"])
print("Credits:", evaluation.credits)

python

# 5. Read the leaderboard: each model's mean and spread across folds, then the baseline.
summary = evaluation.summary()
print(summary[["name", "rank", "log_loss_mean", "log_loss_sd", "accuracy_mean", "accuracy_sd"]])
print("Ranked by:", evaluation.primary_metric)
for warning in evaluation.warnings:
    print("Note:", warning["message"])

python

# Per-fold accuracy for every model, and the out-of-fold predictions.
folds = evaluation.folds()
print(folds[folds.metric == "accuracy"].pivot(index="fold", columns="model_id", values="value"))
oof = evaluation.predictions()
print(oof.head())

The report shows the leaderboard, fold metrics and charts. It appears in a sandboxed frame inside the notebook. A report link lets someone without a Welt account read it; anyone holding the link can open it until it expires, so share it like a password.

python

evaluation.show()

python

link = evaluation.report_link(days=1)
print("Report link expires:", link.expires_at)
# print(link.url)  # Shown once. Share it like a password.

evaluation.fit() runs a new fit of the leaderboard's first model on all the training rows. It does not reuse a fold model. Then check it once on the held-out test rows.

python

# 6. Fit the chosen model on all the training rows.
model = evaluation.fit()
print("Fitted:", model.model, "Predictor:", model.predictor_id_)

python

# 7. Predict the held-out rows, without passing their target column.
predictions = model.predict(query)
print("Test accuracy:", accuracy_score(y_test, predictions))
print(classification_report(y_test, predictions, zero_division=0))

Cross-validated scores describe this table and these folds; no minimum score is promised. Use the leaderboard to choose, and the held-out test rows once to assess the choice. A tie between the top two models means the difference is within fold-to-fold variation.

python

print("Pinned model version:", model.model_version_)
print("Evaluation to reopen later:", evaluation.id)
client.close()

Suggest an edit