Skip to content

Commit

Permalink
README
Browse files Browse the repository at this point in the history
  • Loading branch information
jacobgil committed Mar 11, 2023
1 parent 8533ebe commit b05f6fe
Showing 1 changed file with 11 additions and 9 deletions.
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,6 @@

This is a package that computes common machine learning metrics like F1, and returns their confidence intervals.

In short, a confidence interval gives you a lower and upper bound on your metric. It's affected by the sample size, and the metric variability.
When making decisions based on metrics, you should prefer narrow intervals. If the interval is wide, you can't be confident it's not just by luck.


While confidence intervals are very commonly used by statisticans, with many R language implementations,

it is astonishnly rarely used by python users, although python took over the data science world.


⭐ Very easy to use, with the standard scikit-learn naming convention and interface:

e.g roc_auc_score(y_true, y_pred).
Expand All @@ -26,6 +17,17 @@ e.g roc_auc_score(y_true, y_pred).

⭐ East to use interface to compute confidence intervals on new metrics that don't appear here, with bootstrapping.

## The motivation

A confidence interval gives you a lower and upper bound on your metric. It's affected by the sample size, and the metric variability.
When making decisions based on metrics, you should prefer narrow intervals. If the interval is wide, you can't be confident that your high performing metric is not just by luck.

While confidence intervals are commonly used by statisticans, with many great R language implementations,
it is astonishingly rarely used by python users, although python took over the data science world !

Part of this is because there were no simple to use python packages for this.


## Getting started

```python
Expand Down

0 comments on commit b05f6fe

Please sign in to comment.