tarexp.component.stopping module

Stopping rules are a particular focus of TAR research.

class tarexp.component.stopping.StoppingRule(target_recall: float | None = None)

Bases: Component

checkStopping(ledger: Ledger, *args, **kwargs) → bool

class tarexp.component.stopping.NullStoppingRule(target_recall: float | None = None)

Bases: StoppingRule

checkStopping(*args, **kwargs)

class tarexp.component.stopping.FixedRoundStoppingRule(max_round, *args, **kwargs)

Bases: StoppingRule

checkStopping(ledger, *args, **kwargs)

class tarexp.component.stopping.KneeStoppingRule(target_recall: float | None = None)

Bases: StoppingRule

See also

[1]

Gordon V. Cormack, and Maura R. Grossman. “Engineering quality and reliability in technology-assisted review.” Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 2016. https://dl.acm.org/doi/10.1145/2911451.2911510

checkStopping(ledger: Ledger, *args, **kwargs) → bool

class tarexp.component.stopping.BudgetStoppingRule(target_recall: float | None = None)

Bases: StoppingRule

See also

[2]

Gordon V. Cormack, and Maura R. Grossman. “Engineering quality and reliability in technology-assisted review.” Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 2016. https://dl.acm.org/doi/10.1145/2911451.2911510

checkStopping(ledger: Ledger, *args, **kwargs) → bool

class tarexp.component.stopping.ReviewHalfStoppingRule(target_recall: float | None = None)

Bases: StoppingRule

checkStopping(ledger: Ledger, *args, **kwargs) → bool

class tarexp.component.stopping.BatchPrecStoppingRule(prec_cutoff=0.025, slack=1)

Bases: StoppingRule

checkStopping(ledger: Ledger, *args, **kwargs) → bool

class tarexp.component.stopping.Rule2399StoppingRule(target_recall: float | None = None)

Bases: StoppingRule

checkStopping(ledger: Ledger, *args, **kwargs) → bool

class tarexp.component.stopping.QuantStoppingRule(target_recall: float, nstd: float = 0)

Bases: StoppingRule

See also

[3]

Eugene Yang, David D. Lewis, and Ophir Frieder. “Heuristic stopping rules for technology-assisted review.” Proceedings of the 21st ACM Symposium on Document Engineering. 2021. https://arxiv.org/abs/2106.09871

checkStopping(ledger: Ledger, workflow, **kwargs) → bool

class tarexp.component.stopping.CHMHeuristicsStoppingRule(target_recall: float, alpha=0.05)

Bases: StoppingRule

See also

[4]

Max W. Callaghan, and Finn Müller-Hansen. “Statistical stopping criteria for automated screening in systematic reviews.” Systematic Reviews 9.1 (2020): 1-14. https://pubmed.ncbi.nlm.nih.gov/33248464/

checkStopping(ledger: Ledger, *args, **kwargs) → bool