| commit | b97c122cf6f9d06fc8ace542735b0d9e9f18e9bc | |
| author | Olly Betts <olly@survex.com> | |
| Thu, 20 Jun 2024 04:49:45 +0000 (20 16:49 +1200) | ||
| committer | Olly Betts <olly@survex.com> | |
| Thu, 20 Jun 2024 05:42:47 +0000 (20 17:42 +1200) | ||
| tree | bb8399424b7a34a7f26dfe35e4ff794cdc19bdb3 | treesnapshot (tar.gz zip) |
| parent | 23a58da02b64779407a64d0087a6cd0aa6e20da9 | commitdiff |
Reimplement Language Modelling weights
LMWeight was using incorrect formulae for all the smoothing schemes.
The replacement implementation separates each smoothing into a separate
class (except that Dirichlet and Dir+ are combined into one class),
since that seems to give a cleaner public API, and also simplifies the
implementation. It also means we can now clearly say "don't use
LMWeight" without risk of confusion.
Fixes #779, reported by Sourav Saha.
Closes https://github.com/xapian/xapian/pull/230
LMWeight was using incorrect formulae for all the smoothing schemes.
The replacement implementation separates each smoothing into a separate
class (except that Dirichlet and Dir+ are combined into one class),
since that seems to give a cleaner public API, and also simplifies the
implementation. It also means we can now clearly say "don't use
LMWeight" without risk of confusion.
Fixes #779, reported by Sourav Saha.
Closes https://github.com/xapian/xapian/pull/230