Abstract or Summary:
Several password guessers have been proposed in recent years. Comparing the reported performance numbers is difficult, as the experiments were performed on different datasets, with miscellaneous pre-processing applied, with varying numbers of guesses, and with different parameters. Re-running experiments under controlled conditions thus is essential for a fair comparison. Furthermore, re-running the experiments on newer password leaks not available earlier eliminates the risk of over-training a guesser to the available datasets.

In our first contribution, we developed a framework to automate and facilitate the comparison of password guessers on a large set of configurations. Central design criteria were ease-of-use, modularity, and easy expandability. Similar software is available in other disciplines (such as fingerprint matching and face recognition), and we believe the framework will help to drive and facilitate the future development of password guessers. In our second contribution, we used this framework to compare four well-known password guessers on a range of different password lists, resulting in a total of 148 experiments. We will make the framework publicly available and provide regular updates for the password guesser comparison online.

PasswordResearch.com Note: Video of presentation: https://www.youtube.com/watch?v=orRNwa1Fsrs Project page: https://www.password-guessing.org/static/index.php