Evaluating Recommender Systems: A (Semi-)Synthetic Dataset Approach

Recommender system evaluation is predominantly conducted through offline evaluations, which utilize historical datasets of user behavior (e.g., clicks, purchases, or music listening histories) to assess whether the recommendations align with user preferences. However, a recent survey has revealed that only a limited number of datasets are commonly used for such evaluations, thereby restricting the assessment of algorithm generalizability. To address this limitation, we aim to develop synthetic datasets that allow testing of algorithms across diverse data conditions, characteristics, and larger scales. The objective of this bachelor’s thesis is to create (semi-)synthetic datasets for evaluating recommender systems and to assess the performance of a set of widely used recommendation algorithms on these datasets.