In this talk, we study a feedback form framework for preference learning called ranked choices. In this setting, participants rank their top-k choices from an individualized display set. We introduce a distance-based (Mallows-type) ranking model using a new distance function termed reverse major index (RMJ). Despite the requirement to sum over all permutations, the RMJ-based ranking model allows simple closed-form expressions for (ranked) choice probabilities. As a result, it enables efficient methods to infer model parameters from ranked choice data with provable consistency. Comprehensive numerical studies demonstrate the model’s favorable generalization power, robustness, and computational efficiency.

We also consider a prescriptive application. Suppose a company aims to learn customer preferences over a range of product alternatives and identify the most preferred one with high probability. The company can sequentially decide on the display set and request customers’ top-k ranked choices. Under a sequential experimental design framework, we characterize the (asymptotic) sample complexity under the optimal display set offering procedure. We also investigate the interplay between feedback sophistication (represented by k) and its information efficiency. We find that, although the information efficiency always increases with k, a small value of k=2 is already close (and sometimes equal) to that of full-ranking feedback.

Paper link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4539900