Research

Associate Professor

Wolfgang Jank

Research on statistical challenges in online marketplaces

The following tutorial provides an overview of different statistical challenges when modeling electronic commerce data. Ecommerce data originates from many different behavioral, social, or economic processes and interactions online which have not been observable and measurable in the offline world. This data-rich environment allows for the questioning of existing theories and the uncovering of new phenomena. However, eCommerce data, and the new research questions associated with this data, are often not supported by classic statistical machinery. New dependency structures arise due to factors such as online competition and user interaction. In this paper, we discuss three key aspects of eCommerce data: eCommerce process dynamics, competition between processes, and user networks.

“Statistical Challenges in eCommerce: Modeling Dynamic and Networked Data”, Shmueli, Dass and Zhang.

Research on forecasting using prediction markets and functional data analysis:

Prediction markets are increasingly used to aggregate information from online communities and to forecast a wide range of events, e.g. presidential elections, sports wins/losses, and more recently, demand for new products. For example, a stock is IPO-ed when a new product development is publicized. The trading price at any time (e.g., $70 per share) reflects the traders’ current, collective expectation of the initial demand for the product (e.g., $70m revenues). Similar to real-life stock markets, new information about the product emerges over time and influences traders’ beliefs: those who believe a higher demand (e.g., $80m), compared to the current trading price, tend to buy, with real or endowed play money, and those who believe otherwise sell or short. Traders may buy, sell, or short at any time; and the winning (if bought low and sold high) and losing amount (if otherwise) is recorded in their personal account with the online VSM. Trading often terminates when the new product is released. The liquidation price of the stock equals the realized initial demand for the product.

In the following papers we propose new ways to make early & dynamic marketing decisions using information from prediction markets. We use functional shape analysis to tease-out mood-swings from the observed trading patterns or to predict decay rates of movie revenues. We use these trading patterns in a dynamic way and identify optimal points for decision making.

“Functional Forecasting of Demand Decay Rates using Online Virtual Stock Markets,” with James and Foutz

“Using Virtual Stock Exchanges to Forecast Box-Office Revenue via Functional Shape Analysis,” with Foutz

“The Wisdom of Crowds: Pre-release Forecasting via Functional Shape Analysis of the Online Virtual Stock Market,” with Foutz

Research on modeling dynamics and interdependencies of online auctions:

In this paper, we propose a novel automated and data-driven bidding strategy. Our strategy consists of two main components. First, we develop a dynamic, forward-looking model for price in competing auctions. By incorporating dynamic features of the auction process and its competitive environment, our model is capable of accurately predicting an auction’s price, taking into account information from simultaneous auctions. Then, using the idea of maximizing consumer surplus, we build a bidding framework around this model that determines a triumvirate of decision points: the best auction to bid on, the best bidtime and the best bid-amount. In simulations, we compare our automated strategy to early and last-minute bidding and find that our approach extracts considerably higher expected surplus. We also argue that our approach devotes significantly less effort to the process of bidding.

“An Automated and Data-Driven Bidding Strategy for Online Auctions,” with Zhang.

The following paper is new in the sense that it studies dynamics in a special class of auctions, simultaneous auctions for Indian contemporary art. We develop a novel forecasting model that predicts price in ongoing auctions, using the concept of price dynamics. We also study the source of the predictive power of dynamics and find that dynamics capture bidder competition within and across auctions. The importance of this finding is both conceptual and practical: price dynamics are simple to compute at high accuracy, as they require information only from the focal auction and are therefore a parsimonious representation of different forms of within-auction and between-auction competition.

“Dynamic Price Forecasting In Simultaneous Online Art Auctions,” with Shmueli and Dass.

The following papers focus on the dynamics during online auctions. We propose different ways of capturing dynamics using functional data analysis. We also propose several new ways of modeling auction dynamics via differential equation models and differential equation trees. One of the key insights of this research is that dynamics exists and that they matter for the outcome of an auction.

“Modeling Price Dynamics in Online Auctions via Regression Trees,” with Shmueli and Wang.

“Modeling Price Dynamics in eBay Auctions Using Principal Differential Analysis,” with Wang and Shmueli and Smith.

“Studying Heterogeneity of Price Evolution in eBay Auctions via Functional Clustering”, with Shmueli.

“Price Formation and its Dynamics in Online Auctions,” with Bapna and Shmueli.

“Modeling the Dynamics of Online Auctions: A Modern Statistical Approach”, with Shmueli.

In the following set of papers we focus on forecasting the price in online auctions. We develop a dynamic and real-time forecasting model based on functional data analysis. Another aspect that makes this work novel is that the forecasting method takes advantage of the changing dynamics during an online auction.

“Dynamic, Real-time Forecasting of Online Auctions via Functional Models,” with Shmueli and Wang.

“Explaining and Forecasting Online Auction Prices and their Dynamics using Functional Data Analysis,” with Wang and Shmueli.

In the following set of papers we focus in competition between auctions. We are particularly interested in understanding the price-process of concurrent auctions and whether certain types of price-processes occur in groups. We also investigate the effect of competing products in the associated feature space and whether time plays a factor in online auctions.

“Modeling Concurrency of Events in Online Auctions via Spatio-Temporal Semiparametric Models” with Shmueli.

“Investigating Concurrency in Online Auctions through Visualization,” with Hyde and Shmueli.

“A Family of Growth Models for Representing the Price Process in Online Auctions,” with Hyde and Shmueli.

In the following set of papers we explore auction data visually. We develop several static visualizations to better sift through large amounts of auction data. We also develop a novel interactive tool to explore & forecast online auction price processes.

“Visualizing Online Auctions", with Shmueli.

“Visualizing Functional Data with an Application to eBay’s Online Auctions”, with Shmueli, Plaisant, and Shneiderman.

“Representing Unevenly-Spaced Time Series Data for Visualization and Interactive Exploration,” with Aris, Shneiderman, Plaisant and Shmueli.

“Exploring Auction Databases through Interactive Visualization,” with Shmueli, Aris, Plaisant, Shneiderman.

The following paper proposes a novel model for the bid arrival. In particular, our model captures the 3 distinct bidding phases that characterize a typical auction. The model is based on a non-homogeneous Poisson process.

“The BARRISTA: A model for Bid Arrivals in Online Auctions,” with Shmueli and Russo

The following paper proposes a novel way to distinguish between private and common value auction. The idea is based on the functional residual analysis.

“A Pre-Theory Functional Approach for Detecting Private- or Affiliated-Value Auction Settings,” with Bapna and Shmueli, (2006). [R software code for the “Winner’s Curse Test” can be found here.]

The following paper features a novel data collection method to gauge consumer surplus in online auctions.

“Consumer Surplus in Online Auctions," with Bapna and Shmueli.

The following papers feature methodological work motivated by the data challenges in online auctions. The first paper gives an overview of functional data methods in the context of electronic commerce and argues why many data structures found in electronic commerce motivate the use of functional data analysis. The second paper proposes a novel way of smoothing unevenly sampled price curves (such as those found in online auctions). The third paper proposes several ways of how to deal with data that is neither entirely continuous nor entirely discrete.

“Functional Data Analysis in Electronic Commerce Research,” with Shmueli.

“Smoothing Sparse and Unevenly Sampled Curves using Semiparametric Mixed Models: An Application to Online Auctions,” with Reithinger, Tutz, and Shmueli.

“Transformations for Semi-Continuous Data,” with Hyde and Shmueli.

Research on dynamic and geographical models of customers choices:

In this line of research we look at the geographical scatter of customer choices. In particular, we want to better understand how the choices customer make are determined by other customers in close (geographical) proximity. We develop a novel spatial model for customer choices. We also derive a dynamic implementation of that model that can react to changes in the population and give updated decisions in real-time.

“Dynamic Spatial Models in Online Markets” with Kannan.

“Dynamic E-Targeting using Learning Spatial Choice Models” with Kannan.

“Understanding geographical markets of online firms using spatial models of customer choice", with Kannan..

Research on aviation and flight departure delays:

In this paper we propose a novel way for capturing flight departure delay. We differentiate between short-term daily patterns and long-term trends. We use global optimization methods to best fit a mixture distribution to the observed data.

“Estimating Flight Departure Delay Distributions – A Statistical Approach with Long-Term Trend and Short-Term Pattern,” with Tu and Ball.

Research on stochastic estimation & optimization, in particular stochastic versions of the Expectation-Maximization (EM) algorithm.

The following papers develop different methods and algorithms to make the EM algorithm amenable for finding the global solution. Like many other deterministic optimization methods, EM can get stuck in local, sub-optimal solutions. We develop a variety of methods that can overcome these local traps.

“New Global Optimization Algorithms for Model-Based Clustering,” with Heath and Fu.

“Global Convergence of Model Reference Adaptive Search for Gaussian Mixtures,” with Heath and Fu.

“Ascent EM for Fast and Global Model-Based Clustering: An Application to Curve-Clustering of Online Auctions.”

The following papers propose several new ways of implementing EM (and in particular its stochastic version, Monte Carlo EM) more efficiently. Among the proposed solutions are new (automated) rules for increasing the Monte Carlo sample size and new rules for diagnosing convergence.

“Ascent-Based Monte Carlo EM", with Caffo and Jones.

“Implementing and Diagnosing the Stochastic Approximation EM Algorithm.”

“Quasi-Monte Carlo Sampling to Improve the Efficiency of Monte Carlo EM.”

“Fast and Efficient Model-Based Clustering with the Ascent-EM Algorithm.”

The following paper proposes several ways to implement Simulated Maximum Likelihood (SML) more efficiently. Among the proposed solutions are using SML in stages and using variance-reducing Quasi-Monte Carlo simulation methods.

“Efficient Simulated Maximum Likelihood with an Application to Online Retailing.”

The following paper (empirically & theoretically) compares the efficiency of Monte Carlo EM to that of Simulated Maximum Likelihood.

“Efficiency of Monte Carlo EM and Simulated Maximum Likelihood in Two-Stage Hierarchical models.”

The following papers give a detailed overview over problems and challenges associated with stochastic implementations of the EM algorithm.

“Stochastic Variants of EM: Monte Carlo, Quasi-Monte Carlo and More.”

“A Survey of Monte Carlo algorithms for Maximizing the Likelihood of a Two-Stage Hierarchical model.”

“The EM algorithm, Its Stochastic Implementation and Global Optimization: Some Challenges and Opportunities for OR.”