Design Strategy

To improve the initial audit of a new supplier, we performed an analysis relating a supplier’s initial audit data to their future monthly scorecard data. The dataset for the analysis contained 70+ unique supplier operational excellence assessments (OEA) conducted by Steelcase. The features considered were the supplier’s score in each of the assessment categories of the OEA. Each of the models trained on 80% of the dataset and the remaining data was held-out to test on the final selected model. The random forest had the highest R-squared and lowest mean-squared error, though its performance is comparable to the decision tree.

The feature importance plot created from the random forest provides insight into which OEA categories are the most indicative predictors of scorecard performance, which is important for understanding the utility of the OEA itself. Steelcase has agreed that the relative importance of "Order Management and Scheduling," "Logistics and Inventory Management," and "Quality Systems" intuitively make sense as being the most important issues since they have to do most with the primary KPIs they track: reliability (OTD), and quality (DPPM).This information can be used to estimate a new supplier’s performance since the OEA can inform a prior distribution of the expected supplier performance. If Steelcase needs to add a supplier to its portfolio to enable a product launch, the PD&L team can have a much better understanding of the risk of using the new supplier and how they might perform relative to similar suppliers. 

Now that a new supplier’s prior distribution on expected performance can be initialized, we can compare new and existing suppliers on the basis of their projected performance. Our back-end algorithm uses a Thompson Sampling framework which leverages Bayesian updating to model performance distributions contextually based on observed data.

Thompson Sampling starts each round with a "prior" distribution derived from the previous information that has been collected on the supplier. This can include both the initial prior calculated from our random forest model as well as monthly scorecard data collected in previous rounds. We then use the most recent monthly scorecard score to update our prior distribution and calculate our posterior distribution for the supplier. Since both our prior and posterior distributions belong to the family of normal distributions, we can use conjugate distributions to update our beliefs.

We decided to use the Normal-Inverse Gamma conjugate prior distribution in our algorithm because it works with normal priors where the true μi and σi of the supplier are unknown. We also included exponential time weighting into our updating algorithm to prioritize recent information. In each round, the Normal-Inverse Gamma conjugate distribution outputs a posterior predictive. By mapping this posterior predictive to a confidence interval, we can visualize the performance and uncertainty associated with different suppliers in a way that is not possible by looking at the scorecards alone. This also gives us a common language for comparing suppliers with different amounts of data. 

With our Bayesian updating algorithm, we can accurately compare the predicted performance of individual suppliers. In practice, Steelcase not only wants to use our predictions to compare suppliers but to evaluate the strength of entire supply chains. We use the Sharpe Ratio to represent the risk-adjusted performance of a cohort of suppliers. The Sharpe Ratio is traditionally used in finance to represent the risk-adjusted performance of a portfolio of investment products and is calculated as S = E[Rp−Rf ] where Rp is portfolio return, Rf is the σp risk-free rate, and σp is the portfolio standard deviation. We altered this formula for our use case by defining S =μp where μp is the average predicted performance calculated by our Bayesian updating algorithm and σp σp is a portfolio variance metric. The Sharpe Ratio is a relative metric where a portfolio with a higher score has a higher risk-adjusted performance than one with a lower score. Steelcase can use this score to track supply chain performance over time and evaluate candidate supply chains for a new product.