The main challenge in implementing IAMs in a corporation, as with many innovations, is securing buy-in that the time invested is worth the potential benefits. It helps that certain teams are forward thinking and some of these same teams have been burned by poor forecasting performance in recent years. We generally look for those customers first. In fact, we have had little trouble finding volunteers–teams–that want to try something new. At present we have as many teams wanting to run experiments as we can accommodate.

As we propose market mechanisms to aid with forecasting, potential participants and managers have most often expressed three concerns: incentives, anonymity, and groupthink. Regarding incentives, why does it make sense to pay for performance when employees are already paid to do their jobs? This is an interesting question because most businesses think nothing of offering commissions for sales. Do businesses not already pay the sales force, and should they not be selling anyway? We learned that the first time an Intel factory achieved all of its performance targets across a suite of metrics was when a program offered direct incentives, i.e., cash to each individual employee, for that precise outcome. We do not feel it is out of line to offer forecasters incentives for performance or general market participants incentives for good information. The potential value of the improved forecast is orders of magnitude greater than the cost of the incentives.

The feature of anonymity is somewhat incongruous with Intel's culture of direct, constructive confrontation. If employees disagree they engage and resolve their differences. Allowing employees to participate in systems without identification (to others, not to the research staff running the system) is foreign and may be difficult for some employees to swallow. However, Intel is also a company that values results, and there is room in the culture for improvement.

Can IAMs enable or even cause groupthink? A classic approach toward defeating groupthink is assigning private roles to individuals. For instance, everyone on a team gets a card, and everyone knows that some cards say "devil's advocate." With some individuals assigned the role but no one knowing who those individuals are, everyone is able to dissent with less fear of reprisal. A market system where all participants are anonymous and incentivized for performance takes this approach to the limit, freeing individuals to express themselves. Interestingly, although some IAMs that enable participants to observe the group forecast develop could potentially lead to artificial consensus, in all market-like mechanisms the primary opportunity to win and win big comes from being right when everyone else is wrong. This feature certainly helps prevent too great a consensus.

A few more specific challenges have also been faced. Running synchronous IAMs across a global corporation is a problem, given that it is always 2 a.m. somewhere. Teams are reluctant to schedule anything out of normal hours, and it is challenging to find a good time for any large group of people to do something together. This issue is forcing us to consider asynchronous approaches as well.

Another issue has been dealing with a world made up of local geographies. If global sales are the sum of several geographies' sales, how does one tap local knowledge to forecast the global outcome? We have found our experts within a geography reluctant to try to forecast global results because they feel they do not have enough information to perform the task. That leaves three choices: limiting the markets to global forecasts and participants with a global view, running multiple markets specific to local geographies, or swaying the local experts to participate in a global forecasting market. In the latter case, participation is a critical consideration. If sales are 50% geo A, 30% geo B, and 20% geo C, do we need participation roughly proportional to sales from each geo? Or, is a result weighted by recent sales preferable to the formal market result, which is weighted by participation?

Two remaining challenges we have identified are scalability and long horizons. Forecasting total sales for a product family is valuable, but it does not address the mix of products or SKUs within those products. The market solution probably cannot scale to forecasting all SKUs, and it may not even be suited for that task. Perhaps the right balance is forecasting total product family sales and key products–new or of strategic importance–that will have the greatest impact on financial performance. Regarding horizons, markets are better suited to the short term. Incentives lose power if the payoff is too remote, and feedback is important for driving participation and performance. Forecasting a result within a few quarters seems to work, but over a year begins to feel like a stretch. We are experimenting with alternative market structures that might help forecast the distant future while paying incentives more quickly.