Forecast models can become complex, but the principles for gathering and vetting data for good predictions should remain basic.
An article from The Scientist – these look like principles that can be applied in lots of places, including journalism.
Would you like to be as successful in your forecasting? Here are five steps necessary for making good predictions.
1. Anchor your assumptions.
Before entering the meeting room with their pharmaceutical suitor Schering-Plough, Schoeneck, who is now CEO of San Diego-based BrainCells, and his team searched for data that they could confidently put into their forecast model. On the treatment side, they decided the best sources would be physicians currently dealing with patients. So they hired a marketing firm to conduct a survey of 500 rheumatologists and gastroenterologists, who were asked: "If there was a drug that fit this profile, what would you do?" To help predict how much they could charge for their drug, the team commissioned a similar survey of payers and managed-care organizations.
Gathering information is the most important part of building a forecast, say experts, and the process should be scaled up as a company grows. Smaller companies may rely on their scientist advisory boards as well as relationships with thought leaders. Novozymes, a leading supplier of enzymes and microorganisms, uses sophisticated S&OP (sales and operations planning) software and protocols to link different units for data input as well as directly tap into its customers' data. These tools help in developing and tracking forecasts in order to avoid supply-chain errors.
2. Offer a clear link to sources of data.
Sources of data should be clear and made visible internally to every key member of your company. That way, everybody on the team can probe the assumptions. In negotiations, offer up this transparency. "Both sides should have a transparent forecast structure and [should] share; otherwise you have a ‘my dad is bigger than your dad' argument, and you need a meeting of the minds," says Martin Joseph, head of information management and forecasting for global operations at AstraZeneca in Chesire, UK. This may mean taking a potential partner through your assumptions and their sources as well as your forecast model, says Schoeneck. For Centocor, the marketing team did the forecast by modeling on an Excel spreadsheet.
"The forecast is only as good as your assumptions," says Schoeneck. "In this case, we did so much back-up work, it gave us confidence. You're going in with more confidence in your data; forecasting becomes a more powerful weapon."
3. Focus on the process.
While forecasting is often seen as a results-oriented document - ‘Here's the numbers, here's what we need to do based on them' - its true value may be in the way it is created. "The emphasis should be what you can learn from the process," says James Stutz, director of corporate development at InterMune in Brisbane, Calif. Here, transparency extends to how the different units contribute the underlying assumptions and then review the results. Joseph says this gives the assumptions "no place to hide." By offering a clear link between the numbers and assumptions, each member of the team contributing to a forecast - including functional team heads in marketing, clinical operations, and sales, as well as forecasters and c-level executives - can probe the assumptions.
While the complexity of assumptions will differ radically depending on the company's focus and how the forecast is being used (for example, long-term strategy, short-term inventory management, sales targets), experts say a similar process should be in place. The heart of the process is gaining consensus throughout the organization. Bring in different parts of the organization, have them contribute their assumptions, and then force them and yourself to drill down on all the assumptions, says Stutz. Otherwise, "garbage in, garbage out" will prevail. Some companies, such as Aspreva Pharmaceuticals in Victoria, BC, have a dedicated forecaster to lead this effort. Others, such as BrainCells, a startup with 17 employees and one compound slated to enter Phase II trials in a few months, rely on the CEO and other executives to do the work.
4. Use the right person to build the forecast.
The job of the person leading the consensus meetings for data review is to "keep everybody honest," says Daniel Kiely, senior manager of strategic forecasting and market analysis for Celgene in Summit, NJ. "For example, a sales team may want to lower target number to get sales compensation. Is there gaming going on some times? Yes, there is. But the job of forecaster is to develop a consensus-based forecast, with forecaster as mediator to make sure bias is not introduced." To do so, says Kiely and others, the forecaster certainly needs technical skills in statistical modeling. But more important, he or she needs to be expert at facilitating meetings and gaining consensus among various functional areas. At larger companies, different product teams compete for resources; they know a bigger number is better, says Joseph. It is his job and that of his staff to ferret out bias and remove it.
5. Don't stop.
It is obvious that when events change - a new publication, approval of a competitor's new drug, a change in the standard of care for a patient population, etc. - underlying assumptions driving a forecast will change, and the numbers will need to be updated. However, the process must not be events-driven, say forecasters. Rather, it needs to be ongoing; some experts suggest monthly meetings to review the forecast. "The numbers here, fine, but a forecast is way out," says Joseph. "It is important to communicate and have discussions around it. People hitch a ride on numbers, and assumptions change, but people cling to the number."
