Cougaar Software, Inc. (CSI) applies predictive analytics to improve forecast accuracy in their cognitive computing solutions. Improvements in predictions are achieved through the use of machine learning techniques such as support vector machines, which have been proven to be superior to traditional forecasting methods. CSI has previously applied predictive analytics to improve forecast accuracy for ammunition consumption by taking into account correlations, indicators, and patterns. In CSI’s previous systems, cognitive agents collect, fuse, and analyze battlespace conditions, mission activities, and supply/inventory data to continually identify and update trends in local consumption rates and develop and/or revise predictions for future demand of ammunition (Class V).
Below is a brief summary of predictive analytics and machine learning followed by additional information related to the application of these techniques for ammunition management.
Predictive Analytics with Machine Learning
Predictive analytics is a type of data analytics that attempts to understand what could happen in the future. Typically, predictive analytics encompasses a variety of statistical techniques, including machine learning. Tom Davenport explains that there are three underlying components of predictive analytics [1]:
- Data: historical data of what is being predicted
- Statistical model: the model applied to predict outcomes
- Assumptions: what is assumed to be true
One class of predictive analytics techniques, machine learning, can be used as the statistical model for prediction of future outcomes. Machine learning is a subfield of computer science and statistics that provides mechanisms for systems to learn in a generic way, without being explicitly programmed to learn about a particular field. Machine learning algorithms build a model based on data inputs and use that model to make predictions and interpret new data. If done properly, the models developed by machine learning techniques are far more accurate than traditional forecasting methods while simultaneously having the ability to automatically adjust and improve over time.
Forecast Improvements with Machine Learning
To demonstrate the improvements of machine learning techniques, researchers often use the “Naïve” approach as a benchmark against which more sophisticated models can be compared. Naïve forecasts estimate values by using the previous period’s actuals as the current period’s forecast, without adjusting them or attempting to establish causal factors. Basically, the Naïve approach is an educated guess.
Mean Absolute Error (MAE), a standard measure of forecast error, can be used to evaluate the accuracy of a prediction model. MAE measures how close forecasts or predictions are to the actual outcomes by summing the absolute error (difference between the prediction and the actual observation) divided by the number of observations. Perfect forecasting would result in an MAE of 0, indicating that every prediction was correct. In practice the goal of forecasting models is to minimize this measure.
Several studies have compared different forecasting methods to understand which forecasts result in the most accurate predictions. The Microsoft analytics team found that when helping a forecasting firm predict monthly auto sales, predictive analytics reduced the MAE by 40% compared to traditional time-series models (similar to the Naïve approach) [2]. A reduction in MAE ensures that the forecasting firm will have more accurate forecasts than previously obtained, making it possible to improve efficiency by having the right inventory at the right places. Another study comparing the performance of forecasting methods was able to demonstrate that advanced machine learning techniques including neural networks, recurrent neural networks, and support vector machines, have the best forecasting performance when applied in supply chains [3]. This study applied traditional (e.g. Naïve approach) and advanced machine learning techniques to forecast manufacturer’s orders and production orders and found that advanced learning techniques reduced MAE between 16.2% and 37.6%, with recurrent neural networks performing the best followed by support vector machines. A thorough study of traditional and machine learning forecasting techniques concluded that traditional models (including the Naïve approach) are generally inferior, and are more likely to produce inaccurate predictions compared to advanced techniques such as machine learning [4].
Applying Predictive Analytics to Ammunition Management
Using predictive analytics, CSI’s systems incorporate the same operational factors and other relevant information it used for initial planning to recommend quantities for upcoming resupply requests. Additionally, they are able to incorporate data feeds directly from the battlefield and all echelons of command and react to unanticipated circumstances and trends – operational or situational – alerting users and recommending changes to ammunition allocations. Machine learning techniques provide software systems with the ability to learn from actual consumption in specific operations and operational conditions to provide adaptive demand estimation throughout the operation.
The predictive analytics and machine learning techniques at the technical core of CSI’s systems are advance, but the results are well-documented. These techniques have reduced forecasting error by as much as 40%. Improving forecasting error by even 1% can have significant impacts on operational efficiency and costs within an organization. Considering the advanced analytical techniques used in CSI’s systems, a conservative estimate for improvement of resulting ammunition consumption predictions would be in the range of 10 to 15 percent*.
While CSI has applied these techniques to ammunition management, these techniques are not domain specific; they can actually be applied to any domain that requires forecasting, including warehouse operations, workload prediction, and power management, resulting in improved operations and reduced costs.
References
- Davenport, Thomas H. “A Predictive Analytics Primer.” Harvard Business Review. 02 Sept. 2014. Web. 16 Aug. 2016.
- LaRiviere, J., McAfee, P., Rao, J., Narayanan, V.K., and W. Sun. “Where Predictive Analytics Is Having the Biggest Impact” Harvard Business Review. 25 May 2016. Web. 16 Aug. 2016.
- Carbonneau, R., Laframboise, K., and R. Vahidov, Application of machine learning techniques for supply chain demand forecasting, European Journal of Operational Research, Volume 184, Issue 3, 1 February 2008, Pages 1140-1154.
- Pritzsche, Uwe. Benchmarking of Classical and Machine-Learning Algorithms (with Special Emphasis on Bagging and Boosting Approaches) for Time Series Forecasting. Thesis. Ludwig-Maximilians-Universität München, 2015. 1-109. Print.
For more information about how Cougaar Software, Inc. builds solutions using predictive analytics for the military, please visit https://www.cougaarsoftware.com/supported-industries/military-dod/.
*The conservative estimate for prediction improvement is based on industry studies comparing the standard measure of forecast errors, mean absolute error (MAE). (Advanced techniques, including machine learning reduced MAE between 16.2% and 37.6%.). CSI’s systems continually update its forecast models using machine learning. These systems evaluate actual consumption under specific battlespace conditions against previous estimates, continually refining the norms based on expanding knowledge of activities and their consumption rates.
About the Author: Angela Garza is an Operations Research Engineer at Cougaar Software, Inc., who specializes in optimization, simulation, and analytics.