|
|
Boletín de Dinámica de Sistemas
System Dynamics Model Calibration
Giuseppe Noce
nocegius@yahoo.it
When a model is well developed, well-structured and simulates properly, it becomes necessary to build the confidence in this model. Calibration is a tool to achieve that. Calibration is a strong test by which the confidence that model structure reproduces the observed behaviour is reinforced. By calibration, the difference between the model output and the real data are minimizing. The first step is the identification of the key model parameters (e.g. growth rate, contact rate for a disease, delays, etc) and assign them an initial range. Sensitivity analysis could be useful to identify which parameters affect the model output. The second step, to a well-done calibration, is the selection of the correct real data for the variables which the model tries to replicate.
Vensim has an automatic procedure to calibrate model but this capability is only for the Vensim DSS, the top version of Vensim . The other versions do not have this full capability … and the modeler has to manually calibrate the model according the expertise and experience of the modeler. Hand calibration is a visual check. The real data are loaded into the model, then model is simulated.
Vensim generates a simulated dataset … and the modeler looks at how close the curve of real data and simulation data is close and near, otherwise the parameters are adjusted, such as:
In blue the model output, in red the real data (baseline). If the fraction infected from contact is 0.4 the real data do not fit with the model output
While if the fraction infected from contact is 0.07943 the real data better fit with the model output, then the prior value. 
It is easy if the model is simple and with poor data, parameters, variables. 
But if does the model reproduce a more complex problem? There is the solution.
Manually the parameters are adjusted and each simulation files are exported as CVs or TAB or XLS file and use with statistic software (Python, R, Spreadsheet) to calculate and minimize the error between real data and simulated data.
The Fundamental error measure is the Sum of Squared Errors (SSE). This method is simple and intuitive because measures the total amount of error directly and is smooth and differentiable is fundamental for calibration, and represents the basis for MSE/RMSE, but increases with number of observations, sensitive to outliers, units are squared.
Other used error measures are: 
Where yi are real or observed data and y^I represent the model output. 
Calibration has caveats. One is the overfitting. The model fits very well the real or historical data but does not perform well for the future scenario. Another is the model fits but the structure does not represent the real system which it should simulate.
In definitive, calibration of a SD Model is not to find the best fitting between the observed or real data and the model output. To have a well-done SD model, the modeler has to find the best fitting parameter values and simultaneously the best capture of the observed historical behaviour. Hand calibration is an art which tests the skill and patience of the modeler.
| |
|||
|
|||
| |
|||
|
|||
| . | |||
