NPHET Models include impossible results = Proof models have not been properly tested and validated

 

The model results charted in NPHET’s letter

https://www.rte.ie/documents/news/2021/06/138789-3fe6b992-45e8-4a1c-b4c2-5e4a6131ad53.pdf ,

include impossible results, therefore NPHETs models are inadequately and/or incompetently tested and validated, and therefore utterly unreliable and unsuitable for use as a decision-making aid for matters of public policy.

 

Allow me to explain:

Figure 3 on Page 12 of the letter graphs the spread of results for 1,000 runs of their "Central 1" estimate.

The only number reported for this estimate is the 187,000 cases forecast by "end September". This forecast comes from the dark purple line in the center of the chart.

I copied the chart into Excel, and overlaid a chart with the vertical and horizontal axes aligned with the NPHET chart. Two benchmark lines with fixed values at 5,000 and 19,000 were used to align the vertical axes of each chart.

I then input numbers to create a line that matched the maximum outcome as charted by NPHET.

(Update 22 July 2021: In discussion elswhere, it has been pointed out that the maximum value in the chart may be due to differing model runs at different points in time. This is correct, there is likely some switching of leading position between several of the maximum model runs. Therefore, the cumulative value of the maximum run is likely to be somewhat below the estimate I derived from the chart. Ho-hum. The manner in which I made this estimate is quite crude. I acknowledge the uncertainties and round down the estimate to the nearest 100,000. The matter of which model run is max at any time is one of the uncertainties and is within the margin of error in my estimate.)

The unit of time of the NPHET chart are not defined. Units of time for models are typical very small, the unit of time for NPHET’s model is likely just a fraction of a day. 

The unit of time for the overlaid chart is Week. The value of the Central 1 Max input is therefore the average for a one week period. The total cases for each week would be 7 times the average. The sum of these from 5 July to 29 Nov is 1,528,100. This is an approximation, subject to some uncertainty, rounding it down to the nearest 100,000 brings it to 1,500,000.

This number, 1,500,000 by end November, should be considered in the context of the following:.

•  Over 3,000,000 have received a first dose already.
• Vaccinations are progressing at a rate of about 200,000 per week,
• There are almost 300,000 confirmed cases to date
• Therefore, today, there are only 1.7 Million uninfected or unvaccinated people in the country.
• It is estimated that there are about 2 un-notified Covid cases for every notified case.
• This would bring the uninfected & unvaccinated down to 1.1 million.
• Meanwhile, vaccinations proceed, by end Nov, everyone who wants to will have been vaccinated.

NPHET’s model obviously takes insufficient or no account of vaccination current and forecast progress.  If NPHET's model produces such an impossible number at the maximum end of its range, how can we have any confidence in the inputs and output of the model?

Bottom Line:

Impossible results in the model output are proof that the model has not been properly tested and validated and therefore its output cannot be considered reliable and it is not suitable for use as a decision-making aid for matters of public policy.