This is an oldie I just came across.
Software engineers that got hold of IC’s code were gobsmacked.
Rank amateur stuff, with no verification of the program - mainly because they knew it didn’t work properly anyway, so they just took averages of the runs. Looks like in developing the code, or fixing bugs they tacked stuff on, and had been doing that for years, keeping the program going. It seems not to have had a formal documentation process. So ‘The program’ still contained old idle code from way back, that people could not explain. (You can imagine “oh the guy working on that left…”). It’s hard to imagine doing modelling, varying parameters and the like, using a tool that you have not got proper control of.
Critical decisions around the world were supposedly based on IC’s output, claimed to be part of ‘the science’.
As the first btl says, heads should have rolled for this.
Writing good, scientific code is hard effing work. For what its worth, I do remember other teams trying a bit more successfully to reproduce Fergies magic numbers
However it’s a good argument to force all researchers to open source their code immediately. Especially on such an important issue…
Also, FWIW (not that much really) having spent some time looking at agent based models, I have to say I have little confidence in them out of the gate…
Greetings PP
Yeah I agree code should be checked, but here no-one who mattered was even checking the inputs. I think the real issue revealed by this (the program was said to be ‘unsalvageable’) is that the ‘results’ were for show.
Cheers
