In modern archaeological research, some portion of the research will take place in a Geographic Information System or GIS. Whether it’s fully integrated into the project both documenting and directing research, or utilized solely for the archiving of mapped data, there is a GIS. The seeming precision of GIS mapped data, coupled with the ability to calculate complex spatial statistics for research and guidance results in a very friendly environment for black box fallacies. This is the problem of closed source, undocumented analytic software tools being used to conduct research. When a black box is used uncritically, without thought for research design and control the results of the research conducted with the black box can become suspect.
Take for example, the issue of analyzing site inter-visibility with a viewshed analysis. This is a common use of this GIS tool in landscape archaeology. By tracing lines of sight the software is able to create a binary map, where a pixel will be recorded as either visible or not visible. Commercial software package ArcMap from ESRI has one set of algorithms for this purpose. The most popular open source GIS, Qgis has a different set of very similar algorithms. The result is that with the same input parameters and data Qgis and ArcMap can produce slightly different binary maps of visibility.
Because Qgis is open source, an interested party could very easily examine the source code responsible for the resultant visibility map. None of this is possible with a black box, whether it be for viewshed analysis, point cloud processing or interpolating data. It’s impossible to view the source and impossible to modify or know exactly why you get the results you get. But because the results are from a computer they can very often be treated uncritically by researchers despite the lack of information on how exactly they arrive at a given result.
This last point is actually a larger problem within the use of GIS and archaeology. As an example, I relay the findings of Thomas J. Loebel of St. Xavier University. Loebel examined the distribution of paleoindian stone tools across one county in the midwest US. Loebel found that the distribution of tools across space was more highly correlated with modern human activities than any environmental or ecological features, which he interpreted as resulting from a higher incidence of ground disturbing activities. The result is the same, an uncritically used tool can result in seemingly precise results which have absolutely no bearing on the physical world.
There is no real uniform solution to the black box problem with GIS, however some of the resultant questions can be mitigated by clear and explicit explanation of what software, what version of the software and what settings are used. Each of these factors can have an impact on on the final result, which is then used for analysis and interpretation. Documenting these factors is a critical baseline step for mitigating the impacts of software on research programs. Further steps would include documenting and maintaining precise descriptions of all steps in a processing chain as well as critical research done on comparable closed and open source versions of a given set of tools. It’s especially critical when interpolating or extrapolating data to recognize the importance of source data. The saying garabage in garbage out has very real relevance here. If the input data is a poor representation of the phenomena being modeled, the model will also be a poor representation of the phenomena.
References