Exploring the Scope of Using News Articles to Understand Development Patterns of Districts in India

Understanding what factors bring about socio-economic development may often suffer from the streetlight effect, of analyzing the effect of only those variables that have been measured and are therefore available for analysis. How do we check whether all worthwhile variables have been instrumented and considered when building an econometric development model? We attempt to address this question by building unsupervised learning methods to identify and rank news articles about diverse events occurring in different districts of India, that can provide insights about what may have transpired in the districts. This can help determine whether variables related to these events are indeed available or not to model the development of these districts. We also describe several other applications that emerge from this approach, such as to use news articles to understand why pairs of districts that may have had similar socio-economic indicators approximately ten years back ended up at different levels of development currently, and another application that generates a newsfeed of unusual news articles that do not conform to news articles about typical districts with a similar socio-economic profile. These applications outline the need for qualitative data to augment models based on quantitative data, and are meant to open up research on new ways to mine information from unstructured qualitative data to understand development.

Ensuring Responsible Outcomes from Technology

We attempt to make two arguments in this essay. First, through a case study of a mobile phone based voice-media service we have been running in rural central India for more than six years, we describe several implementation complexities we had to navigate towards realizing our intended vision of bringing social development through technology. Most of these complexities arose in the interface of our technology with society, and we argue that even other technology providers can create similar processes to manage this socio-technological interface and ensure intended outcomes from their technology use. We then build our second argument about how to ensure that the organizations behind both market driven technologies and those technologies that are adopted by the state, pay due attention towards responsibly managing the socio-technological interface of their innovations. We advocate for the technology engineers and researchers who work within these organizations, to take up the responsibility and ensure that their labour leads to making the world a better place especially for the poor and marginalized. We outline possible governance structures that can give more voice to the technology developers to push their organizations towards ensuring that responsible outcomes emerge from their technology. We note that the examples we use to build our arguments are limited to contemporary information and communication technology (ICT) platforms used directly by end-users to share content with one another, and hence our argument may not generalize to other ICTs in a straightforward manner.