Data is often described as the new oil, but the analogy is limited by the fact that oil loses value when consumed while data does not. The same data can be stored, sold, and used in multiple different ways without the knowledge of its original owner. This brings complex ethical concerns for evaluators in balancing the potential risks to research participants against the generalised benefits and cost-effectiveness of the research.
Some key concerns of ethical issues that arise as a result of digital data collection include – consent, privacy, and confidentiality, ownership and authorship, governance and custodianship, data sharing.These are not the only ethical concerns but are highly relevant to impact evaluation research using digital data. This demands that evaluators prioritize these concerns and respond accordingly at different levels.
Any research, evaluation, and monitoring activity should:
- Seek to maximise benefit and minimize harm
- Respect people’s rights and dignity
- Act with honesty, competence, and accountability
- Deliver work of integrity and merit
At 4th Wheel, we constantly aim to align with these principles in all our activities at each stage of the project through the data’s supply chain by asking some difficult questions. This helps us be proactive on ethical issues in digital data collection.
- Is the research useful, necessary, and feasible?
Given the constraints of time and availability of resources and participants, we want to make sure that there is more value for money for potential beneficiaries.
We do a thorough review of our previous evaluations of similar programmes and other evaluations in the space to make sure that we do not repeat but use appropriate research methods to draw relevant impact inferences for the programmes. This helps us be more precise in collecting relevant data for the programme over digital platforms.
We spend sufficient time initially during the inception of the project to make sure that the expectations of the research are reasonable given the resources and time frame available. We also want to make sure that there is sufficient funding allocated to disseminating the research findings to different stakeholder groups, including beneficiaries in a manner that is accessible to them. This contributes to transparency in data usage.
- Is the research design fit for purpose and appropriate to context?
With participant observation, rapport building with stakeholders (observing local customs and norms), and other sorts of inductive inquiry (including in-situ snowball sampling of interviewees) becoming almost impossible, the struggle is going to be real for evaluators to develop a rich and contextualized perspective of the different programmes.
We, at 4th Wheel, with our decade-long expertise in evaluation research design, make sure that our design evolves and adapts accordingly and is developed collaboratively with the different stakeholders of the programme. This helps us account for the local context (social, cultural, religious values, and beliefs) of the programme.
Sampling bias is also a huge concern with weaker sections of the society in the context of several development programmes not having access to quality Information and Communications Technology (ICT). There is a high possibility that they may not even show up in our sampling frame. We make sure that there is representative participation from different sections of the society and if not, we explicitly communicate the limitations of the research beforehand to our clients.
- Is the participation based on informed consent?
We, at 4th Wheel, make sure that respondents know that their participation in the research is voluntary and know how their data is going to be used in the future. Very often, terms like digital security and data protection may not be understood by the participants in development programme evaluations and we make sure that our enumerators are trained to explain these concepts in an accessible language.
We also make sure that any data about individuals specifically, like names, are removed by providing the participants with unique IDs. This is our first and foremost step during data analysis for any of our projects.
We make sure that our enumerators are trained to build rapport over Computer Assisted Telephone Interviewing (CATI) with the help of our digital data collection software like Survey CTO. The software also enables us to encrypt information like telephone numbers to prevent its misuse but still be able to interview the participants over the phone.
- Are the roles and responsibilities for data authorship and governance clearly defined?
The management, organisation, access, and preservation of digital data are all vital to research integrity and represent great challenges of the information age.
We, at 4th Wheel, ensure that we clearly state the data authorship and governance responsibility at different stages of the project in all our project documents. We clearly define who has access to the data, who is responsible for providing the metadata, who is responsible for long-term maintenance of data, and who is responsible for disposal of data at the end of its cycle.
We remain aware that the field of digital data collection is continuously evolving with its different pros and cons and there is no “one size fits all” solution to the problem. While our research methods may change and adapt to the demanding situations, 4th Wheel remains committed to maintaining high ethical standards when it comes to fulfilling our vision and mission in improving the way social programmes are conceptualized, implemented, monitored, and evaluated.
Works cited
Baalen, Sebastian Van. “‘Google Wants to Know Your Location’: The Ethical Challenges of Fieldwork in the Digital Age.” Research Ethics, vol. 14, no. 4, 2018, pp. 1–17., doi:10.1177/1747016117750312.
“Because Your Data Is Worth It.” SurveyCTO, surveycto.com/.
Clark, K. Duckham, M. Guillemin, M. Hunter, A. McVernon, J. O’Keefe, C. Pitkin, C. Prawer, S. Sinnott, R. Warr, D. Waycott, J. Guidelines for the Ethical use of Digital Data in Human Research. The University of Melbourne, 2015. http://ethics.iit.edu/codes/Ethical-Use-of-Digital-Data.pdf.
Hand, David J. “Aspects of Data Ethics in a Changing World: Where Are We Now?” Big Data, vol. 6, no. 3, 2018, pp. 176–190., doi:10.1089/big.2018.0083.
“Let There Be Change.” Accenture, www.accenture.com/us-en
Thorley, Lisa, and Emma Henrion. “DFID ethical guidance for research, evaluation and monitoring activities”. DFID, May 2019, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/838106/DFID-Ethics-Guidance-Oct2019.pdf
