Anticipate impacts

Anticipating impacts of an AI system involves reflecting on and assessing the potential short-term and long-term effects the system may have on impacted individuals and on affected communities and social groups, more broadly.

Why is this kind of anticipatory reflection important? Its purpose is to safeguard the sustainability of AI projects across the entire project lifecycle instead of taking an approach of dealing with issues as they appear. There is no guarantee that a team will be able to anticipate all potential impacts, but dealing with the most relevant ones before they become a problem ensures more sustainable systems overall (it is also a much more efficient use of resources over time).

How does one ensure that the activities and outputs of the AI system are socially and environmentally sustainable? Project team members must proceed with a continuous responsiveness to the real-world impacts that their system could have.

The way to translate into practice as we have seen, is through concerted and stakeholder-involving exploration of the possible adverse and beneficial effects that could otherwise remain hidden from view if deliberate and structured processes for anticipating downstream impacts were not in place.

Attending to sustainability, along these lines, also entails the iterative re-visitation and re-evaluation of impact assessments. To be sure, in its general usage, the word “sustainability” refers to the maintenance of and care for an object or endeavour over time. In the context of AI, this implies that building sustainability into a project is not a “one-off” affair.

Rather, carrying out an initial impact assessment at the inception of a project is only a first, albeit critical, step in a much longer, end-to-end process of responsive re-evaluation and re-assessment. Such an iterative approach ensures that continuous attention is payed both to the dynamic and changing character of the project lifecycle and to the shifting conditions of the real-world environments in which studies are embedded.

Methodical impact evaluation should involve an initial adoption of normative criteria that function as metrics for scoping and assessing the possible harms and benefits of the research and its outputs. Taking GPAI’s “12 Principles and Priorities of Responsible Data Innovation” as an example, relevant impact assessment questions could include:

• How, if at all, could our research and its outputs impact each of the following twelve principles and priorities as they relate to all affected stakeholders, especially those who are vulnerable, marginalised, or historically discriminated against? (Affected stakeholders include research subjects and participants, subjects of data collected for or used in the study, researchers, and all other impacted people and social groups.)

12 Principles and Priorities of Responsible Data Innovation

- Respect for and protection of human dignity
- Interconnectivity, solidarity, and intergenerational reciprocity
- Environmental flourishing, sustainability, and the rights of the biosphere
- Protection of human freedom and autonomy
- Prevention of harm and protection of the right to life and physical, psychological, and moral integrity
- Non-discrimination, fairness, and equality
- Rights of Indigenous peoples and Indigenous data sovereignty 
- Data protection and the right to respect of private and family life
- Economic and social rights
- Accountability and effective remedy
- Democracy
- Rule of law

• How could our research and its outputs advance each of these twelve principles and priorities or hinder their realisation?

• Are there particular stakeholder groups who could disproportionately enjoy the benefits of the research and its outputs, or suffer from the potential harms they generate, as these harms and benefits relate to each of the twelve principles and priorities?

• If things go wrong in our research or if its outputs (especially tools produced or capacities enabled) are used out-of-the-scope of their intended purpose and function, what harms could be done to stakeholders in relation to each of the twelve principles and priorities?

It is important to note here that stakeholder involvement in the impact assessment process can be a critical safeguard against evaluative blind spots and omissions.

Methodical impact evaluation should also involve an assessment of the severity of potential adverse impacts. This brings clarity to the prioritisation of impact mitigation actions by allowing the severity levels of potential negative effects to be differentiated, elucidated, and refined. As explained in the United Nations Guiding Principles on Business and Human Rights (UNGP), assessing the severity of potential negative impacts on fundamental rights and freedoms involves consideration of their scale, scope, and remediability, where scale is defined as “the gravity or seriousness of the impact,” scope as “how widespread the impact is, or the numbers of people impacted,” and remediability, as the “ability to restore those affected to a situation at least the same as, or equivalent to, their situation before the impact” (UNGP, 2011, Principle 14).

One notable challenge faced by researchers who are assessing the severity of potential adverse impacts is identifying cumulative or aggregate downstream impacts, which can be much more difficult than identifying harms directly or proximately caused by a project.

Discerning these impacts may require additional research and consultation with domain experts and other relevant stakeholders. This difficulty results from the fact that cumulative impacts are often incremental and more difficult to perceive, and they frequently involve complex contexts of multiple actors or projects operating in the same area or sector or affecting the same populations.[@gotzmann2020] Some “big picture” questions to reflect on when assessing cumulative or aggregate impacts include:

• Could the project contribute to wider scale adverse impacts when its deployment is coordinated with (or occurs in tandem with) other projects or innovation activity that serve similar functions or purposes? For example, if the impacts of a project that aims to discover an effective method of behavioural nudging at scale are considered in combination with the proliferation of many other similar projects or computational systems in a given sector, concerns about wider cumulative effects like mass manipulation, objectification, and infringement on autonomy and human dignity become relevant.

• Could the project replicate, reinforce, or augment socio-historically entrenched legacy harms that create knock-on effects in impacted individuals and groups? For example, if a project analyses sensitive personal information contained in databases scraped from social media websites without gaining the proper consent of research subject in accordance with their reasonable expectations, it could add to the legacy harms of companies that have used data recklessly and eroded public trust regarding the respect of privacy and data protection rights in the digital sphere. This can create wider chilling effects on elements of open communication, information sharing, and interpersonal connection that are essential components for the sustainability of democratic forms of life.

• Could the production and use of the system be understood to contribute to wider aggregate adverse impacts on the biosphere and on planetary health when its deployment is considered in combination with other systems that may have similar environmental impacts? For example, a project that involves moderate levels of energy consumption in model training or data storage may be seen to contribute to significant environmental impact when considered alongside the energy consumption of similar projects across research ecosystems.

Once impacts have been evaluated and the severity of any potential harms assessed, impact prevention and mitigation planning should commence. Diligent impact mitigation planning begins with a scoping and prioritization stage. Team members (and engaged stakeholders, where appropriate) should go through all the identified potential adverse impacts and map out the interrelations and interdependencies between them as well as surrounding social factors (such as contextually specific stakeholder vulnerabilities and precariousness) that could make impact mitigation more challenging. Where prioritization of prevention and mitigation actions is necessary (for instance, where delays in addressing a potential harm could reduce its remediability), decision-making should be steered by the relative severity of the impacts under consideration. As a general rule, while impact prevention and mitigation planning may involve prioritization of actions, all potential adverse impacts must be addressed. When potential adverse impacts have been mapped out and organised, and mitigation actions have been considered, the research team (and engaged stakeholders, where appropriate) should begin co-designing an impact mitigation plan (IMP). The IMP will become the part of your transparent reporting methodology that specifies the actions and processes needed to address the adverse impacts which have been identified and that assigns responsibility for the completions of these tasks and processes. As such, the IMP will serve a crucial documenting function.

Establishment of protocols for re-visitation and re-evaluation of the research impact assessment:

Impact assessments must pay continuous attention both to the dynamic and changing character of the project lifecycle and to the shifting conditions of the real-world environments in which research practices, results, and outputs are embedded. There are two sets of factors that should inform when and how often initial impact assessments are re-visited to ensure that they remain adequately responsive to factors that could present new potential harms or significantly influence impacts that have been previously identified:

1. Lifecycle and production factors: Choices made at any point along the workflow may affect the veracity of prior impact assessments—leading to a need for re-assessment, reconsideration, and amendment. For instance, design choices could be made that were not anticipated in the initial impact assessment (such choices might include adjusting the variables that are included in the model, choosing more complex algorithms, or grouping variables in ways that may impact specific groups). These changes may influence how a computational model performs, how it is explained, or how it impacts affected individuals and groups. Processes are also iterative and frequently bi-directional, and this often results in the need for revision and update. For these reasons, impact assessments must remain agile, attentive to change, and at-the-ready to evaluatively move back and forth across the decision-making pipeline as downstream actions affect upstream choices and evaluations.

2. Environmental factors: Changes in project-relevant social, regulatory, policy or legal environments (occurring during the time in which the research is taking place) may have a bearing on how well the resulting computational model works and on how the research outputs impact affected individuals and groups. Likewise, domain-level reforms, policy changes, or changes in data recording methods may take place in the population of concern in ways that affect whether the data used to train the model accurately portrays phenomena, populations, or related factors in an accurate manner. In the same vein, cultural or behavioral shifts may occur within affected populations that alter the underlying data distribution and hamper the predictive and explanatory efficacy of a model, which has been trained on data collected prior to such shifts. All of these alterations of environmental conditions can have a significant effect on how research practices, outputs, and results impact affected individual and communities.