Evaluation as a negotiation of ‘value’ and ‘values’

Critical discourse regarding aesthetic and social values appears to be growing, but internal debates remain stuck in competing and divisive questions of assessment which lead to a number of challenges in the politics of cultural measurement (MacDowall et al., 2013).

The problem with ‘impact’ – An expectation of impact usually accompanies the designation of art. The immediate effects of art are usually found in the aesthetic experience of the audience in terms of beauty or transcendence when viewing a painting or listening to music. However, social impact is a difficult thing to measure, particularly in the short term. While typical logic frameworks (inputs, aims, indicators, outputs, measures, outcomes) are useful planning tools for management in complex projects, the language of ‘impact’ is not productive. Impact is a violent word – the action of one object coming forcibly into contact with another. In socially engaged art, does this assume the artist will have such a marked effect or influence on a community? Further, such effects are impossible to link as a causal relationship back to art.

Advocates of culture more broadly also have concerns for the measuring of social impact. Arlene Goldbard argues there is a counterproductive mismatch of methods: “This quantifiable approach to value does not serve us, does nothing to advance our campaign for pluralism, participation and equity in cultural and community life” (2015, p. 4). John Holden also fears this preoccupation masks what we truly value: “The attempt to make the effects of culture transparent and manageable, in order to support it effectively, has somehow obscured the true nature of the activities and experiences themselves” (2004, p. 14).

In praise of complexity and trust – Collaborative artistic expressions are born out of a long exchange of ideas, the building of trust, and through the start to an understanding the cultural complexity that makes up any context. This entails a preliminary phase of indeterminacy when collaboration is first underway. The myriad of different ways in which art can function and how it unfolds over time is complex, making it difficult to determine measures without predetermining outcomes. Composer John Cage describes this element of uncertainty in his collaborative electronic music scores. He believes indeterminacy brings about a situation in which things would happen that are not under the control of the composer (Malt, 2010).

In a short period of time, external evaluators are unable to capture the nuances and shifts in the social terrain even when interviewing stakeholders and perhaps even more so when examining project artifacts and archives. They will read the enthusiasm of what may sometimes result in overly positive reporting. Projects can affect the artist, collaborators and even institutions in quite deep and unexpected ways and we are not always able to articulate this effect in the short term. These experiences are difficult to put into language necessary to write a report. Overly affirmative advocacy reporting may skew results and not leave room for the expression of negative value. Questions may not be asked about what could be done differently and if they are, participants will most likely not want to give negative feedback as they would like to have the project continue or funding increased. This power relationship is amplified particularly when collaborators are asked to disclose political concerns or intimate feelings.

Critical audiences and multiple stakeholders – When art is comprised of these intimate social interactions and the outcome is a private event, the remaining second-hand accounts, documentation, and debris of the project are typically difficult to assess. It has been argued that socially engaged arts has a closed and uncritical audience, that assessment needs to be undertaken over time, and that one may lose objectivity as they become implicated in it (Bishop, 2012). The primary audience for a project’s outcomes may be firstly those who collaborated in the project. Like Hirschhorn, I argue that useful judgement can come from this ‘non-exclusive Other’ who may hold alternative values of expression, participation, and difference as local criteria for aesthetics distinct from the cultural hierarchies of art institutions (2008).

Socially engaged arts projects come about in two ways. An artist may be hosted by a cultural institution to broaden public engagement or an artist will connect with a community directly regarding collaboration on a social issue. A limitation of the first model is that formal evaluation will include discussion from institutions and critical reflection from artists, but typically lacks engagement with collaborators outside of the contemporary art dialogue. This second model can be situated in a long history of artist and non-arts organisation partnerships. These creative collaborations often involve multiple stakeholders with competing aims and clashing theoretical frameworks. This plurality of voices can confuse the clarity of purpose usually afforded to the singularity of authorship. When evaluating, we must take stock of all of these interests.