Evaluation for understanding meaning and significance

Evaluation aims to assess the nature and qualities of something in a meaningful way. Instruments of social science have difficulty organising multiple forms of expression into categories, but aesthetics is all about assessment and finding meaningful ways to talk about art. The arts sector generally agrees that evaluation is an important part of any publicly funded project; however, very little resources of time and money are dedicated to such assessment. What is more alarming is when evaluation is undertaken, data is collected, but the results are not analysed, fed back to collaborators, publicly disseminated, or applied in future planning. Evaluation practices can encompass important conversations on ‘value’ and ‘values’ and describe otherwise unregistered forms of cultural production.

Towards an integrated approach to the evaluation of community based arts^[1] was a three-year Australian research project examining the range of competing value frameworks – spectres that were haunting the arts – such as health, youth, justice, urban planning involved in these collaborations. Through a series of interviews and case studies, we unpacked the relationship of impact evaluation to the instrumentalisation of the arts when projects focused on outcomes over process. The practice of evaluation was discussed in more than 40 interviews with socially engaged artists from across the country. We quickly discovered evaluation represents multiple things to different people, but two primary forms of assessment were described: (1) follow up reports for funders and (2) self-reflection on practice. A long list of tools like ‘cost benefit analysis’, ‘social return on investment’, and ‘theories of change’ were discussed briefly in the interviews with little understanding of benefit. A number of artists described the process of funding reports for accountability as painstaking. Reporting often happened months after the project was completed when they were unable to connect to participants or contracted artists. The data requested in these reports depended on the priorities of the funding program; however, financial records and audience numbers were always requested. Additional ‘tick boxes’ regarding engagement with ‘communities of interest’ were included without qualification. These reports appeared not to be evaluation of practice at all, but were simply an uncritical reporting mechanism against funding policy aims more specifically.

The second area of discussion unpacked in these artist interviews included the robust efforts made in processes of critical self-reflection on practice. This iterative process of analysis is most essential for bettering practice. Artists and the communities they work in generally can instinctively recognise when their work is successful through creative and dialogic means integrated in practice; however, this knowledge is largely unregistered. We might call these ‘ways of knowing’ that artists hold instinctively in their bodies and memories as ‘tacit’ or ‘embodied knowledge’. As Donald Schön explains, “the situations of practice – the complexity, uncertainty, instability, uniqueness and value conflicts which are increasingly perceived as central to the world of professional practice” (1983; p. 14). As practice becomes more professionalised and funding becomes more competitive, artists are developing new ways to access critical feedback through peer review, training, and practice-led research. Some artists articulated informal, yet embedded forms of evaluation – check-in circles, collaborative feedback loops, and inviting in an ‘outside eye’. Others described how they aim to create transparent and accountable processes in projects to hear the interests of participants.