Debunking the myths on visitor evaluations and testing – Lizzy Bakker tells us why NorthernLight incorporates visitor evaluations into its design process

NorthernLight has been designing transformative experiences for more than 15 years: exhibitions, educational programmes and apps. From the very beginning, our most important starting point has been to design not only for but also with the visitors and users of our experiences.

Imagine that you and your team are designing a new attraction. It’s supposed to be completely innovative, bring all kinds of new insights to the visitors, let them have a fantastic outing they will want to repeat and increase their consumption in the gift shops. You’ve completed your concept, you’ve got some preliminary ideas on activities and you’ve carefully formulated how all this planning is going to meet the goals of your attraction. What’s next? You can either move on to the next design phase, or you could take a moment to test out your ideas with a group of potential visitors. We believe that at this point in the design process it is best to implement the second option and incorporate visitor evaluations to create better designs, better exhibitions and better experiences.

FORMATIVE EVALUATIONS
Although most attraction professionals agree that visitor evaluation is important for the optimisation of a design, formative evaluation is often still one aspect of the design process that’s easily omitted. In some cases, the needs of visitors are simply overlooked in the development team’s excitement for a specific topic or design solution. Furthermore, a team might think that a test group won’t understand the still conceptual design. But more often than not, the decision to not share designs with a test group is of a pragmatic nature. Visitor evaluations cost time and money – two resources that are always scarce at a design company.

But fear not. Evaluating your design with future visitors can be less effort than you might think. We’ve developed our own philosophy on how and when to incorporate visitor tests during our design process, including paper prototyping, concept testing and usability testing of the experiences, exhibits and apps we design. Without spending huge budgets, the acquired insights are of major value.

DEBUNKING MYTHS
It’s time to debunk four myths that make visitor evaluation seem like an impossible mountain to climb.

Abstract design
If it’s difficult to convey the meanings and purposes of a design to a client, how could potential visitors – who have never looked at design renderings or floor plans – evaluate exhibit designs in the making or report on the usability or level of intuition of an app by looking at printed non-responsive screenshots?

We’ve started to take the abstract to the public. Inspired by evaluation methods from marketing, we’ve incorporated concept testing into our design process. During these tests, we show artist impressions of our concept design to small focus groups and test their response to our idea. The tested material can include preliminary floor plans, stills from the conceptual design and even preliminary ideas in text. All quite abstract and undetailed, but still giving a picture of where the exhibit is heading.

In our experience, test groups – regardless of age – don’t experience problems understanding these concept designs. However, it is key to first provide the test groups with a clear overview of the whole experience by presenting a full walk-through of every part. Subsequently discuss each part separately, leave room for questions and keep an open conversation so the test groups can give valuable comments and suggestions.

In a concept test NorthernLight carried out for the Philips Museum in the Netherlands, we tested several interactive projections that show the visitor how their daily contexts link to Philips’ product innovations. The test groups showed a clear understanding of the designs.

“The test panel consisting of people from the target audience gave us some precious feedback from their perspective,” explained Olga Coolen, director of the Philips Museum. “Children, for example, came up with the idea of adding game elements to the interactive screens. While we were being very serious on the content, they opened our eyes to include fun elements. An elderly couple told us that the level of interactiveness was acceptable for them and even fun. So they took away our assumption that it might be too ‘futuristic’ for older people.”

Abstract designs are no issue for test groups; they can think alongside from their own perspective and this can assist you in the optimisation of your design.

Sample size
When talking about visitor evaluation, one might think of surveys filled in by countless numbers of people and elaborate statistical analyses. From a research perspective, large sample sizes are needed to formulate generalisations from your test results. However, when you consider visitor evaluation as a moment of reflection, consultation or ideation with your target group, a few focus groups can give you sufficient ideas to improve the design for the next phase. Some of these insights seem so simple, but are easily overlooked. In a concept design test with five small focus groups (each comprising two to five children), we asked whether they’d keep the attraction’s blue RFID wristband and show it off to their friends. “I would, if I could get it in pink,” a child replied, with agreeing nods from the others. There you go – an idea as simple as letting children choose the colour of their RFID wristband can potentially lead to increased exposure.

After each session with each focus group, a lively discussion arose between the design team and the client; everyone was full of new ideas for the experience.

Gathering data
When you’re interviewing your test group or discussing ideas with them, you have to listen emphatically and analytically at the same time. You need to understand what they’re saying, think of the right probing questions to get them to share more of their underlying values, while connecting the dots and auditing their answers with the test goals in mind. It sounds like you would need years of training before mastering these skills.

Training and experience will no doubt improve your moderator skills. But even without them, the right preparation for a test can point you in the right direction. Ensure that your test goals are explicit beforehand, and formulate possible indicators to focus on during the discussion. If you want to know if the visitors will learn anything new from the exhibit, be aware of sentences like “I didn’t know that!”. Have a structure ready beforehand to analyse people’s replies and formulate questions in a script or protocol that lead to the desired insights.

Getting the information you’re interested in can be even easier when carrying out (paper) prototyping tests on usability. Instead of discussing a 3D view of an exhibit with a test group, you can try out a prototype of the exhibit and pinpoint possible obstacles. When using this method for apps, you can use a working model, non-responsive screens or prints of the planned screens of the app. Any of these will allow the user to walk through the interaction to test its user-friendliness and intuitiveness and then look for faults or obstacles in the use of the app.

In collaboration with Museum Volkenkunde (National Museum of Ethnology) in the Netherlands, we’ve conducted usability tests with both a paper and a real prototype of the new Museum Explorer app. By letting test groups perform specific tasks with the app – finding a certain object in an exhibit, finding information within a video, navigating with enclosed maps – and letting them think aloud while using the app, we directly obtained the information we were interested in.

We observed that people could find interesting artefacts using the iBeacon-linked information in the app and could use the app to navigate between exhibit spaces. It even provided practical insights, such as their preference for either a headphone or earplugs. If you think carefully about the things you want to test and choose the appropriate tasks for that goal, you won’t have much trouble getting the right information.

Effort versus outcome
Evaluations do cost time and money but the expenses can be limited and a lot can be achieved by carrying out simple workshops, brief discussions or short surveys. It is not per se necessary to rent a room with surveillance mirrors, hire multiple moderators or observers and spend hours transcribing audio tracks and analysing notes. The abundance of insights gained from simple methods is well worth the effort and expenses.

NorthernLight held workshops with both local school children and stakeholders to provide feedback on a new exhibition in a shopping mall on the Norwegian island of Sotra Kystby, for the country’s VilVite science centre. We kept it simple: we gave them cards with pictures of exhibits and short descriptions. We let them choose their favourite exhibits, suggest improvements for some of them and make sketches of new exhibit ideas. As an example, it turned out that the pride the workshop cohort took in being Sotra Kystby islanders was an important theme to include in the new VilVite exhibit.

The target group suggested creating a multimedia flight over the island and its highlights and activities to learn more about the sea or sources of energy used on the island. The direct takeaway from this low-cost evaluation: an improved and expanded exhibit list validated by potential visitors.

DESIGN, TEST, OPTIMISE
Overall, it is clear why designers should include the public in the design process – how to go about it isn’t always quite so clear. But now that we’ve debunked the most common myths, hopefully visitor testing won’t seem like such a big hurdle.

Instead of seeing objections, see opportunities. Talking to potential visitors when designs and ideas aren’t yet set in stone is such a valuable step in the design process that it’s worth trying. From our experience, visitor evaluation doesn’t have to be that difficult. You just need to get started and incorporate it into your system. Make your life easier – let the public tell you what they want to see. ?