SHOW  ME  THE  CHANGE
Complexity and the Art of Evaluation – Reporting Sheet

Leader:    Fay Valcane
fvalcane@museum.vic.gov.au

Participants: Kathryn McCullum    Deb Orb
Stephanie Madeley    Jane Lieman
Viv Benton    Linda Duffy
Greg Bruce

Key Points:

•    Telling stories around climate change in an engaging (immersive way).  Creating experiences that put people in “changed environment”.

•    Prof Sam Hom US Thematic Communication & Tourism Industry Int Physiologist was quoted as a good contact to make.  He will be based at Latrobe Uni, …………?? Oct / Nov 2010.

•    Working in partnerships with key organisations, where the museum can be the neutral trusted authority.  Provides platform to engage with experts in the field under one collective point – and the museum not re-inventing great work being done, eg Deakin Uni and Wheeler Centre lectures.

•    Targeting mums with young kids – museum can provide safe place to bring mums together and equip them with strategies to change behaviours, creating safe spaces for discussions to occur.

•    Work with something that goes to where people are, not wait for them to visit you (at the museum).

•    Museums are places that provide opportunities for brave discussions, provide balance …….?? Sceptic views & science non-sceptic views.

•    Follow up with Monteray Bay Aquarium in UK who are also looking at ways of addressing climate change issues.

•    Use people and their stories to help communicate issues to engage audiences rather than tackling it by a “we are experts approach”.  Sited Jane (surname ?????) a Professor at Flinders University, for eg of great approach on thematic communications.  Example was given where Jane talks about her passion for geology – rocks via taking people on a journey that they are interested (eg their migration history) and finding a creative link to a story on the geology of the area (this sounded very intriguing)!

SHOW  ME  THE  CHANGE

Complexity and the Art of Evaluation – Reporting Sheet

Leader:    Rebecca Petit

Participants: ZandyTibballs    Martin Hausenblas
Fay Valcanis    Darlene Pertland
Jonathan Day    Megan Hughes
Merryl Whyte    Felix Acher
Debbie Megyar    Michael Baranovic
Helga Svendsen    Michelle Lambert
Millicent Burke    Anjali Pal
Bridge Wetherral    Ian Blair
Brian Hardy    Marcia Hewitt
Greg Campbell    Julia Davies
Danielle Kennedy

Key Points:

•    CBSM – use of tools, prompts etc. case studies work, may not go deep enough with values – moving from energy – water – www.cbsm.com

•    Future perfect – green home – way of visioning a utopia

•    YVW behaviour change barrier breakdown process, levels of engagement, water smart – based on social data methodology, 2 yrs of data before 9000 – based on.  Produced data around self reported change

•    Internal and external barriers eg can’t affect my water bill – cost

•    Board – aware of change, energy knowing its usage and cost eg usually swimming pools.  Black balloons campaign marketing not tangible.  Was successful in some visual aspects.

•    People are so different – amount of positive thinking – 3 important parts of it – what will be the outcome of the action for me?  2 – social pressure – is there a group of people around me who care – males, gov, neighbours etc. and 3 – barriers – what are they – money, time etc?   ……………………..????? these barriers there are people who are willing and people who are not willing.  Survey – what are those beliefs?  Eg Townsville example, people will do it for environment etc.  I…………………………………………….????Positive thinking to support this target. (name of the model) Author fishbeinicekajzen – predicting action and change behaviour, the reasoned approach.

•    Need a multi-prong approach eg legislation, cultural norms, beliefs, etc.

•    Household relationship & habits how to effect

•    Transition towns – read the transition handbook by Rob Hopkins.  Connecting communities and building resilience in peak oil, conversations, relationships then transactions.  The sharehood – www.thesharehood.org   CSA – community sustainable agriculture – similar views

•    Princes Hill High School & Lauriston on the school building – solar panels, water tanks etc.  Assi – Australian Sustainable Schools – resource smart is the Victorian Section – SEC – SV-website

•    Jika – Jika – growing greening neighbourhoods house

•    ANU – does network analysis look at how connected the communities are and what model will you use to target group eg aids example on unit address large group and disconnected groups.

•    Transition book self funded from day one – to do your own and have active people – steering committee to be Rachel Lowery – palm oil.  Act for nature campaign – commitment chosen

•    Solar critics – Townsville – magnet and follow – 6 months later

•    Increase your own commitment to remind yourself

Themes
•    Behaviour change models and theories CBSM

•    Community grass roots organisations that work eg transitions towns.  The sharehood etc. examples of demonstrations that work eg Princes Hill high – solar panels on school, raintanks etc – growing greener neighbourhoods etc.

Examples of programs, other councils, sustainable homes, greenhouse ACF etc SLAH – sustainable living at home – evaluation on website.

Topic:    Storytelling as a form of evaluating change behaviour

Key Points:
•    Storytelling can be in narrative or anecdotal process
•    DSE fire have used it as a debrief for recent fire season – same participants had trouble “storytelling” (adapting to change in debrief method)
•    To help use stories as data, sifting process allow for “themes” to come out of process for evaluation
•    Storytelling can be used as a tool and measure in behaviour change
•    Is storytelling really that different from one on one interview or focus groups?
•    How can we place storytelling with statistics?
•    Need to have storytelling throughout project – ongoing evaluation

•    Obama campaign used narrative / conversation
- self         )
- group     )    used by local people to spread
- now        )    questions / stories

•    Need to record failures and successes as part evaluation
•    Storytelling is significant to tellers of their story – individual learning and recognition can come from storytelling
•    Self story / narrative – Terese Riley – Penny Hall – Prof Stuart Hill – purposeful lying ??
•    Narrative analysis -
-    anecdotal
-    used debrief DSE – begin / middle end – challenge, choice, outcome
-    personal stories have / give credibility
-    anecdotal circle
-    digital storytelling, ………………..    ?????  children

•    How is it different from focus group – case study
•    Stories resonate with people
•    Sift stories – the story that resonates the greatest in the group
•    Significant – significant for you? – for the people for the storyteller?
•    Conversation – interview – action tool – Barack Obama / self / grip / now
-    measure of change
-    measure of of tools / behaviour change
-    could develop measures

•    Use community to help
•    Social element
•    Not the only way to evaluate change
•    Start process at beginning
•    Stats v’s story ??
•    Recording failures and successes
•    How can we allow for change / wave of change
•    Evaluation and transference of story experience

SHOW  ME  THE  CHANGE

Complexity and the Art of Evaluation – Reporting Sheet

Leader: Damien Sweeney

Background to Toolbox:

• Four councils involved with their project and Swinburne University is the academic partner

• Current stage – putting tools together, have piloted some already with participating councils

• Phone contact for case studies

• Online forum would be useful

• Have limitations of evaluation technique is useful

• Qualitative is common – interview and qualitative

• Cost benefit analysis is useful for CSIRO examples

• Attribution – were you influenced by something else or this project? (Categorises the other influences)

• Links to online surveys and tips to use that would be useful

• Need to say what evaluation to have when ie. Evaluation plan for duration of project

• City of Ballarat – project on planning and how engaged people are in programs – would be willing to pilot toolbox

• Behaviour Change blog – through conference, could be useful to have this on site for blog

• Survey – difficult to follow up on outcomes. Survey monkey is useful

• Needs to be able to change the toolbox – not have it static, eg, add case studies, reviews of tools etc – make it interactive

• Have volunteer from City of Ballarat willing to pilot the toolbox

• Wicki page – people can add their own information and use as a blog

• The National Social Marketing Centre in the UK is a good example of an online toolbox

• Clear Water – storm water project

SHOW  ME  THE  CHANGE

Complexity and the Art of Evaluation – Reporting Sheet

Leader: Candice Bruce

Key Points:
•    Ask for input – pose a question = ownership – outcome
•    Run perpetual initiates
– normalise behaviour
•    Develop trust
–    Adaptation
–    Solution focus
–    Visuals – “in a perfect world?” Grading from 1 to 10
–    where are you now? What are the good things and the learnings? What is one thing that you can do to move forward?
•    Collective Social Learning Process – Professor Valerie Brown – “Leonardo’s Vision”
•    Spacial marking (buy special chalk to mark on carpet) – participants step in and out
•    Structured workshops
–    home work before
–    brain storming
–    objectives -  business plan
–    options
–    time frame
–    post-it notes
•    Action learning (W.I.L)
•    Getting action at the end – pair up – what I’m going to do
– follow up in 2-3 weeks – accountable
•    Never make a statement when you can ask a question
•    Two ears, one mouth – listening – hear how you are being heard.
•    Telling stories about our own journey
–    photos – pick a picture you identify with and say why
–    share with 1 other person
–    then share with the table
•    Disposable camera – photos of what are their favourite areas or their home
•    Shared Resources:
- “Estelo” specialist training supplier Sydney
- Book – Climate Action – Diesendorf
- Edward de Bono – 6 Thinking Hats
- Storms of my grandchildren – Hanson
- The Rational Manager – Kepner and Tregoe
- “Kids Teaching Kids” Aaron and Richard Wood
- MS Project – Microsoft Excel
•    Make time for individual reflection and to do visual stuff
–    Role model deliberate actions
•    Transition handbook – share slides – use art and participant centred
•    Use of PCs – SKYPE – Web Cam – interactive learning solutions
•    Conversation Tech World Café – small group, slightly different question
•    Avoid Death by PowerPoint – ½ page is read by each – feedback to group
•    Lend out lecture notes eg. Harvard – ask for question out of the lecture (day before) – get student to answer questions – 5 minute lecture
•    Participants to present chapters in book
•    Question – what is in it for them? List main outcomes – collect at beginning, everyone walks out satisfied
•    Appreciative enquiry process
•    Look for criteria – start where they are at
•    Australia Standard on Governance – transition towns: Heart, Head, Hand, grassroots, self-sustaining
•    Favourite warm up – yes, but = blocker, yes, and opens up possibilities

SHOW  ME  THE  CHANGE

Complexity and the Art of Evaluation – Reporting Sheet

Leader: Zandy Tibballs

Participants: 12

• Identify tools to use that will lead to wider uptake.  Tools such as:
• Focus group work – contact primary leaders to help shape the type of information that is needed to be obtained,  > contact secondary group to determine banners and the type of information that they need to have to be willing to change.
• Target specific members / leaders and tune into their radio station to identify and understand the needs of the target audience > identify pathways that the change leaders go down – what made them change, what’s needed to help others change.
• Be adaptive to needs of target group audience – be willing to switch needs – identify “hook” & type of messaging to be used.
• Identify & work with user groups capable of influencing the target audience group.
• Pick the right champions / community leaders to help influence the target market.
• Consider the use of creative evaluation, eg films, media to influence.
• Utilise envisioning tools to help people identify type of future they woud like.
• Make links to accreditators / certificators / cost savings / rewards / recognitors.
• Undertake more independent measurement to highlight benefits and provide proof of effects that the program / products provides.
• Go to where they go eg conferences that they attend and present findings.
• Establish / or link “people” into a reference group where they can share information and network – “create opportunities” for target users to learn from each other.
• Identify champions that the target group would value and respect.
• Use formative evaluation / action research to help determine next steps.
• Utilise simple marketing tools eg

1. 1. peer review articles in relevant magazines;
   2. develop fact sheets adapted from case studies, suitable for different audiences;
   3. include different information for different audiences;  and
   4. explain & promote the benefits / use diagrams.