Analysing Survey Data Training

Sally Muir

Image shared under a creative commons license by Sally Muir.

Today I attended the second of Tabetha Newman’s workshops on Survey methodology – Analysing Survey Data. The aim of the session was to provide attendees with a greater understanding of analysis and to prove that it is not as dry a subject as many textbooks make it seem. My thoughts on the previous session – Designing and Running Successful Online Surveys – can be found here.

The workshop was broken up into two sections:
The Theory – focusing on managing data, checking for validity and an introduction to descriptive statistics and statistical analysis.

The Practice – looking at some practical examples of statistical analysis in the context of survey data and how these might be reported.

Tabetha distinguished the two sections of the days as Statistical literacy (understanding data is not just a mathematic skill – to critically analyse and evaluate we need a kind of literacy) and Statistical calculation (the more computational, testing side of statistics).

The Theory
This portion of the workshop been with a consideration of the analyst’s mindset when beginning data analysis – are they undertaking exploratory data analysis, looking to see what patterns emerge, or are they hypothesis testing, looking for specific answers. From the definitions given, I would argue that the DigiLit project will be approaching its data with a mixture of these mindsets – we have variables that we are interested in, but no defined hypotheses related to them.

As with the previous workshop, Tabetha gave us a handy acronym for handling our raw data:

Collate your data – download if online, input if handwritten
Outliers – look for them, unusual data, often very high or low values
Data Management – best practice for storing and organising
Error Checking – could also include input of symbols or zeros that will affect data analysis tools
Record every edit – the importance of keeping track of changes during coding

We then looked at the key questions to ask yourself in order to consider the validity of your data and Tabetha provided a short but effective introduction to the concepts of data types, descriptive statistics and statistical analysis in the context of survey data.

The Practice
After a much needed tea and biscuit break, we began looking at some examples of the kinds of data you might collect and the tests you might apply to them. We looked at likert scale quests data and also nominal (categories with no scale) data.

The most useful part of the session for me personally, was looking at different ways of presenting and reporting data. As a key element of the DigiLit Project lies in the summary reports which will be issued to schools, it was useful to get some tips on how to make these simple and effective in conveying the message of the data. Tabetha’s main points for presenting the data of a survey question are:

Question options – showing what was available
Overall average
Summary table data- including any subgroups (e.g. Faculty)
Average per subgroup
Frequency histogram per subgroup
Results of stats analysis, plus a sentence or two in English explaining it!

We also looked at ways to represent data from free text questions and Tabetha introduced the idea of using info-graphics to represent survey data. Whilst this would not be of sufficient detail for our school level summary data – it may be interesting way to externally share the overarching city data we collect.

Tabetha’s key message throughout the day was that it’s important to get away from the numbers and remind yourself of the broader question (and context of the data).

Overall the session was incredibly useful (and well worth the journey)! Whilst I have studied data analysis in the past, I often find textbooks on the subject are dry and far too wordy – almost as if they are trying to make everyone believe that analysis is something to fear!  Tabetha’s session was an easy to follow, easy to digest refresher for me and it was great to have the opportunity to think about analysis specifically in the context of surveys as I haven’t worked with qualitative data in this way in the past.

And I’m almost surprised to be writing this – but now I’m actually a little excited about getting to work with our data later in the project – you can quote me on that Tabetha 🙂

Advertisements

Electronic Books Workshop

paper bookPaper Book – Planning our iBooks

Last Wednesday, we held our second Learning and Innovation Group workshop. The Learning and Innovation group is the BSF ICT group that looks at the practical uses of technology to support creative, engaging and effective learning, teaching and school community development. The sessions follow a train-a-trainer approach, holding high quality workshops for staff who can then disseminate skills and information across their schools.

Following the success of our Developing Enquiry using iPads workshop, we invited Steve Bunce, Apple Distinguished Educator and experienced training provider, back to introduce us to a new topic – Electronic Books. The workshop was held in one of De Montfort University’s Mac suites.

The workshop was a great opportunity for school staff to experiment with some new tools, with Steve’s expertise on hand to guide them through the day. We discussed scenarios in which the applications covered could be used to support teaching and learning. The session attracted a great mix of staff from 8 Leicester secondary schools, including technical, teaching and Learning Resource Centre staff.

We began the session by making folded Paper Books – using Story Cubes for inspiration, we created our own short stories and planned them out on simple folded paper booklets (see image above). Steve highlighted how this could be used to help learners prepare and plan out their work before moving on to the real thing – like a story board. This draft was then used to structure our work with Book Creator iPad app.

The Book Creator App is a simple app for creating iBooks from an iPad – it is easy to use and has fairly basic features, making it beginner friendly and a great way to introduce the concept of creating your own iBook. We had a number of first time iPad users in the session and they found the application easy to use. You can add photos, text or sound and then adjust these, using the ‘Inspector’ feature, to get your desired effect. It was noted that the addition of sound could be used for learners who are less confident in their written skills, or who have a language barrier.

Staff discussed how this could be used in a classroom context and felt that it would be an interesting way for learners to summarise a topic of work – to present their learning to others.

book creator screenshotBook Creator – iBook Output

After exporting our creations to iBooks – Steve covered some of the in-built features of the iBooks application and looked at how these might support teaching and learning. For example:

  • Text can be highlighted – offering opportunities for analysis of a passage of writing (highlighting verbs, nouns, adjectives in different colours)
  • Notes can be added – which could be used for revision, adding explanations in their own words, etc.
  • In-built dictionary – can be used to look up unfamiliar words, this also means that learners are not pulled away from the text to do this.

iBooks featuresiBooks Features

Just before lunch, Steve gave an introduction to the iBooks Author program – showing us how to set up new chapters, sections and pages. After lunch, we began working through the different features of the program, from image manipulation to interactive features such as quizzes and slideshows. It was useful to have an opportunity to play with all the different features available, to really test drive the program and look what opportunities it affords.

Feedback following the workshop was very positive – attendees felt that the session was informative and well pitched and that it offered many ideas for use in the classroom.

 

Designing and running successful online surveys

Photo credit: Imaged shared by idleformat, under a Creative Commons license

Today I attended a workshop on designing successful online surveys, facilitated by Tabetha Newman. The workshop, held at Bristol’s M Shed, aimed to help attendees understand the basics of good online survey design; from choosing software to the practicalities of running a survey. This was the first in a two part series of workshops – the second, to be held next spring, will focus on the management and analysis of data collected via online surveys.

The most useful section of the workshop, for me, was the introduction of Tabetha’s POETS model of design:

Population

Objectives

Ethics and data protection

Timeline

Sampling

Each letter of the acronym holds within it a range of questions and points to consider. For example, under Timeline, Tabetha notes the importance of factoring in extra time for testing, receiving feedback, chasing respondents and data analysis.

This method has helped me to organise my thinking around the requirements of the framework tool. The self-evaluation framework is by no means  a standard survey, and having a structured way of considering how to present the framework will be very useful.

Tabetha also touched on some important points to keep in mind when designing the look of the survey. She noted the importance of font size,  users should be able to see the question they are answering. The overall organisation of the page was also considered, it is good practice to keep questions uniform, changing the style or size of a question can throw participants off. Tabetha also commented on the advantage of fresh eyes – asking someone else to look over the framework and examine where their eye naturally falls on the page.

Through today’s workshop I was also introduced to Mike and Tom, who work on Bristol Online Surveys. They had some very useful advice regarding the final presentation of the framework- since it is rich in content I have had serious concerns about it being overwhelming for participants and they had some really great solutions to this.

Overall, the workshop has been very useful to me – personally I am someone who prefers to hear tips and advice from someone who can relate them to actual practice – seeing someone talk about something they clearly have a lot of experience in is far more informative than reading a textbook!

Digital Futures in Teacher Education

Photo credit: Picture shared under a Creative Commons Licence by James Armstrong

Yesterday, I attended the Digital Futures in Teacher Education (DeFT) dissemination conference in Sheffield.

The aim of the DeFT project is to develop guidance on digital literacy practice and to create an Open Textbook on Digital Literacy for Education. The Open Text book is a big online space used to share the outcomes of the individual projects.

The conference began with an introduction from Richard Pountney and Dr. Anna Gruszczynska. This was followed by a brief review of the main themes emerging from the project by Cathy Burnett and Julia Davies.

Some interesting points arose during this presentation, including the notion of the ‘Knight’s Move’ as a metaphor for staff development. Julia Davies began by discussing that often when new technology is brought into the classroom is it used for existing practices and ways of teaching. Julia argued that perhaps this is necessary in the early stages of innovation, in order for confidence to grow. That like the chessboard Knight, a practitioner must take a few steps before veering in an entirely new direction.

Julia talked about how ICT can be used to break down the barriers of time and space for CPD – meaning that opportunities for teachers are no longer restricted to particular times and physical locations. Devices learners use on a daily basis are blurring the boundaries between the classroom and the home environment, which Julia referred to as ‘the third space’.

The Professional Development Issues session involved four case studies:

Mick Connell made some insightful comments regarding their project with PGCE students to produce digital videos about different themes around Sheffield. He felt that the process of producing the videos was just as much, if not more valuable to the students than the end product itself, – being the creator of a digital resource had developed their skills. He also commented on the student’s ‘Digital Daring’ – the willingness to take risks when trying new approaches with technology. This was one of the biggest areas of gain from the project.

Sarah Butler showed us videos of Hallam’s Digital Agony Aunts – a forum developed to allow the student teachers to air concerns and share their experiences. Sarah’s presentation highlighted the student’s interests in sharing best practice. Her students saw sharing resources and ideas as a part of their professional identities and struggled to understand why anyone wouldn’t want to share what works openly online?

Michael Payton-Greene introduced us to Wales High School’s staff development blog. The blog is used to share best practice throughout the school, with faculties commenting on blog posts when they had taken the ideas into their own subjects. Here’s a good example looking at the use iPads across the school.

Christine Bodin demonstrated how she uses Moodle for her French and Spanish teaching. Christine looked at how she has adapted to technological change over her career and how she has utilised Moodle to deliver a varied Modern Foreign Languages (MFL) curriculum.

After a short tea break, the first Keynote was delivered by Dr. Doug Belshaw (Badges and Skills lead for the Mozilla Foundation).

Doug talked about the definitions of open educational resources and digital literacy and used a fairly funny video of DMX battling with Goggle to highlight the dangers of making assumptions about a person’s ability with digital tools. He also commented on the spectrum of digital literacies: from tool-based knowledge to critical and reflective skills. Doug has also blogged about the event.

The mobile learning session included three presentations:

It was fascinating to look at the use of mobile devices across Early Years Foundation Stage, Primary and Secondary. At Sharrow Nursery school, cameras and voice recorders had been used to document the children’s progress through a module on Farm life. The images and recordings where then posted on the Nursery’s blog and parents were informed on how to access these.

At Winterhill, the pupils were inspired by Magna Science Adventure Centre to create artwork, compose music and write poems. QR codes were then developed to link the physical Magna environment with the school children’s creations.

Photo credit: Picture shared under a Creative Commons Licence by Winterhill Comprehensive School

QR codes were also used by Halfway Junior School in a treasure trail activity that aimed to use the technology in order to develop a narrative. At each stage of the trail the children were given a clue as to what happened at the site of the activity and a clue for finding the next QR code. The children were also asked to write about what they believed had happened in the park.

The second Keynote of the day was delivered by Bob Harrison, drawing on his work with the New Technologies Advisory Board. Not only was Bob’s presentation studded with funny anecdotes, it also included a number of insightful one-liners:

“You can do a lot WITH teachers, but not an awful lot TO them”

“Twitter is the best CPD tool I’ve had in all my 40 years in education”

One of the key questions asked in the closing panel was: In the current policy vacuum, where do we go for advice, guidance and resources? Bob Harrison’s reply was:

“The answer is teachers helping teachers”

We are supporting this approach across Leicester through our interests in networked learning and the sharing of best practices. The purpose of the Digital Literacies Research Associate role is to develop a self-evaluation framework. This will be used by staff in order to reflect on their use of technology to support teaching and learning. Alongside the framework we hope to support the growth of a bank of open educational resources that will aid staff in developing their own digital literacy practices.

The conference was closed by Academic Lead, Professor Jackie Marsh and Principal Investigator, Professor Guy Merchant.

The Open Textbook is due to be released on November 1st 2012.