Measuring Frequency on Likert Scale

I’m putting together a questionnaire for my survey. I was thinking what is the best way to let the respondents to describe their study environment. And I came up with asking the “how often?” question.

For example:

How often…

  • … were you alone while studying?
  • … did you study in silence?
  • … did you study in front of the computer?

etc.

Here’s why I ask the frequency and do not let people to describe their surroundings:

1. From the former studies I know that it took several study sessions to complete the first level of the course. So if I would ask How crowded was the room? or How many people were in the room? then the respondent would not know which session to describe. Some people would naturally describe an average situation but not everybody.

2. A factor can either distract you from learning or not (in the CLT point of view – something is either using your cognitive resources or not, taking your attention or not). Measuring the amount of distraction would be too difficult and more to the qualitative side. So there is no point asking “How much”. Instead I ask “How often” to get the amount of distractions.

3. I know that measuring frequency on the Likert scale must be done very carefully. But right now I cannot imagine having numeric choices – once a week, once in a lifetime, every other second. People took the course in different tempo. The number of sessions was different and the time span too. After all, some people did it a while ago so very specific questions would be rather annoying and the answers not reliable. Besides, I have “never” at one side of my scale and “always” on the other side. Only if everybody answer “never” to all questions can I argue that the environment had now effect to their learning.

So here’s my Likert scale (in Estonian):

  1. mitte kunagi (never)
  2. harva (rarely)
  3. mõnikord (sometimes, occasionally)
  4. sageli (frequently)
  5. alati (always)

I browsed through many examples but the language kicks in: the meaning of sometimes in Estonian (mõnikord) is rather less than a half. But I want to have an equal scale with a proper center. Still, since I will have to translate it back later, I have chosen the “mõnikord” (sometimes).

To have a numeric scale that is even by default? No, that would be describing time through other numeric scale that we are used to (seconds, minutes, hours etc).

 

 

Thoughts after hearing Dr. Dieter Sturma’s Lecture on Cyborgs, Robots and other Persons

The lecture was entitled The Culture of Enhancement. On Robots, Cyborgs, and Other Persons The lector, Dr. Dieter Sturma is a professor of philosophy at the Bonn University.

Some points from the lecture:

An human being as a representative of a biological species and a person can be approached differently.

Technically, everybody who wears glasses, implants or other technological aids is a cyborg. How many alien parts should a man have to become non-human? 200 is normal today. A million? More? We cannot have a limit like this.

When creating robots humans have slaves in mind. Asimo is just a walking battery, but should robots gain personality then slavering them would be unacceptable.

My thoughts:

1. In educational technology, research has proved that human feedback in multimedia study materials has advantages over automated feedback. But if we give the choice to students, they would choose automated feedback. Is that because of the poor level of the current “artificial intelligence” or rather because of the fair to show one’s stupidity to another person? Being stupid and feeling shame is something that only a person is capable of (according to today’s lecture).

2. My children watch The Cars, Pelle the Police Car, The Little Rescue Boat Elias etc. All these animated movies have machines talking and acting like persons. But when the children go on the street they don’t expect a car to talk. More evidence to the fact that “real” experience convinces more than any multimedia presentation?

3. An addition to the thick list of features that helps us to distinct humans from other biogical species – generosity, willingness to share. There are even theories that argue that generosity was the trigger to human evolution.

 

Effects of the Environment When Starting to Play a Musical Instrument Instructed by a Multimedia Study Material

Studying a musical instrument with the help of a multimedia study material is different from studying any theoretical discipline because there is an extra source of information: the additional visual and audio information that the learner creates by playing the instrument.

As the learner’s cognitive resources are limited (CLT), there is a higer risk of cognitive overload. But before starting to optimize the study material we should see what else can use learner’s cognitive resources.

250+ beginners studied guitar with the help of the www.guitarschool.ee multimedia course and recorded 4 test exercises. The study process took place in a natural environment (75% home, 20% office, 5% elsewhere) and was not controlled (most common case when studying a musical instrument). The study material was the same for everyone so it is possible to measure the effect of the study environment on study results. If the effect is significant then it is important to reduce it in order to improve study results. Otherwise whatever improvements we make to the study material to use of the learner’s cognitive recourses more efficiently, the overall effect may be not big enough.

It is not possible to change the actual environment, we only can change the study material. There are two ways to neutralize the effect of the study environment by the study material:

a) reducing the cognitive load caused by the study material. This means putting through less information and causing the study process to slow down.
b) reducing the effect of an environment by demanding more engagement from the learner (gamification, more interaction)

I will need 3 bits of information:

1) Facts about the environment (What was it like? from questionnaire)
2) Cognitive effects of the environment (How did it feel? from questionnaire)
3) Study results (recordings, achieved level of total 8 levels of the course)

The first set of questions asks the respondents to describe the actual characteristics of an environment, the second set wants to know how did it feel. For example, if there were potentially annoying factors in the room but the learner didn’t notice them, then we cannot consider them as extraneous load (CLT).

Reading: Garcia Rodicio, H., Sanchez, E., Aids to Computer-based Multimedia Learning: A Comparision of Human Tutoring and Computer Support

Interactive Learning Environments, 20:5, 423-439 (2012)

Main point:

Multimedia instructional materials have to be enriched with support devices in order to be effective. There are two approaches: inserting the support into material (canned  support) or using human tutoring. An experiment was conducted (n=84 into 3 conditions: human aid, canned support, no support) and it showed that human tutoring was better.

From the literature overview: one can experience a human-human interaction with computer.

Further reading – Ginns, P. (2005). Meta-analysis of the modality effect. Learning and instruction, 15

Reading: Fogg. BJ, A Behavior Model For Persuasive Design

Persuasive ’09, Apr 26-29, Claremont, CA. www.behaviormodel.org

Main point

Fogg Behavior Model (FBM) says that behavior is a product of three factors: motivation, ability and trigger. These have to be present in the same moment.

The combination of motivation and ability defines the likeliness a person is to behave in a desired way. It is possible that if one of the factors is too low then the other can compensate it but most common case is that they are not at extremes.

Trigger has to occur on time. Timing can often be the missing element.

Elements of motivation:

  • Pleasure/Pain
  • Hope/Fear
  • Social Acceptance/Rejection

Elements of simplicity (ability=simplicity because people don’t want to learn to increase their ability)

  • Time
  • Money
  • Physical effort
  • Brain Cycles (need to think)
  • Social deviance (am I “normal”?)
  • Non-routine (people like routine)

Types of triggers

  • Sparks (boosts motivation, can be annoying)
  • Facilitator (says that “your ability is enough for this”)
  • Signal (reminder, traffic light)

Triggers are more important than ever because in electronic world it is possible to act immidately.

FBM can also be used to prevent behavior but it is more difficult to do.

 

Reading: Perry, G., T., Schnaid, F., A Case Study on the Design of Learning Interfaces

Computers & Education 59 (2012) 722-731

Main problem:

The design of educational software interface has a multidisciplinary nature. It takes at least two different experts to create a high quality interface: a designer and a teacher.

The Case:

Authors observe four experts designing an interface. The experts worked in pairs: An experienced designer together with a chemistry teacher and a design student with chemistry professor. Each pair was asked to design two interfaces about chemistry. They had 2 h sessions for both designs after which they briefed the outcome to the experimenter.

Results:

Designers used different approaches: (“integrate as fast as you can” and “structure then design”). Both designers were unfamiliar with the domain and thus the effect of interaction in pair could be causing the differences (partly). Educators played different roles in pairs: one pair shared all tasks, the other divided them. The lack of knowledge of learning theories on the experienced designer’s side was the major source of debates. The structure of all designs was the same: web-page-like. The graphs presenting the quantitative data shows how the experts used their time.

Reading: Muntean, C. I., Raising Engagement in E-learning Through Gamification

The 6th International Conference on Virtual Learning ICVL 2011

Main points:

  • Gamification is the use of game-play mechanics for non-game applications. Theoretically anything can be gamified.
  • G’s main goal is to increase user’s (learner’s) engagement
  • Gaming elements are already present in some applications
  • There is little research regarding the usefulness of G in education

Issues to be improved in “normal” e-learning process:

  • E-learning cannot transmit emotion as a teacher would. Thus the e-material has to compensate it by other means
  • Motivation, ability, trigger (Fogg’s Behavior Model)

Dangers:

  • Students may learn to learn only when provided with an extrinsic motivator (Pavlov ?? 😛  )

How to gamify an e-learning app?

Who is the subject? (type of personality and context of learning: how much information is expected to come through the e-course – 0-100% – and the rest comes from direct contact with humans). Some elements to use (some work better for one subject :

  • Uncover content progressively (in the case of linear material)
  • Focus on exercises, offering extra points for solving the odd theory problem
  • Any student need an account, personal profile with avatar etc
  • The course should be organized into smallest coherent units of content
  • At the end of each chapter the student gets to the next level and the rest of community should know about it (leaderboard, top scores)
  • Constant feedback (progression bars)
  • Support periodical learning by deadlines or virtual appointments
  • System should be as social as possible
  • Special bonuses for complex or extra tasks
  • Compensation for proper behavior and social engagement not only academic achievements
  • Possibility to convert points into something else (virtual goods, even reduction of tuition fee)
  • After advancing a level, student should be informed what happens next

Conclusions

Gamification is not creating a game. It helps to motivate and engage students. Engagement can be measured (how many visited pages, time spent on site, frequency of visit). G offers proper tools to generate positive change in behavior according to Fogg (A behavior model for persuasive design: using computers to change what we think and do. Ibiquity 2002, Dec).

 

 

Reading: Lee, J., Hammer, J. Gamification in Education: What, How, Why Bother?

Academic Exchange Quarterly, 15 (2). A short article playing around with the idea of using gamification in schools:

  • Too many people drop out from schools. Gamification could help.
  • Schools already have game-like elements: grades, points etc. But it doesn’t feel like game. Feedback is slow to come and there are only a few “lives” to experiment with.
  • More game-like elements to use: possibility and need to experiment (Angry Birds), challenges that match the student’s level, multiple routes to success, failure is the teacher, short feedback cycles, low stakes (less risk, more freedom to try), role play.
  • Gamification is the future of everybody and there is a little chance that schools will not be affected.

Reading: Chi, T., H., Active-Constructive-Interactive: A Conceptual Framework for Differentiating Learning Activities

Topics in Cognitive Science I (2009)

Main point:

Active, constructive and interactive are commonly used terms in the cognitive and learning sciences. But the definitions are not always clear except for constructive. This article provides a framework and proposes a way to differentiate those terms.

From best to worst: Interactive>constructive>active>(passive)

Interactive often describes a system rather than the interactions between a learner and a system. In this article active, constructive and interactive are viewed as types of overt learning activities (from learner’s side).

Taxonomy

Being active is doing something while learning (often involving physical movements). Examples:

  • steering and peddling a stationary bike while traveling through virtual environment
  • looking and searching some specific locations on a chessboard
  • pointing and gesturing at what one is reading or solving
  • copying and pasting some parts of a text
  • repeating sentences verbatim
  • manipulating video tapes (pausing and rewinding etc)
  • rotating objects
  • selecting from a menu of choices

Note: in memory literature repeating words is a passive learning strategy but in this taxonomy it is active

Being constructive is another set of overt activities that cause learners to produce additional output that can contain new relevant ideas that were not in the initial material. Being constructive includes being active. Examples:

  • self-explaining
  • drawing a concept map
  • taking notes
  • asking questions
  • posing problems
  • comparing cases
  • making plans
  • integrating text and diagram
  • reflecting and monitoring own understanding
  • inducing hypotheses
  • constructing timelines (history)

Being interactive is having a dialogue with a human or a machine. Not all dialogues are interactive. Dialogues that are interactive:

  • instructional dialogues with an expert (teacher)
  • joint dialogues with a peer

CLT and this framework are different but can complement each other: CLT can always reduce memory load whether the learning is active, constructive or interactive.

Using FB to Identify Gender

I have a large database (11 000 names) and needed to identify people’s gender by the name. Most of the time it is very easy but there are some uncommon names that can belong to a male or a female person.

That’s when I used Google and Facebook. The latter helped most of the time. There are about 400 000 FB users in Estonia. Pop of the country is 1.3 M, so pretty representative sample that FB has :)