Numberword Trials in Self Paced Reading Quiz
Week iv practical
Collecting reaction time information, more circuitous nested trials
The program for calendar week 4 practical
This week nosotros are going to look at a bit more of the Online Experiments with jsPsych tutorial, then look at code for a elementary self-paced reading experiment - as you should know from the reading this week, in a self-paced reading experiment your participants read sentences discussion by word, and y'all are interested in where they are slowed down (which might indicate processing difficulties). We therefore intendance about reaction times (which we didn't for our grammaticality judgments task last week, although jsPsych collected them for united states of america anyway). We are also going to see some slightly more complex timelines, with trials that consist of several parts and fifty-fifty a javascript function that simplifies the process of building those complex trials for u.s.a. (the tutorial section you work through this week volition teach you lot some basics of javascript to help you understand that). Finally, I'll add an case at the end of how to collect demographic info from your participants, which is often something y'all want to do.
Remember, as usual the idea is that you do as much of this every bit you lot can on your ain (might be none of it, might be all of it) and and then come to the practical drop-in sessions or use the chat on Teams to become assist with stuff you lot need help with.
Tutorial content
Piece of work through section 05 of the Online Experiments with jsPsych tutorial. It'southward up to you whether yous want to do the exercises dotted through the tutorial or not. The key things you demand to take away from the tutorial are:
- How to stand for an assortment and loop through it - you have actually already seen arrays, they are only lists enclosed in square brackets, then the new thing is the
forloop. - The basic info on javascript functions - what a role looks like, and how you make a function render some value.
A cocky-paced reading experiment
Later on you have looked at department 05 of the tutorial y'all will be equipped to look at the self-paced reading experiment, which uses a piddling bit of javascript (an array, a loop, a function) to make building a complex trial listing a scrap easier.
Getting started
Every bit per terminal calendar week, I'd like you to download and run the lawmaking I provide, look at how the code works, then attempt the exercises below, which involve editing the code in simple ways and puzzling over the output.
You demand two files for this experiment, which y'all can download through the post-obit two links:
- Download self_paced_reading.html
- Download self_paced_reading.js
Again, the code makes some assumptions about the directory structure it's going to live in - regardless of whether you are putting this on your own calculator or on the jspsychlearning server, these should sit down in a folder called something like self_paced_reading, alongside your grammaticality_judgments folder from terminal week and a jspsych-half-dozen.1.0 binder containing all the jsPsych code - and then your folder volition now await something like this.
Assuming you have the directory structure all correct, this code should run on your local computer (merely open the self_paced_reading.html file in your browser) or you tin can upload the whole self_paced_reading binder to the public_html folder on the jspsychlearning server and play with it there (if your directory structure is equally suggested the url for your experiment volition exist http://jspsychlearning.ppls.ed.ac.great britain/~UUN/self_paced_reading/self_paced_reading.html).
Starting time, get the code and run through it and so you can cheque it runs, and you can meet what it does. And then take a look at the HTML and js files in your code editor (eastward.thou. Atom).
Nested timelines
Each individual trial in a self-paced reading experiment is really rather complex: information technology involves word-by-word presentation of a sentence, followed by a comprehension question - the comprehension questions are there to prevent participants simply rattling through the sentence without actually reading it.
The way we are going to do this in jsPsych is to accept multiple trials per judgement: one trial for each word in that judgement, and and so a concluding trial for the comprehension question. These are all type:'html-keyboard-response' - for the word-by-word presentation we just desire the participant to hitting spacebar to progress, so we will make the comprehension question a yes/no answer (basically just like in the grammaticality judgments lawmaking).
We could do this all manually, and only specify a huge long trial list like this:
var spr_trial_the_basic_way = [ { type : ' html-keyboard-response ' , stimulus : ' A ' , choices : [ ' space ' ]}, { type : ' html-keyboard-response ' , stimulus : ' self-paced ' , choices : [ ' space ' ]}, { type : ' html-keyboard-response ' , stimulus : ' reading ' , choices : [ ' space ' ]}, { type : ' html-keyboard-response ' , stimulus : ' trial ' , choices : [ ' infinite ' ]}, { type : ' html-keyboard-response ' , stimulus : " Was that a self-paced reading trial? " , prompt : " <p><em>Answer y for yes, n for no</em></p> " , choices :[ ' y ' , ' n ' ]} ] That will present the sentence "A self-paced reading trial" one word at a time, waiting for a spacebar response after each discussion, and so nowadays a y/north comprehension question at the finish. Nonetheless, that is quite unwieldy - there is lots of redundant information (we accept to specify every time the trial type, the spacebar input), edifice the trial list for a long experiment with hundreds of sentences is going to be very error prone, and it would be impossible to randomise the order without messing everything up horribly!
Thankfully jsPsych provides a nice way around this. A slightly more sophisticated solution involves using nested timelines (explained under Nested timelines in the relevant part of the jsPsych documentation: we create a trial which has its own timeline, and so that is expanded into a serial of trials, one trial per particular in the timeline (so each of these complex trials functions a bit like its ain stand-lone embedded experiment with its own timeline). Nosotros can use nested timelines to form a more compressed representation of the long trial sequence to a higher place and get rid of some of the back-up.
The simplest fashion to do this is to split the long sequence for a single self-paced reading trial into a pair of trials: the self-paced reading function, which has its own nested timeline of several words, and then the comprehension question. That would look like this:
var spr_trial_using_nested_timeline = [ { blazon : ' html-keyboard-response ' , choices : [ ' space ' ], timeline : [{ stimulus : " A " },{ stimulus : " cocky-paced " },{ stimulus : " reading " },{ stimulus : " trial " }]}, { blazon : ' html-keyboard-response ' , stimulus : " Was that a self-paced reading trial? " , prompt : " <p><em>Answer y for aye, northward for no</em></p> " , choices :[ ' y ' , ' n ' ]}] The showtime part of that is an html-keyboard-response trial, which accepts space as the only valid input, and which has a nested timeline that volition expand out then that nosotros accept our judgement presented in a sequence of four trials. So we have the comprehension question, another html-keyboard-response trial only no nested timeline and looking for a y-northward response.
I discover that quite articulate to wait at, but y'all'll discover that there's still some redundancy (nosotros accept to specify twice that type is html-keyboard-response), plus we are just producing a apartment assortment of reading trials then a comprehension question - you could imagine that if we want to randomise the lodge somehow, we might accidentally seperate a sentence and its comprehension question.
In that location is an even more compressed way of representing this trial sequence, which looks like this:
var spr_trial_using_very_nested_timeline = [{ blazon : ' html-keyboard-response ' , timeline : [ { choices : [ ' space ' ], timeline : [{ stimulus : " A " },{ stimulus : " self-paced " },{ stimulus : " reading " },{ stimulus : " trial " }]}, { stimulus : " Was that a cocky-paced reading trial? " , prompt : " <p><em>Respond y for yes, n for no</em></p> " , choices :[ ' y ' , ' n ' ]}]}] And then that'due south a unmarried html-keyboard-response trial which has a nested timeline; the commencement detail in the nested timeline is the spacebar-response trials, which itself has a nested timeline, so the second detail in the timeline is a single trial with dissimilar choices, stimulus and prompt. Personally I find that slightly more confusing to look at in the code, but I like that what is conceptually a single trial - a sentence plus its comprehension question - is at present a single (complex) trial in the experiment.
It's important to emphasise that these three ways of representing a cocky-paced reading trial all work, and look the same from the participant perspective - which one you choose might be decided by things like what you plan to do for randomisation, or how confident you are that yous empathize what the nested trial lists are doing!
Nested trial lists therefore get in quite like shooting fish in a barrel to build a single self-paced reading trial. However, it's still going to be a flake laborious to build a sequence of such trials. In order to build two trials we'd have to do something like this:
var two_spr_trials = [ { type : ' html-keyboard-response ' , timeline : [ { choices : [ ' space ' ], timeline : [{ stimulus : " A " },{ stimulus : " self-paced " },{ stimulus : " reading " },{ stimulus : " trial " }]}, { stimulus : " Was that a cocky-paced reading trial? " , prompt : " <p><em>Answer y for yes, due north for no</em></p> " , choices :[ ' y ' , ' n ' ]}]}, { type : ' html-keyboard-response ' , timeline : [ { choices : [ ' infinite ' ], timeline : [{ stimulus : " Another " },{ stimulus : " cocky-paced " },{ stimulus : " reading " },{ stimulus : " trial " }]}, { stimulus : " Fob question: wasn't that a self-paced reading trial? " , prompt : " <p><em>Respond y for yes, n for no</em></p> " , choices :[ ' y ' , ' due north ' ]}]} ] So that's a list of two trials, both of which are identical in all their details except for the give-and-take list and the comprehension question. Building a long list of trials like that is definitely exercise-able, but is probably quite error prone - to change the word listing or the comprehension question I have to jump into exactly the right spot in the nested timelines and change the right thing, and inevitably I will forget at some point or make a mistake. Plus it's an entirely mechanical process - if you lot know the judgement it's obvious how to slot it into our trial template - and computers are good at doing mechanical stuff methodically, so information technology makes more sense to automate this.
What nosotros'll exercise is employ a little bit of javascript and write a function which takes a sentence and a comprehension question and uses this template to build a trial. It splits the sentence into an array of words (splitting the sentence at the spaces using a built-in javascript function called separate), and then uses a petty for loop to build the give-and-take-by-word stimulus listing. Then information technology slots that give-and-take-past-word stimulus list plus the comprehension question into our trial template, and returns that trial.
Hither's the office. I have called information technology make_spr_trial (spr = self-paced reading), and it takes two arguments (Alisdair calls them parameters in section five of the tutorial, I gauge he was raised wrong): a judgement to present word past word, and a aye-no comprehension question.
part make_spr_trial ( sentence , comprehension_question ) { var sentence_as_word_list = sentence . divide ( " " ) //split the judgement at spaces var sentence_as_stimulus_sequence = [] //empty stimulus sequence to outset for ( var give-and-take of sentence_as_word_list ) { //for each word in sentence_as_word_list sentence_as_stimulus_sequence . button ({ ' stimulus ' : word }) //add that word in the required format } var trial = { blazon : ' html-keyboard-response ' , //plug into our template timeline :[{ choices : [ ' infinite ' ], timeline : sentence_as_stimulus_sequence }, { stimulus : comprehension_question , choices :[ ' y ' , ' due north ' ], prompt : " <p><em>Reply y or n</em></p> " } ]} return trial //return the trial you accept congenital } Now it is very easy to build multiple trials using this role. Note that the arguments we pass in - the sentence and the comprehension question - are strings, so enclosed in quotes.
var spr_trial_1 = make_spr_trial ( " A self paced reading trial " , " Was this a self paced reading trial? " ) var spr_trial_2 = make_spr_trial ( " Another self paced reading trial " , " Trick question: Wasn't this a cocky paced reading trial? " ) Other $.25 and pieces, including collecting demographics
As usual, your experiment will demand a consent screen and some instruction screens. Those $.25 are basically the same as final calendar week and so I won't carp showing the code hither, but two quick comments:
- Equally per last week, for the initial consent screen I am using a button response - I like a push response here because I think information technology makes information technology less likely that people will keypress through before they realise what they are supposed to be doing.
- For the instruction screen I am using a keyboard response trial in the same way I did last week. But annotation that jsPsych provides an instructions plugin (https://world wide web.jspsych.org/plugins/jspsych-instructions/) which would be better if you were providing several pages of instructions.
For this experiment I take also added a trial (just before our very final final_screen trial) where we collect some additional info from the participant. Often you want to collect demographic information from your participants - e.g. age, gender, whether they are a native speaker of some language - and give them the opportunity to provide gratis-text comments (east.grand. in case in that location is a problem with your experiment that they have noticed). In full general you shouldn't collect information you don't actually need - it wastes the participants' time, potentially means you are storing unnecessary personal information most your participants, and too opens up diverse temptations at analysis time ("Hmm, this experiment doesn't looked like it worked, how deadening. But expect! If I separate it by gender and age, which I collected for no real reason, then I get a weird pattern of significant results, maybe I can pretend I predicted that all along and publish this?"). So don't feel you e'er demand to include the exact questions I have put here, these are merely some examples of how to collect some common response types.
The survey-html-course plugin provides a style to mix various response types on a single form - in this instance I am going to include a radio-button response (select one from a number of options), a text-box response that only accepts numbers, and a larger text box for more open comments. But at that place are lots of other options - if you lot are wondering "tin I do 10?", expect at the documentation for the input, textarea and select tags.
Nosotros include our demographics questionnaire by creating a unmarried trial - note that it has blazon: 'survey-html-class', and in my html file I therefore have to load the appropriate plugin (line viii of self_paced_reading.html does that).
var demographics_form = { type : ' survey-html-class ' , preamble : " <p mode='text-align:left'> Please answer a few final questions about yourself and our experiment.</p> " , html : " <p style='text-marshal:left'>Are you a native speaker of English language?<br> \ <input type='radio' proper noun='english language' value='yeah'>yes<br> \ <input blazon='radio' proper name='english' value='no'>no<br></p> \ <p style='text-marshal:left'>What is your age? <br> \ <input proper noun='age' type='number'></p> \ <p style='text-align:left'>Whatsoever other comments?<br> \ <textarea name='comments' rows='10' cols='60'></textarea></p> " } The interesting stuff happens in the html parameter of this trial, so I'll interruption that downwardly for y'all. Once once again this is a string (enclosed in double quotes) that includes some HTML markup tags. The simplest role of that is the the flake of code that collects the historic period info:
" <p style='text-align:left'>What is your age? <br> \ <input required name='age' type='number'></p> " And then it's a paragraph (enclosed in <p> ... </p>), and I have used style='text-align:left' to get in left-justified so information technology doesn't wait too awful. There'due south a question ("What is your age?"), then a <br> tag to produce a line pause. Then <input required name='age' type='number'> creates an input field, which will exist referred to as age in our code (that's how it appears in the results, equally y'all will see later), and nosotros tell information technology that this trial is of the number type, which lets the browser know how to display it (east.g. if we bandy type='number' for blazon='text' then you lose the little scroller to increase/subtract the number). The required flag means that this response has to exist provided - your participants will non be immune to progress unless they provide a response. You can similarly brand radio button, bank check box and text area input required (in the same manner, by adding this required flag), but employ it sparingly - e.one thousand. if you make an open-ended comment obligatory that is likely to annoy people.
The comments box is the same idea, but instead of using an <input> tag we are using <textarea><\textarea>. Note that there is null between those tags - if you put in some text (e.m. <textarea>Initial text<\textarea>) then that would appear in your textbox without needing to exist typed in by the participant, which is not really useful for us here. Nosotros tin likewise specify the size of the box in rows and columns - people ofttimes take their cue well-nigh the length of the response desired based on the size of the box.
The radio buttons (yes vs no for "Are y'all a native speaker of English language?") are slightly more complex. The relevant part looks like this:
" <input type='radio' proper name='english language' value='yes'>yes<br> \ <input type='radio' name='english' value='no'>no<br> " So that is two input fields, ane for yep and ane for no, but they have the same name (proper noun='english'). The browser knows in those circumstances to only allow people to select one option from amid those options that share the aforementioned proper noun. And so we have value='yes' for the yes input and value='no' for the no input - this is the response that will be recorded depending on which button the participant selects (if yous leave the value bit out then the code records a very unhelpful reply of "english language=on", i.e. it just tells you that the participant answered the question merely not which respond they gave). Then finally we have the text that appears alongside the button, yes and no respectively, with a <br> in between the buttons to get in wait prissy.
The total timeline
The full timeline for this simple 2-trial experiment then looks like this:
var full_timeline = [ consent_screen , instruction_screen_1 , spr_trial_1 , spr_trial_2 , demographics_form , final_screen ] And and so we use jsPsych.init to run information technology - over again, nothing fancy going on hither, and nosotros are just dumping the information to the brandish at the stop. Next week I'll show you lot how to do something a fleck more useful with the data, i.due east. relieve it as a CSV file.
jsPsych . init ({ timeline : full_timeline , on_finish : office (){ jsPsych . information . displayData ( ' csv ' )} }) Exercises with the grammaticality judgment experiment code
Attempt these problems.
- How would you lot add actress trials to this code, i.east. additional sentences and related comprehension questions? Replace my lightheaded placeholder sentences with some more advisable ones, due east.k. inspired by the items in Table 2 of Enochson & Culbertson (2015).
- Add another demographics question, eastward.g. a text box to list other languages spoken, or some additional radio buttons with more than two options.
- Accept a expect at the data that is displayed at the end of the experiment. Can you see where the stimulus for each trial is recorded? Can y'all encounter where the crucial reaction time information for each trial is recorded? Can you see how the demographics data is recorded? Tin can you work out what the "internal_node_id" cavalcade is doing (which looks like east.grand. "0.0-2.0-0.0-0.0" … "0.0-two.0-0.0-one.0 … "0.0-ii.0-0.0-two.0")?
- If you were going to analyse this kind of information, you would demand to pull out the relevant trials (i.east. the ones involving self-paced reading, and comprehension questions). Is it going to be like shooting fish in a barrel to practise that based on the kind of output the code produces? How would you identify those trials? If y'all were particularly interested in sure words in certain contexts, is it going to be piece of cake to pull those trials out of the information the lawmaking produces?
- [Optional, more challenging] An alternative to self-paced reading is the Maze task (east.g. Forster et al., 2009; Boyce et al., 2020); like self-paced reading your participants work through a sentence word by word, but unlike in self-paced reading at each step they chose one of two continuations for the sentence (see epitome below from Boyce et al., 2020 - G-Maze refers to mazes where the distractors are English words which would be ungrammatical continuations, 50-maze has non-give-and-take distractors). Can you catechumen the self-paced reading code to run a maze job? For each discussion presentation you will need an alternative continuation, and some manner of the participant selecting their continuation (e.g. keyboard? push?). Maze tasks also don't feature comprehension questions so you tin drop those (the thought is that selecting the right continuation throughout shows you lot are paying attention). Mazes as well abort the sentence when the participant makes a mistake - we haven't covered this withal and information technology is somewhat catchy, but is possible using
on_finishandjsPsych.endCurrentTimeline(see example in jsPsych core documentation).
References
Boyce, V., Futrell, R., & Levy, R. P. (2020). Maze Made Easy: Improve and easier measurement of incremental processing difficulty. Journal of Retentiveness and Language, 111, 104082.
Enochson, K., & Culbertson, J. (2015). Collecting Psycholinguistic Response Fourth dimension Data Using Amazon Mechanical Turk. PLoS ONE, 10, e0116946.
Forster, M. I., Guerrera, C., & Elliot, L. (2009). The maze task: Measuring forced incremental judgement processing fourth dimension. Behavior Research Methods, 41, 163-171.
Re-utilise
All aspects of this work are licensed under a Creative Eatables Attribution four.0 International License.
Source: https://kennysmithed.github.io/oels2020/oels_practical_wk4.html
0 Response to "Numberword Trials in Self Paced Reading Quiz"
Post a Comment