multiple mouse/touchscreen autolocations

There is an interesting feature in the specification of mouse/touchscreen input. You can cause the rectangle corresponding to the click region be determined by the (possibly varying) size and shape of the visual stimulus.

What I don’t see is any way to use this method with more than one active click region–to do that, it appears necessary to specify the height, width, and center of each click region manually and as fixed regions.

It seems to me that when there are mouse/touchpad inputs defined when a visual event is set up, there should be an option to associate the coordinates of that event with a certain input so that it would linger until (1) the display was erased, (2) a new association is made to that input, or (3) the end of the trial (maybe you could make a case for allowing it to linger beyond the trial).

Suppose you’ve defined four mouse/touchscreen input regions called ma mb mc and md, defined to correspond to the coordinates of the visual stimulus. You then display a set of visual stimuli, sa, sb, sc, and sd at various points on the screen, keeping the first three invisible and not erasing the display. Now you want the subject to choose one of the four by clicking or touching it, so inputs ma, mb, mc, and md are correct responses for sd. But what is “the” visual stimulus? There are (from the subject’s and experimenter’s point of view) four different ones on the screen. However, by the time you have put sd on the screen, it is too late to tie the response region to the coordinates of sa, sb, or sc.

So, the new feature suggestion is that it be possible to associate a certain mouse input region with a visual stimulus for later use during the trial. When defining sa, you would say “use coordinates with input ma”, and similarly with sb:mb, sc:mc, and sd:md. This is independent of specifying the correct choice: in the above scenario, for events sa-sc, “the subject must not respond” would be the correct choice; for event sd, all of ma-md would be correct choices. Then, as each of the four stimuli gets displayed, its coordinates would be saved in one of the mouse input regions, essentially building a possibly very complex clickable array in real time.

What about overlaps? I don’t know. What do you do about this with manually defined clickable regions? I’d say, just do the same thing.

As always, if there is already a way to do this, that would be great.

Greg Shenaut

This can be done. I discovered the other day that there’s no writeup for how to do it, though. I’ve attached a really old experiment that demonstrates this functionality.

The trick is that all events must look for input for their position to be registered. The trick is to only let SuperLab look for input for 1ms. Each event also still needs to have a correct response associated with it, and the actual correct response must come last.

It’s incredibly counter-intuitive, but it is possible with all version of SuperLab 4 (I think).

I should probably also point out that the experiment uses variable time limits from a trial variable, text stimulus lists, variable stimulus positions, variable text colors, and feedback.

Test Standard Mouse with Automatic Responses.sl4 (8.4 KB)

OK

I think I get it. Apparently the association of display coordinates to a certain response lasts until it is changed(?), but they are set only when a visual event specifies that response as “correct”, and actually looks for that response while running for at least 1 ms (and of course, the response itself has to be be defined as “automatic”). Each visual event specifies a different mouse click response as “correct”, including the last one.

The trick seems to be to end the last event after “any response” rather than after a “correct response”. If you wanted to do this and limit the responses (for example, to not end the event on a button press or a voice key trigger), you’d be out of luck.

Is that about correct?

Maybe if you absolutely had to limit responses, you could make a zero-length audio file and present a dummy audio event that had all of those mouse-click responses selected as “correct responses”??

Actually, what happens if you select more than one automatic mouse-click responses as correct on a single visual event? I suppose they all get set to the current display region.

Anyway, thanks for that. I probably won’t use it this time because I already set things up manually, but next time…

Greg

Basically, SuperLab will only configure the automatic stim position for the current event and only if it looks for a response. If the time limit is 0ms, SuperLab treats this as an event that doesn’t look for a response, so it doesn’t bother with calculating the position for the correct response.

The association of the coordinates happens during the start of the trial, so in theory, I believe it would apply to the entire trial. It’s plausible that (like you said) the coordinates would simply stay the same until they change. ( I don’t believe this has been explicitly tested beyond a single trial.)

So, the trick is to write your experiment with the given limitations in mind. All of your “incorrect” stimuli must have a correct response defined, and they must look for a response, but the stimulus doesn’t actually have to be visible, and it doesn’t have to look long enough for the participant to actually respond. The only defined correct response that matters (for correctness) is the last one–the one that gives the participant enough time to actually respond.

You wouldn’t be able to limit the potential responses to only the mouse responses unless the other Participant Input devices have support for not looking for input. Currently, I believe this is only supported for keyboard or mouse devices.

You’re also limited to only one response being registered as the actual correct response. You could kludge around this if necessary for feedback by differentiating between key press and release, but even then, you’d only be able to manage two.

Unfortunately, there’s a bug in SuperLab where if you set an automatic stimulus position on a non-visual event, SuperLab crashes, so you wouldn’t want to set one of these as the correct response on an audio file.

I haven’t personally tested what would happen if you select more than one on a single visual event, but I would expect that they would all be given the same value. Therefore, the one that is registered would simply be the one that comes first in SuperLab’s internally list.

This is a feature that many, many people have asked about, and I know I’ve told people how to do it many, many times. Frankly, I was surprised the other day when I couldn’t find it described anywhere. Since you brought it up, I thought it would be appropriate to move your suggestion here to the experiments section and post one of my test experiments.

Nasty potential problem

In pilot testing, I found problem with the mouse click autolocation kludge.

Remember that in order for this to work, you have to define the mouse regions associated with each screen location as the “correct” reponse for its event, but then in the event where the response will actually be made, you have to end the trial after “any response”.

Well, SuperLab defines two default mouseclick responses: “left mouse click” and “right mouse click”. It is critically important to delete the default responses after you’ve added the new autolocating click responses. Otherwise, depending (I suppose) on the order of the list of responses, when you click one of your regions, it will be recorded as a “left mouse click” instead of the response associated with the region.

In two out of the six experiments I just set up, I discovered this after we’d run a couple of pilot subjects.

However, in many cases, there is a simple workaround if you make this mistake and need to recover the response: SuperLab also records the mouse click coordinates. In our case, both of the messed up experiments were simple two alternative forced choice procedures, and the two responses were to the left and to the right of the screen. So, by looking at the sign of the X coordinate, I can salvage the data (unless the subject made a false click somewhere else).

More on the nasty potential problem

It turns out that merely deleting the default “left mouse click” response isn’t quite the correct way to get rid of this problem, which is slightly subtler than I thought. You also need to go into Experiment > Participant Input > Mouse or Touch Screen and click on the Options tab.

Under “Look for Input From Mouse or Touchscreen”, enable “Only if an event designates a mouse response as correct”.

Under “Other Options”, enable "Ignore a mouse click unless if falls inside an area defined as a response.

That seems to be fairly solid.

Greg Shenaut

That sounds right. I apparently forgot to mention that when describing the process, but the sample experiment I posted should be configured that way on both counts.

It would be a good idea to delete the “right mouse click” response too–just in case.