RoboSubject

SuperLab SuperLab 4 Support
1

After the discussion a week or so ago regarding timing over USB with SuperLab, I decided to enlist the assistance of an entity I call “RoboSubject” to see if any light could be shed on the situation. Hisham had stated that the expected variability in timing would be on the order of ± 10-12 ms; this seemed large to me, but I wanted to find out more.

RoboSubject is a “Basic Stamp” with a photoresistor and a reed relay. The photoresistor is fed into a voltage comparator. When running, the photoresistor points roughly upwards so that it touches the place on a folded-down MacBook Pro screen where a stimulus consisting of a “=” sign is presented. The comparator is biased so that it is triggered by the difference in light between the presence and absence of the stimulus.

The reed relay is connected to a switch input of one of our X-Keys USB Switch Interface boxes, which is what we have been using to gather experimental data.

The Stamp is programmed to wait for a stimulus onset, then pause for a programmed number of milliseconds, and then close the relay. After 100 ms (I also tried 75 ms and 250 ms for the button press duration), it opens the relay, and waits for the stimulus to go away before looping back again.

The Basic Stamp is no ball of lightning, but it is fast enough for this purpose. I ran it several times with different RTs programmed, and here are a couple of typical results:

At 300 ms programmed RT, the mean was 411 ms (!), with a range of 21 and an SD of 5.0. At 500 ms, the mean was 605, the range 33, and the SD 6.2 ms.

I believe that the extra 100+ ms is a function of the switch interface, but it’s really strange. Why would it wait ~100ms to send the key code? Some kind of aggressive debouncing? In any case, it seems to be roughly a constant.

Some of the constant lag could be due to the position of the stimulus on the screen, but that couldn’t account for more than about 1/4 of a vertical period, since it was about 1/4 of the way down (unless LCDs scan differently from CRTs, which I guess is possible).

The more important and interesting numbers are the range and SD, which are roughly of the magnitude suggested by Hisham. There seems to be a tendency for the variability to increase with longer RTs, I don’t know why this would be the case. I suppose some of this could be the product of my instrumentation, for example variability in the relay closing or the comparator triggering, or in the timing within the Stamp. I suppose I’d feel more comfortable about the timing if I could glue the photoresistor to the screen, but I’m not going to do that with my Macbook Pro.

I’m putting RoboSubject away for now, but I’ll undoubtedly resurrect him/her at some future point. For one thing, I’d like to figure out what’s going on with the X-Keys box and that ~100 ms constant delay. Or is this a general USB or HID phenomenon? For another thing, I think it might be better to set up the Stamp to deliver several different RTs in the same run so that extraneous factors such as the exact positioning of the photoresistor would not change. As it is now, I have to open the Macbook, start the next Superlab run, then close down the lid so it touches the photoresistor. Small differences in positioning could change the sensitivity of the stimulus detection enough to account for the small differences I’m seeing between runs.

Greg Shenaut

2

Greg,

First, let me start off by saying that I think you’re awesome. :smiley:

I’m e-mailing you a link to an updated prerelease build of SuperLab 4.0.3.

A few details:

  1. The RT timer on this version is better matched with screen refreshes. Unfortunately, this information isn’t available for built-in Apple displays. Attempts to get the refresh rate or beam position all fail on Apple LCDs (but not on any of my third-party DVI LCDs). So, when you rerun this experiment, try it with an external non-Apple display and see how that affects your timing results. Unfortunately, previous versions of SuperLab 4 would give stimulus onset times as if the refresh information were not available. You can blame me for this. Sorry. :o

  2. On OS X, the RT timer is reset when SuperLab tells the OS to flush the drawing to screen. In this new version, SuperLab waits until the display starts drawing at the top of the screen to flush the buffer (when this information is available). This means that the timer is reset before the stimulus is visible–not after. This also means that on 10.4, the timer is reset up to one refresh before the display actually starts the physical drawing process. This is forced by 10.4’s “coalesced updates” feature, which synchronizes all drawing to the screen’s refresh rate. This can be disabled with “Quartz Debug,” an Apple program that is part of the developer tools. With beam sync turned off, then SuperLab would flush directly to the screen instead of the OS’s own offscreen buffer, as it also does on 10.3.

While the potential extra frame in the timing may seem bad, waiting until after the vertical blank produces much more consistent results on 10.4 relative to flushing immediately before the vertical blank, as it’s impossible to predict how long the call to CGContextFlush will take. Flushing immediately before the VBlank could produce results where the image shows up on the screen right away OR if it takes a hair too long, the image will appear one frame later, variably inserting 16ms on a 60Hz monitor.

The above timing theory is based on what I’ve read of Apple’s documentation as well as my experience with SuperLab’s own internal time stamping. We expect to be doing more hard-core verification in the not-too-distant future like you’re doing, as we’ve received a prototype for a hardware time-stamping product we’re working on.

3

I totally forgot to mention that Hisham did a lot of research on photoresistors and found quite a bit of variation in their response times. He also commented on the possibility of variation in your switching mechanism, as he agreed that an extra 100ms is a bit much. What specific photoresistor are you using to detect the change on the screen? What are you using to push the button?

4

Weekend

The photoresistor doesn’t have a part number! When I ordered the Basic Stamp from Parallax, I just snagged several of the only photoresistor they had in their “accessories” sub-catalog. I went back to their site today to see if I could find a part number or any specifications for it, but I couldn’t. This weekend, I plan to scope the photoresistor’s response to various stimuli, such as an LED being triggered at a fixed frequency. That should clarify the photoresistor’s timing characteristics. I can also fiddle with the comparator circuit a bit, depending on what I see on the scope.

As for the “response button”, I’m using an ordinary 250 Ohm, 5 VDC reed relay I picked up at the local Radio Shack, and driving it with one of the Stamp’s output pins (with a protection diode, of course). I’ll try to scope out its timing characteristics too, along with those of the mystery photoresistor.

I’m also curious about the Basic Stamp’s timing accuracy, so I’ll be playing around with that a bit too. For example, it could be that the Stamp’s “millisecond” is really 1/1024 s, or some other slightly “off” interval.

Greg

5

We had great difficulties getting anything like good results from regular photoresistors on an LCD so I do suggest you make sure you get a good one. We ended up needing one that also was sensitive to the horizontal sweep on a CRT. But, we got it to work for the general case.

I think I may apply our robosubject to SuperLab and see what we get.

I hope people who are worried about 10-20 msec variance of presentation and response times aren’t presenting stimuli on LCDs!! (well, maybe that new 4 msec Viewsonic)

6

More on robosubject, sort of

Well, I played around with the robosubject a bit more, this time with oscilloscope probe in hand.

First, the photodiode is definitely the weak link. I added a trimpot to the circuit that allowed me to adjust the tigger voltage very precisely, and that helped reduce the latency a bit, but there was still a delay. I also scoped the X-Keys switch interface, very crudely, and think that the latency from it is on the order of 4 ± 2-3 ms. I also changed the robosubject program to give 300, 400, and 500 ms responses, looping through this sequence forever. Here is a crude histogram of the results of one run:

–300–
392 02 **
393 02 **
394 00
395 00
396 02 **
397 00
398 02 **
399 10 **********
400 08 ********
401 10 **********
402 08 ********
403 06 ******
404 08 ********
405 04 ****
406 10 **********
407 04 ****

–400–
489 02 **
490 02 **
491 00
492 02 **
493 00
494 00
495 00
496 02 **
497 00
498 04 ****
499 10 **********
500 06 ******
501 14 **************
502 08 ********
503 06 ******
504 12 ************
505 02 **
506 04 ****

–500–
593 02 **
594 02 **
595 00
596 02 **
597 00
598 06 ******
599 06 ******
600 00
601 02 **
602 08 ********
603 04 ****
604 14 **************
605 04 ****
606 06 ******
607 10 **********
608 04 ****
609 02 **
610 04 ****

I must admit that I am less confident about the repeatability of the photodiode circuit, but I offer this data for what it’s worth. I also tried the same run with a copy of a pre-release version of Superlab that Hank was kind enough to give me, with results that were virtually identical.

The three things I notice about this, which may or may not be relevant to anything, are (1) that for each of the three programmed RTs, the range of the results is about one vertical period of the display, (2) that there seems to be a fairly consistent delay of about 100 ms relative to the programmed RT, and (3) that the distribution is skewed, with a few relatively shorter RTs and then the bulk of the RTs clustered near the top of each range. How much of all this is from the LCD display, the photodiode, the ucontroller, the relay, the switch interface, the USB hardware, the OS/X USB/HID software, and Superlab, I don’t know.

Greg Shenaut

7

You said the results were virtually identical with the pre-release version. Were you still running on an Apple display? If you launch SuperLab from the command line with the “–verbose” flag, you’ll get pretty detailed information on what’s happening internally. On all Apple LCDs that I’ve tried (admittedly not a huge selection), I have been unable to programatically determine when a screen refresh is occurring. The verbose information will tell you whether or not this is the case, which would explain your one-refresh variations.

As far as the extra ~100ms… The RT timer is set when SuperLab flushes the offscreen buffer to OS X’s buffer, which means that it is set before the stimulus begins drawing. Assuming things work the way that I think they do, this would be up to one refresh before the stimulus begins visibly appearing on the screen. Therefore, if the stimulus is at the bottom of the screen, this would be less than two refreshes difference–on a CRT. This only accounts for about 1/3 of your extra time, but it will be relatively constant. The variation here comes in how long it takes to flush SuperLab’s offscreen buffer to the OS’s buffer. This can be easily seen in the verbose log.

This still leaves ~70ms unaccounted for, but it’s a start.

8

"The RT timer is set when SuperLab flushes the offscreen buffer to OS X’s buffer, which means that it is set before the stimulus begins drawing. "

I suspected that. Are you using a blocking flush function? If so, it would also be interesting to know the time when that function returns, because this would presumably correspond to the top of the first frame with the new contents. Maybe that information is part of what you get with --verbose, I’ll have to try that.

Cheers,

Greg

9

My understanding is that it only blocks if you try to flush changes more than once in a refresh–hence the changes in the prerelease.

I’m thinking back over these changes, and I think I can do one better… In the prerelease I sent you, SuperLab is manually blocking until the vertical blanking period before flushing the buffer. This has a major benefit on 10.3.9, as this gives you VSYNC and a stimulus onset that corresponds with when the stimulus actually begins to draw on the screen. The better approach on 10.4 would be to wait for the vertical blank after flushing the buffer, and this would have the same benefits as waiting before flushing on 10.3.9. I don’t want to wait twice, as this would make it logically impossible to show an image for only one frame.

Give me a little bit to play around with the results, and I’ll send you an update.

10

Greg,

In addition to the refresh synchronization updates discussed here, I also made improvements to how OS event-based input methods calculate the actual response times, as I alluded to here: http://community.cedrus.com/showthread.php?t=107 These improvements shipped in SuperLab 4.0.3, so if you get a chance at some point to re-run your experiment, I’m really interested in seeing the results.

Thanks

Note to anyone randomly reading this thread: OS X is very different from Windows. No assumptions should be made about SuperLab’s behavior in Windows based on what we are saying about it here in OS X.

11

We’ve looked at many of the issues raised in our rsearch on millisecond timing accuracy. In short you’re right to check and keep on checking! We’ve now an all-in-one commercial device called the Black Box ToolKit which will do this kind of check more easily and accurately. For information on our published papers, the kit and who’s using it see:

http://www.blackboxtoolkit.com/

Unfortunately the issue of timing inaccuracy isn’t going away any time soon regardless of how fast your PC or MAC etc. is or what experiment generator you use :frowning:

Richard