March 23rd, 2011

I screwed up!

When dealing with reams of complicated data, it’s all too easy to make a mistake, which is why a crucial part of all scientific analysis is cross-checking and auditing results: examining it from different angles to verify that it makes sense. While carrying out a cross-check, I discovered that I had blundered in not one but two ways. First, my light curves were based on averages of average values from the different cameras. The data used for the light curves comes from six different cameras, and for some idiotic reason I had taken average values from each of the six different cameras and averaged them together. This was idiotic for several reasons. For example, suppose that one of the cameras had observed only ten Leonids, while another had seen a hundred; averaging them together would give undue weight to the weaker camera. The average should have been weighted by the number of Leonids seen. Think of it this way: the average value of camera A is obtained by adding up all ten values and dividing by ten; the average value of camera B is obtained by adding up all one hundred values and dividing by one hundred. The correct average is obtained by adding up all 110 values and dividing by 110. Oops.

But there was an even worse blunder in my analysis: different cameras have different sensitivities. Camera B might be 25% more sensitive than Camera A, reporting luminance values 25% higher than luminance values of Camera A. This would distort the light curves and incorrectly amplify the error bars for the light curves. So I came up with the mean value of luminance for all flares of all Leonids reported by each camera, and used that value as an adjustment factor to increase the values from the less sensitive cameras. Here’s one the light curves that resulted, with errors bars entered by hand (because my stupid graphing programs won’t do error bars correctly).

Figure 1: light curve for Leonids of length 21 flares

This light curve is unacceptable, because it shows error bars that grow larger further down the Leonid trajectory. There is no justification for this effect: the data points at the far right have just as many source data points as the ones on the far left. This strongly suggests that there’s a fundamental mistake in my computations. So it’s back to the ol’ drawing board to figure out where I went wrong. Ain’t science fun?