Oct 30, my halloween riddle for Mag & Owen:

Q: What's an airplane's favorite holliday?
A: Halloween, because it gets to be in disguise


Nov 3, on the way to work (w/ Owen heading to childcare)

Owen: Dad, what's an airplane's favorite holliday?
Me:  I don't know -- what is it?
Owen: Halloween!  because it gets to dress up!

More E-PL3 noise results...

Last night, I took some more dark frames to try to characterize the noise.  Selecting only the "warm" pixels from the 60s dark frame that I analyzed in the last post, I looked at the signal distribution from those same pixels in a sequence of dark frames with exposures from 30 seconds to 1 second.  The first figure shows that the mean and width of the warm pixel distribution scales with exposure time, and that (based on the 30s exposure) the selected pixel population comprehensively captures the pixels making up the warm pixel population in all exposures.  In other words, the problem pixels are a fixed population.  Furthermore, with this data set, I find that the mean of the warm pixel distribution is linear in time (see 2nd figure).

Warm pixel signal (selected from 60s dark frame) for a sequence of shorter exposure dark frames.  The blue histogram is the full distribution (all pixels) from the 30 s dark.  Note that the selected pixels from the 30 s fill the anomalous noise peak of the full distribution.  This shows that the problem pixels are a fixed population.

The other quick experiment I did was to take three 60 s darks in sequence.  This time, I'm showing the full distributions with the Y-axis log scaled.  Between the first and second dark frames shown here, I took a few shorter dark frames.  The second and third are taken directly sequentially.  The data here shows that the mean of the warm pixel distribution (dark current) is temperature dependent.  This is not a surprising result, except that for the bulk of the pixels, the signal stays put at ~64 ADU, though the main peak does broaden for the later frames.

The upshot of this analysis is that "noise reduction" via subtraction of a dark frame taken immediately after the image is the best way to combat this noise.  The 1-30 second sequence indicates that this becomes important for exposures longer than 8 seconds, though this should be taken as a temperature-dependent statement.

Another interesting note (not shown here, so you'll have to take my word for it).  Within the warm pixel distribution, the noise appears to be uncorrelated.  That is, if I subtract the signals from two frames, the width of the difference's distribution is the quadrature sum of the two distributions.

One more interesting note:  I would expect the variance of the distributions to scale linearly in time, but it does not.  It's faster than linear.  I can't explain right now.

Long exposure noise analysis inspired by the E-M1 noise problem

(Peculiarly, blogspot only shows previous posts, but not subsequent ones.  Here is the follow on to this post: more epl3 noise results)


The noise issue that has come to light recently with the Olympus OM-D E-M1 in long exposures caught my attention last night, so I decided to take a look at the output from my camera, an Olympus E-PL3.  While the E-M1 and E-M5 use a 16 Mpix Sony sensor, my camera has a Panasonic 12 Mpix sensor.  In both cases, the technology is CMOS, not CCD.

So far, I've only taken one image and reduced the data from that single frame:

• 60 sec exposure
• ISO 200
• IS off
• Noise reduction OFF
• Noise filter OFF (not sure if this matters w/ raw image)
• f/22, lens capped to minimize stray light. 

I converted the raw file into a FITS file using dcraw
% dcraw -D -4 -c file.orf | pnmtofits > file.fits
and then imported the file into ipython using the astropy library.

On my sensor, rows 4056 to the end have some overscan garbage, so data in those rows were removed from further analysis.  In the first figure, below, I histogrammed the noise from the sensor.  The main peak is at 64 ADU and does not extend much beyond 100 ADU (top panel).  In the middle panel, the Y scale is expanded, and the second noise peak is evident.  This might be analogous to the noise seen in the E-M1.  The bottom panel shows the noise binned into 50 ADU chunks.  All of the information in the top two panels fits into the 5 left-most bars of this histogram.  There are two more noise peaks, at 500 and 1200 ADU.

I found 93 "high count" pixels, mapped below.  Looking at their spatial distribution and density, I think they are unlikely to cause any imaging problems.

The second noise peak, is more interesting.  It peaks at 179 ADU and is done by ~230 ADU.  There are 37000 pixels in the 130-230 ADU range.  As you can see below, their distribution is (at least by eye) uniform and the worst offenders tend to cluster in the corners.

 

I have no idea if this is the same problem that the E-M1 has, but if anyone wants to send me some E-M1 dark frames, I'd be happy to run the same analysis on them.  I'll probably take a few more frames to verify my hunch that this is a fixed-pattern problem, which is why noise reduction can kill it off easily.

NuSTAR first light

The press release, with the pointing and metrology-corrected data with two orbits of integration can be found here. In this post, you get to see the instant-gratification version of the first light image.
NuSTAR consists of a pair of co-aligned x-ray telescopes. It's sort of like the Chandra and Newton observatories, but also different because it has graded multilayer mirrors that allow it to reflect much higher energy photons than those other two x-ray telescopes (they reflect up to ~15 keV, while NuSTAR is good up to 80 keV). My graduate work, which I started in 1995, involved making the algorithm for multilayer design optimization. The code I developed for the balloon-borne HEFT mission's mirrors was also used for NuSTAR's optics.
Another difference between NuSTAR and Chandra & Newton is the detectors -- NuSTAR uses CdZnTe detectors to catch the higher energy x-rays. The energies we can see with NuSTAR would pass right through Chandra's and Newton's Si CCDs.
Now, under "normal" circumstances, we (the NuSTAR team) get a 12 minute ground station pass every ~90 minutes, and the data processing pipeline takes ~30 minutes to turn packets of data into something comprehensible to astronomers. I was looking for instant gratification, so I talked to Rick Cook (who designed the NuSTAR readout ASICs) about getting a text output map of the focal planes during the ground station pass. We needed it to be as fast as possible, so Rick decided to pack two pixels per ascii character, allowing 3 bits per pixel (ASCII characters are 7 bits). I am a big logarithmophile, so I probably pushed for some form of log scaling. Rick gave me a scheme where 0 counts = 0, and otherwise, the output value is [floor(log_2(counts))+1]. I wrote the program to decode Rick's output stream in real time to generate an log-scaled image of the intensity on the detector. So this is what we saw in the instrument room at Space Sciences Lab, Berkeley about one minute before we lost ground station contact:
Note that this is just a 5 second integration with no pointing/metrology corrections applied (though one may argue that an integration that short shouldn't need much correction. The press release has a much nicer looking image, but this isn't really all that different. In the PR, the scaling is [counts^2], which emphasizes the peakiness of the distribution. Log scaling always shows you all the warts, and as you can see, NuSTAR is (knock on wood) pretty wart-free!

Rhi....No way!

Owen had a rhino on his onesie. He called it something else, so I told him it's a rhinoceros. He tried to say the word, but couldn't, so I told him it's a rhino. After a few rhi rhi rhi tries, he says to me, "Help!" I tell him "Rhi" "No". For some reason, he can't have "no" at the end of a word, so he says "rhi....norhi" whch then morphs into "rhi...no way."

Reading, for real

This past Thursday night (8/12), Mag reads "Go, Dog, Go" cover-to-cover with no help. Then she reads the last few sentences of a printed-out email from Mischa. The girl is verifiably reading! A lot!

"It sounds like he's going to sing."

-- Mags, upon hearing a snippet of Obama's (hawkish) Nobel acceptance speech