Teaching a camera not to look at the sun

The previous post covered hearing a drone through the noise of a moving vehicle. Since then, the system we called Falcon 1 has become two products. GW-VNR1, the Acoustic Early Warning Set, is the microphone array and the crew display. GW-VSR1, the Acoustic-Visual Detection Set, adds a visual module, and that visual module is now in testing.

Sound gives the system a warning and a bearing, but it doesn’t give a picture. The visual module is a pan-tilt head carrying two cameras. A long lens acquires and tracks a target, and a shorter lens searches in elevation along the bearing the acoustic array hands it. The module’s whole job is to point at the sky, so sooner or later it is going to point at the sun.

A magnifying glass with a sensor behind it

A camera lens aimed at the sun does what a magnifying glass does to a dry leaf. It collects sunlight across its whole aperture and focuses it into a small image on the sensor.

How strongly the light is concentrated depends almost entirely on the f-number. The sun is about half a degree across, so its image on the sensor is roughly one hundredth of the focal length. Both our lenses sit at f/5.6. At that setting the light reaching the sensor is about 370 times more intense than the sunlight arriving at the lens. With the lenses wide open it would be several thousand times.

Focal length changes the size of the spot but not its intensity. Through the long lens the sun’s image is nearly half a millimetre across, about 300 pixels wide. Through the short lens it is about a third of that. Even at the same intensity, the larger spot runs hotter. A small hot spot can shed heat sideways into the cooler sensor around it, and a large one mostly can’t.

Our sensors have no mechanical shutter. Exposure is controlled electronically, so the sensor is uncovered whenever the lens is. The colour filters and microlenses on its surface are thin polymer layers, and a concentrated sun parked on them long enough will damage them permanently. The failure isn’t dramatic. The sensor keeps working, but it carries a discoloured patch in every frame afterward, sitting in the image the detector is trying to read.

Why not filter it out

The obvious fixes are optical, and none of them suits us. A neutral density filter cuts the sun’s energy, but it also cuts the light reflecting off a small, distant drone, and stopping the lenses down already costs us light. A solar filter leaves the camera able to see nothing except the sun. A physical shutter adds a moving part to a head that is already slewing hard on a vibrating roof.

Simply never looking toward the sun isn’t acceptable either. Attacking out of the sun is one of the oldest tactics in aerial combat, because a defender looking that way sees nothing. A visual module that can’t tolerate the sun has a blind spot in exactly the direction a sensible attacker would choose. We wanted the module to know where the sun is, keep the long lens from dwelling on it, and keep looking everywhere else.

We settled on two independent layers.

Layer one: the clock

The sun’s position is not a mystery. Given the date, the time and a rough location, its elevation and bearing can be computed to a small fraction of a degree. On a vehicle with no internet and no GPS, the hard input is the time.

A small computer that loses power loses track of time. With no network to correct it, it boots up believing it is whenever it last shut down, and for a vehicle parked over a weekend that could be days out. For most software a wrong clock is cosmetic. For sun avoidance it isn’t. The sun crosses the sky at up to fifteen degrees an hour, so a clock that is an hour wrong puts the computed sun far outside any safety margin.

The fix is a real-time clock with its own battery. A crystal oscillator keeps counting while the rest of the system is off, powered by a small rechargeable lithium coin cell that the computer tops up whenever the vehicle is running. It is the same arrangement that keeps a laptop’s clock right after a month in a drawer.

The clock doesn’t need to be very accurate. Our keep-out zone extends roughly five degrees around the sun, and the sun takes at least twenty minutes to move that far. A crystal clock drifts by seconds a day, so it can run for months before drift matters. It also gets corrected whenever the system does see a trustworthy time source.

The same clock solves a second problem. The acoustic and visual modules run on separate computers, and fusion pairs their detections by timestamp. If the two computers disagree about the time, detections get paired with the wrong events, and nothing reports an error. The battery-backed clock on the acoustic node is the time reference for the whole system, and the visual node synchronises to it over the wired link. One small part gives us both the sun’s position and agreement between modules about when things happened.

What the clock cannot know is which way the vehicle is facing. The computed sun position is relative to true north, but the camera’s pan angle is relative to the hood, and vehicles turn. The clock therefore gives us the sun’s elevation with confidence, but tells us nothing about where the sun sits relative to the vehicle.

Elevation turns out to be most of what matters. The visual module searches a limited band of sky above the horizon. For much of a summer day the sun is above that band and the problem disappears. In the morning, in the evening, and all winter at our latitude, the sun sits inside the band. In December here it never climbs much above twenty degrees. At those times the clock tells us exactly which slice of elevation to treat with care.

Layer two: the image

The second layer is the camera watching for the sun itself. The module already processes every frame, so checking for a large patch of saturated pixels costs almost nothing.

The catch is automatic exposure. As the sun comes into view, the camera darkens the image to compensate, which can hide exactly the saturation we are looking for. The brightness check therefore reads the exposure settings recorded with each frame and judges brightness against them, not against raw pixel values.

The two cameras also look out for each other. The short lens points the same way as the long one and has a field of view several times wider, so the sun appears at the edge of the wide frame well before it reaches the narrow one. The short lens acts as a lookout for the long lens, and because its sun image is much smaller, it is the more tolerant of the two.

Putting them together

Each layer covers the other’s weakness. The clock knows the sun’s elevation before any light reaches a sensor, but not where the sun is relative to the vehicle. The image gives that position immediately, but only once the sun is at or near the frame. When the clock says the sun is inside the search band, the module moves through that band with care. When the wide camera sees the sun, the pan angle at that moment gives its bearing, and the long lens is kept from dwelling inside the keep-out zone until the geometry changes.

The clock knows where the sun is in the sky. The camera knows where it is relative to the vehicle.

The cost is a few degrees of sky where the long lens won’t linger. That is not a blind spot in the warning. As an earlier post argued, the camera sits outside the warning path. The acoustic array hears a drone coming out of the sun as well as one coming from anywhere else, and it still produces a bearing. What the keep-out costs is visual confirmation in a small patch of sky for as long as the sun is there.