Because the U.S. Coast Guard (USCG) icebreaker Healy takes part in a voyage across the North Pole this summer, it’s capturing images of the Arctic to further the study of this rapidly changing region. Lincoln Laboratory researchers installed a camera system aboard the Healy while at port in Seattle before it launched into a three-month science mission on July 11. The resulting dataset, which might be one among the primary of its kind, might be used to develop artificial intelligence tools that may analyze Arctic imagery.

“This dataset not only may also help mariners navigate more safely and operate more efficiently, but in addition help protect our nation by providing critical maritime domain awareness and an improved understanding of how AI evaluation could be dropped at bear on this difficult and unique environment,” says Jo Kurucar, a researcher in Lincoln Laboratory’s AI Software Architectures and Algorithms Group, which led this project.

Because the planet warms and sea ice melts, Arctic passages are opening as much as more traffic, each to military vessels and ships conducting illegal fishing. These movements may pose national security challenges to the USA. The opening Arctic also leaves questions on how its climate, wildlife, and geography are changing.

Today, only a few imagery datasets of the Arctic exist to review these changes. Overhead images from satellites or aircraft can only provide limited information concerning the environment. An outward-looking camera attached to a ship can capture more details of the setting and different angles of objects, equivalent to other ships, within the scene. All these images can then be used to coach AI computer-vision tools, which may also help the USCG plan naval missions and automate evaluation. In keeping with Kurucar, USCG assets within the Arctic are spread thin and might profit greatly from AI tools, which might act as a force multiplier.

The Healy is the USCG’s largest and most technologically advanced icebreaker. Given its current mission, it was a fitting candidate to be equipped with a latest sensor to assemble this dataset. The laboratory research team collaborated with the USCG Research and Development Center to find out the sensor requirements. Together, they developed the Cold Region Imaging and Surveillance Platform (CRISP).

“Lincoln Laboratory has a superb relationship with the Coast Guard, especially with the Research and Development Center. Over a decade, we’ve established ties that enabled the deployment of the CRISP system,” says Amna Greaves, the CRISP project lead and an assistant leader within the AI Software Architectures and Algorithms Group. “We have now strong ties not only due to USCG veterans working on the laboratory and in our group, but in addition because our technology missions are complementary. Today it was deploying infrared sensing within the Arctic; tomorrow it might be operating quadruped robot dogs on a fast-response cutter.”

The CRISP system comprises a long-wave infrared camera, manufactured by Teledyne FLIR (for forward-looking infrared), that’s designed for harsh maritime environments. The camera can stabilize itself during rough seas and image in complete darkness, fog, and glare. It’s paired with a GPS-enabled time-synchronized clock and a network video recorder to record each video and still imagery together with GPS-positional data.  

The camera is mounted on the front of the ship’s fly bridge, and the electronics are housed in a ruggedized rack on the bridge. The system could be operated manually from the bridge or be placed into an autonomous surveillance mode, through which it slowly pans backwards and forwards, recording quarter-hour of video every three hours and a still image once every 15 seconds.

“The installation of the equipment was a novel and fun experience. As with all good project, our expectations going into the install didn’t meet reality,” says Michael Emily, the project’s IT systems administrator who traveled to Seattle for the install. Working with the ship’s crew, the laboratory team needed to quickly adjust their route for running cables from the camera to the statement station after they found that the expected access points weren’t the truth is accessible. “We had 100-foot cables made for this project just in case of this kind of scenario, which was an excellent thing because we only had a couple of inches to spare,” Emily says.

The CRISP project team plans to publicly release the dataset, anticipated to be about 4 terabytes in size, once the USCG science mission concludes in the autumn.

The goal in releasing the dataset is to enable the broader research community to develop higher tools for those operating within the Arctic, especially as this region becomes more navigable. “Collecting and publishing the info allows for faster and greater progress than what we could accomplish on our own,” Kurucar adds. “It also enables the laboratory to interact in additional advanced AI applications while others make more incremental advances using the dataset.”

On top of providing the dataset, the laboratory team plans to offer a baseline object-detection model, from which others could make progress on their very own models. More advanced AI applications planned for development are classifiers for specific objects within the scene and the power to discover and track objects across images.

Beyond assisting with USCG missions, this project could create an influential dataset for researchers trying to apply AI to data from the Arctic to assist combat climate change, says Paul Metzger, who leads the AI Software Architectures and Algorithms Group.

Metzger adds that the group was honored to be an element of this project and is happy to see the advances that come from applying AI to novel challenges facing the USA: “I’m extremely happy with how our group applies AI to the highest-priority challenges in our nation, from predicting outbreaks of Covid-19 and assisting the U.S. European Command of their support of Ukraine to now employing AI within the Arctic for maritime awareness.”

Once the dataset is obtainable, it can be free to download on the Lincoln Laboratory dataset website.