An Earth observation satellite has autonomously identified targets for the first time, utilizing a vision-language model developed by NASA, which may transform space-based sensor capabilities.
In a groundbreaking achievement, an Earth observation satellite has independently located its targets without the assistance of human analysts on the ground. This significant milestone occurred in April and represents the first documented application of a vision-language model (VLM) in space, illustrating the potential for artificial intelligence to revolutionize the functionality and value of space-based sensors.
Traditionally, satellites transmit large volumes of data back to Earth, where analysts utilize machine learning algorithms or their own observations to interpret the information. However, the YAM-9 satellite, developed by Loft Orbital, utilized a software suite from NASA’s Jet Propulsion Laboratory (JPL) that enabled it to determine areas of interest based on natural language queries.
The VLM that facilitated this demonstration, named Gemma 3 and developed by Google DeepMind, is specifically designed for edge computing applications, allowing it to operate on limited hardware situated far from centralized data centers. VLMs integrate the contextual comprehension of large language models with the capability to analyze visual data. For instance, researchers prompted the model to categorize sensor data at the intersection of natural environments and human infrastructure, as well as to identify structures near railway hubs, demonstrating its effectiveness.
This demonstration is noteworthy for two primary reasons. In the immediate future, it promises to enhance the utility of space sensors by performing preliminary data analysis in orbit, which will alleviate the overwhelming amount of raw data that analysts typically need to sift through. In the long-term, it serves as a validation point for implementing larger-scale AI systems in space.
Paul Lasserre, Loft’s head of AI, expressed to TechCrunch that this advancement could lead to continuous monitoring capabilities in space. With a VLM, satellites could be programmed with logic such as monitoring specific borders and reporting any suspicious activities, enabling an interactive dialogue between the satellites and users.
Loft Orbital’s spacecraft are designed primarily as platforms for third-party clients, operating under a business model akin to infrastructure-as-a-service rather than conventional satellite manufacturing. A recent contract involved Loft constructing, launching, and managing six new satellites for EarthDaily, which aims to analyze and market the data gathered by these satellites. The YAM-9 was launched in the fall of 2025 as a precursor to Loft’s orbital AI initiatives and is equipped with a Nvidia Jetson Orin AGX GPU, a leading chip for space computing.
Leading the development of NAVI-Orbital, the software package that served as the interface for the Gemma 3 VLM, was Juan Delfa Victoria from NASA JPL’s AI division. While Gemma 3 is commercially available, software engineers had to optimize the software package to minimize its library and memory requirements.
Although this marks the inaugural use of a VLM in orbit, it is anticipated that other companies will soon replicate this approach. Planet Labs operates satellites equipped with Jetson Orin processors, currently utilizing them for simpler object detection tasks. A spokesperson indicated that research is in progress on deploying additional AI capabilities, including VLMs.
Kepler Communications, which manages the largest collection of GPUs in space, refrained from disclosing whether VLMs have been implemented in their satellites due to non-disclosure agreements with partners. However, they acknowledged several undisclosed applications of their computing environment since launching their spacecraft in January.
Lasserre noted that now that the concept has been validated, the focus will be on expanding the satellite constellation to provide real-time coverage across the globe. He estimated that achieving this would require deploying between 50 and 100 satellites similar to YAM-9, while Loft currently operates 12 satellites in orbit.
Insights gained from deploying these smaller models in space will inform future efforts to establish larger-scale computing frameworks in space, particularly regarding essential aspects like power and memory management.
The development may also lead to innovative scientific tools. The NAVI-Space concept emerged from discussions between Delfa Victoria and JPL researcher Taran Cyriac John about creating digital assistants for astronauts on missions to the moon or Mars. Delfa Victoria explained the challenges astronauts face, stating, 'You have astronauts with pressurized suits who cannot be tapping on a keyboard; whatever they want to do is complex.' He proposed the idea of an interactive AI assistant, similar to those depicted in video games and films.
