AI Can Rebuild Software by Testing Its Behavior Against the Original

AI systems can now recreate software without access to its source code. They can inspect compiled programs or observe what an application does, build a replacement, compare its behavior with the original, and revise it until the differences shrink. That process is beginning to produce playable game clones and reproductions of tools such as Photoshop.

The claim that “software is over” goes too far, Károly Zsolnai-Fehér says. But the underlying capability is advancing quickly—and the research suggests that both functionality and appearance can be reverse-engineered.

The legal boundary is not simply whether a program is copied. A quotation from Sega v. Accolade shown in the presentation says that disassembly can be fair use when it is the only way to access a program’s ideas and functional elements and there is a legitimate reason for doing so. The decision did not shield Accolade’s finished products from a separate infringement claim. Zsolnai-Fehér cautions that he is not a lawyer, but says courts have treated software functionality as not necessarily protected by copyright. Models, textures, and characters, by contrast, can be copyrighted.

One piece of research, MirrorCode, tested whether AI could reconstruct entire software projects without source code. The presenter reports that, on one program, the system initially passed only 15% of the tests. Over nine months, that result rose to nearly 100%. Zsolnai-Fehér says the models behind these results are older and that newer ones do better still.

The source describes a broader approach in which an AI can reproduce a program by observing its behavior, then find differences and keep improving its replacement. The MirrorCode result illustrates how much performance on a particular test changed over time; the presentation does not give further details about that evaluation.

A separate study, REMEND, pushed reverse engineering down to a much smaller model. It used a 12-million-parameter model to recover mathematical equations from binary executables, reaching around 90% accuracy on a specifically constructed test. The example shown was a program whose behavior corresponded to the curve y = sin(2x)e^(-x/3). The point is not only that a model can imitate outputs: it can infer a rule that accounts for them.

Visual reproduction presents a related but distinct problem. In WebUI-95, research on rebuilding webpages from their appearance, the reconstructed pages reached a reported visual similarity score of 0.938—described as nearly identical layouts and visual elements—while using 95% less code than the originals. Zsolnai-Fehér suggests one reason for the reduction: the reconstruction may omit advertising and tracking. A comparison graphic shows the original and reconstructed page sizes.

The core advantage is that a target program can serve as its own reference. Training data can come from open-source repositories: pair the code with the working application, and a model has examples of how programs map to behavior. When asked to reproduce new software, the system can then use the original as a “ground truth oracle.”

The method resembles trial and error. The AI produces a first version; the original can be run to see whether the replacement is close or far away. The model adjusts and tries again. Because the original can be tested repeatedly, the feedback loop can continue until the behavior matches more closely.

That loop helps explain why these systems can improve without being handed the source code for the target. They are not working from a single observation and guessing once. They can repeatedly compare their attempt with the thing they are trying to reproduce.

Frontier labs are also drawing boundaries around what their models will help build. A system prompt attributed to Claude 5.5 Opus, and described as confirmed by Anthropic, instructs the model to build original games or games based on similar mechanics, but not to recreate a specific existing game’s name, characters, art, music, levels, or look—even if the result is renamed or presented as original.

Some developers, Zsolnai-Fehér says, reported refusals on legitimate reverse-engineering tasks as well. One complaint shown on screen came from a user trying to keep using an unsupported printer after its manufacturer stopped selling cartridges and the device rejected third-party ones. The user argued that blocking reverse engineering could prevent people from extending the life of devices that no longer receive updates or support.

Zsolnai-Fehér sees that tension as an argument for owning the compute and using free, local models that cannot be withdrawn by a provider. In his view, safeguards against copying can also limit repair and other uses that are not about reproducing a commercial game.

Zsolnai-Fehér expects it to become plausible to run personal versions of Photoshop or Final Cut Pro, tailored to an individual’s preferences. The open question is how valuable such software will be in settings where people need to collaborate. Someone working at a graphic-design company may need fluency with standard tools such as Adobe’s applications; a home-built alternative that nobody else uses may not transfer well.

On the other hand, if people can complete the work and collaborate through interchangeable file formats, they may have more reason to use tools built for their own needs. Those homebrew programs could become widespread and cost little or nothing, he suggests. The promise is not that established software disappears, but that making a capable alternative may become far less difficult—and that the usefulness of an alternative will depend partly on whether others can work with it.