https://arxiv.org/html/2603.07267v2

also: https://huggingface.co/Jackrong/Negentropy-claude-opus-4.7-9B-GGUF

"Based on current public information, commercial models like OpenAI’s GPT series and Anthropic’s Claude series have clearly hidden their true internal reasoning chains. What we ultimately see through APIs or frontend interfaces are often just “Reasoning Bubbles”—highly compressed and summarized versions of the original massive reasoning content. For small models aiming to improve capabilities through data distillation, these overly compressed reasoning chains fail to provide sufficient step-level learning signals. On the contrary, because the logical leaps are too large and intermediate derivations are missing, forcing small models to learn these summaries directly often leaves them confused and unable to master true reasoning abilities.

Negentropy-claude-opus-4.7-4B is a 4B-level lightweight reasoning-enhanced model. Its name is derived from Negentropy in information theory, symbolizing the reconstruction of highly ordered, logically rigorous reasoning chains from compressed and fragmented information.

Unlike other distillation models in the community that perform simple output distillation from closed-source commercial models, this model employs the cutting-edge “Trace Inversion” technique: using the Trace-Inverter-4B model to restore the compressed “Reasoning Bubbles” leaked by Claude-Opus-4.7 in public interfaces into full Chains-of-Thought (CoT), using them as high-value supervisory signals for fine-tuning.

Negentropy Theory: Why Do We Need “Inversion”? (Introduction)
In the current model distillation paradigm, we face a massive “Information Entropy” trap:
Commercial Black Boxes: Models like Claude-Opus or GPT-5.5 often provide only the final answer (Answer-only) or extremely compressed summaries (Reasoning Bubbles) via API. Information Loss: This compression process significantly increases the data’s “entropy,” causing small models to learn only “imitated conclusions” rather than “reasoning logic” due to the lack of explanations for intermediate logical leaps.

The core idea of Negentropy-4.7-4B is to expand these compressed signals back into detailed, coherent reasoning trajectories through a “Negentropy (reducing disorder)” process. We believe that the essence of reasoning lies in the process, not the result. Thinking chains completed through inversion techniques provide small models with higher-quality “logical gradients.”