Knowledge Graphs w/ AI Agents form CRYSTAL (MIT)

Описание: A knowledge graph is a structured representation of information, consisting of entities (nodes) connected by relationships (edges). It serves as a dynamic framework where an AI agent can store, organize, and reason about knowledge.

In this scenario, the AI continuously expands the graph by integrating new information, aiming to create a "knowledge crystal"—a coherent, interconnected system supporting logical reasoning.

all rights w/ authors for referenced parts:
Agentic Deep Graph Reasoning Yields Self-Organizing Knowledge Networks
Markus J. Buehler
‪@mit‬

code available at:
https://github.com/lamm-mit/PRefLexOR
PRefLexOR: Preference-based Recursive Language Modeling for Exploratory Optimization of Reasoning and Agentic Thinking
Apache 2.0 license
A NEW framework by MIT, that combines preference optimization with concepts from Reinforcement Learning (RL) to enable models to self-teach through iterative reasoning improvements. Central to PRefLexOR are thinking tokens, which explicitly mark reflective reasoning phases within model outputs, allowing the model to recursively engage in multi-step reasoning, revisiting, and refining intermediate steps before producing a final output. The foundation of PRefLexOR lies in Odds Ratio Preference Optimization (ORPO), where the model learns to align its reasoning with human-preferred decision paths by optimizing the log odds between preferred and non-preferred responses. The integration of Direct Preference Optimization (DPO) further enhances model performance by using rejection sampling to fine-tune reasoning quality, ensuring nuanced preference alignment. This hybrid approach between ORPO and DPO mirrors key aspects of RL, where the model is continuously guided by feedback to improve decision-making and reasoning. Active learning mechanisms allow PRefLexOR to dynamically generate new tasks, reasoning steps, and rejected answers on-the-fly during training. This adaptive process enables the model to self-teach as it continually improves through real-time feedback and recursive processing.

#knowledgegraph
#airesearch
#reasoning