Top AI Repos — open-source AI, indexed and scored
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
Top AI Repos tracks AI repositories on GitHub and answers two different questions about each one: is it moving right now, and would you bet a product on it.
The project page for "LOGIC-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning"
| Date | Stars |
|---|---|
| 2026-07-31 | 404 |
| 2026-08-03 | 404 |
| 2026-08-06 | 404 |
Today
— stars today
This week
— stars this week
This month
— stars this month
Momentum
0.0
growth rate 0.00%/day
# Logic-LM
Data and Codes for ["LOGIC-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning"](https://arxiv.org/abs/2305.12295) (Findings of EMNLP 2023).
Authors: **Liangming Pan, Alon Albalak, Xinyi Wang, William Yang Wang**.
[NLP Group](http://nlp.cs.ucsb.edu/), University of California, Santa Barbara
## Introduction
Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems. This paper introduces a novel framework, **Logic-LM**, which integrates LLMs with symbolic solvers to improve logical problem-solving. Our method first utilizes LLMs to translate a natural language problem into a symbolic formulation. Afterward, a deterministic symbolic solver performs inference on the formulated problem. We also introduce a self-refinement module, which utilizes the symbolic solver's error messages to revise symbolic formalizations. We demonstrate Logic-LM's effectiveness on five logical reasoning datasets: ProofWriter, PrOntoQA, FOLIO, LogicalDeduction, and AR-LSAT. On average, Logic-LM achieves a significant performance boost of 39.2% over using LLM alone with standard prompting and 18.4% over LLM with chain-of-thought prompting. Our findings suggest that Logic-LM, by combining LLMs with symbolic logic, offers a promising avenue for faithful logical reasoning.
First, install all the required packages:
```bash
pip install -r requirements.txt
```
## Datasets
The datasets we used are preprocessed and stored in the `./data` folder. We evaluate on the following datasets:
- [ProntoQA](https://github.com/asaparov/prontoqa): Deductive resoning dataset. We use the 5-hop subset of the *fictional characters* version, consisting of 500 testing examples.
- [ProofWriter](https://allenai.org/data/proofwriter): Deductive resoning dataset. We use the depth-5 subset of the OWA version. To reduce overall experimentation costs, we randomly sample 600 examples in the test set and ensure a balanced label distribution.
- [FOLIO](https://github.com/Yale-LILY/FOLIO): First-Order Logic reasoning dataset. We use the entire FOLIO test set for evaluation, consisting of 204 examples.
- [LogicalDeduction](https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/logical_deduction): Constraint Satisfaction Problems (CSPs). We use the full test set consisting of 300 examples.
- [AR-LSAT](https://github.com/zhongwanjun/AR-LSAT): Analytical Reasoning (AR) problems, containing all analytical logic reasoning questions from the Law School Admission Test from 1991 to 2016. We use the test set which has 230 multiple-choice questions.
## Baselines
To replicate the **Standard-LM (Direct)** and the **Chain-of-Thought (CoT)** baselines, please run the following commands:
```bash
cd ./baselines
python gpt3_baseline.py \
--api_key "Your OpenAI API Key" \
--model_name "Model Name [text-davinci-003 | gpt-4]" \
--dataset_name "Dataset Name [ProntoQA | ProofWriter | FOLIO | LogicalDeduction ｜ AR-LSAT]" \
--split dev \
--mode "Baseline [Direct | CoT]" \
--max_new_tokens "16 for Direct; 1024 for CoT" \
```
The results will be saved in `./baselines/results`. To evaluate the results, please run the following commands:
```bash
python evaluate.py \
--dataset_name "Dataset Name [ProntoQA | ProofWriter | FOLIO | LogicalDeduction ｜ AR-LSAT]" \
--model_name "Model Name [text-davinci-003 | gpt-4]" \
--split dev \
--mode "Baseline [Direct | CoT]" \
```
## Logic Program Generation
To generate logic programs for logical reasoning problems in each dataset, at the root directory, run the following commands:
```bash
python models/logic_program.py \
--api_key "Your OpenAI API Key" \
--dataset_name "Dataset Name [ProntoQA | ProofWriter | FOLIO | LogicalDeduction ｜ AR-LSAT]" \
--split dev \
--model_name "Model Name [text-davinci-003 | gpt-4]" \
--max_new_tokenExcerpt of 7,632 characters
Read on GitHubWould you bet a product on this? Bounded 0–100 and slow moving.
matched fp:1b5b1beca854e5eb, llm:Repository description: 'LOGIC-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning' (no topics, language C).
matched fp:1b5b1beca854e5eb, llm:Repository description: 'LOGIC-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning' (no topics, language C).
matched fp:1b5b1beca854e5eb, llm:Repository description: 'LOGIC-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning' (no topics, language C).