Generated Knowledge Prompting (GENKNOW)

LLMs are powerful, but they sometimes fail in embarrassingly simple ways — especially when commonsense reasoning is involved.
Example:

Q: Part of golf is trying to get a higher point total than others. Yes or No?
A: Yes.

That’s wrong. The goal of golf is to minimize strokes, not rack up points.
This isn’t a “math” problem. It’s a knowledge gap. The model is missing a key fact about the world.

---

The GENKNOW Idea

Instead of asking the model to answer the question directly, we first ask it to generate relevant knowledge — then use that knowledge as part of the prompt for the final answer.

This is a two-stage process:

1. Knowledge Generation – Model produces facts or context related to the question.
2. Knowledge Integration – Facts are appended to the question and passed back to the model to produce the final answer.

The hypothesis: reasoning is stronger when the model explicitly states facts before making a judgment.

---

Code Demo: GENKNOW for Commonsense QA

import openai

openai.api_key = "your-api-key"

def generate_knowledge(question):
    """Ask the model for facts related to the question."""
    prompt = f"Generate relevant factual knowledge to help answer the question: {question}"
    
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=150,
        temperature=0.7
    )
    return response["choices"][0]["message"]["content"].strip()

def integrate_knowledge_and_answer(question, knowledge):
    """Ask the model to answer using the generated knowledge."""
    prompt = f"Question: {question}\nKnowledge: {knowledge}\nExplain and Answer clearly."
    
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=150,
        temperature=0.0
    )
    return response["choices"][0]["message"]["content"].strip()

# Example question
question = "Part of golf is trying to get a higher point total than others. Yes or No?"

# Stage 1: Generate knowledge
knowledge = generate_knowledge(question)
print("📄 Generated Knowledge:\n", knowledge)

# Stage 2: Integrate knowledge into answer
final_answer = integrate_knowledge_and_answer(question, knowledge)
print("\n✅ Final Answer:\n", final_answer)

---

What This Code Shows

• Decomposition – Breaks a question into a fact-finding step and a reasoning step.
• Interpretability – The “knowledge” is visible, so you can debug and refine it.
• Domain Adaptability – Swap in domain-specific knowledge prompts for medicine, law, or technical support.

---

📈 Why This Works

By making the model “say the facts out loud” before making a judgment, you:

• Reduce hallucination risk.
• Improve consistency.
• Create opportunities for human review of the generated knowledge before final decisions.

In Liu et al. (2022), this approach improved performance on tasks like commonsense reasoning and multi-hop QA, showing that sometimes the shortest path to the answer is two steps.

---