LLM Reasoning Playbook

Decision Log & Template

A per-task record of which stack you chose and why, plus the workflow for keeping the playbook current.

This turns the playbook from something you read into something you use: a short record, one per task, of which framework or stack you picked, why, and how it went. It's quick to fill in, and it builds into your own evidence base over time. It also holds the routine for keeping the playbook up to date.

When to log. Any time you commit a real prompt design to a task or product. Skip it for quick throwaways.

Copy-paste entry template

markdown### [YYYY-MM-DD] {{Task / feature name}}

**Task.** <one-line description of what the LLM must do>
**Constraints.** latency budget / cost ceiling / accuracy bar / has-verifier?

**Axis walk (4 gates → framework):**
- Q1 external info/tools/compute? → <answer → axis C? or continue>
- Q2 large/messy context?         → <...>
- Q3 explore/merge/decompose?     → <...>
- Q4 dominant secondary need?     → <...>

**Chosen stack.** <e.g., Step-Back × ReAct × Self-Consistency(N=7)>
**Alternatives rejected.** <framework — why not (cost / mismatch / overkill)>

**Result.** shipped? · eval score (from [[Reasoning-Framework-Eval-Rubrics]]) ·
measured cost/latency vs. budget
**Lesson.** <what you'd change next time — feeds the anti-pattern gallery>

Worked example entry

markdown### [2026-07-01] Vendor security-posture summarizer

**Task.** Summarize security posture across 6 audit reports; flag contradictions.
**Constraints.** No hard latency bar; factual accuracy critical; sources provided.

**Axis walk:**
- Q1 external info? → No, all sources in-context.
- Q2 large/messy context? → YES → Thread of Thought.
- Q3 explore/merge? → No.
- Q4 secondary need? → Factual stakes high → add Chain-of-Verification.

**Chosen stack.** Thread of Thought × Chain-of-Verification.
**Alternatives rejected.** Plain CoT (loses mid-context evidence); Self-Consistency
(no discrete answer to vote on).

**Result.** Shipped. Eval 11/12 (Efficiency 1 — verify pass verbose). Latency fine.
**Lesson.** Draft overstated a claim ("fully remediated"); CoVe caught it. Add
"quote the source verbatim" to the thread phase to prevent paraphrase drift.

Log

<!-- Newest first. Append entries above this line using the template. -->

Keep-the-playbook-current workflow

The playbook goes stale if new techniques and your own lessons never flow back in. Keep it fresh with a light routine:

  1. After each decision — once you've written a Lesson, ask whether it's a

mistake that could happen again. If so, add a case to Reasoning-Framework-Anti-Pattern-Gallery.

  1. Monthly (5 minutes) — skim your recent entries. If one framework keeps

losing, revisit its row in the choice matrix.

  1. When you find a new technique — before adding it to

Advanced-AI-Reasoning-Framework-Playbook, place it on one of the 7 axes. If it doesn't fit any axis, it's either a genuinely new axis (rare) or a variant of something you already have (common) — file it as Tier-2 unless it earns the full write-up.

  1. Promotion rule — a Tier-2 variant moves up to a full Module 1 entry only

after you've logged at least 2 real tasks where it was the right first choice.