misc reading notes, mostly from the commute, clean up later

tree of thoughts paper (yao + others, 2023 i think, the one with game of 24) -
generalizes chain of thought into actual search. instead of one linear chain you
keep a tree of partial "thoughts", evaluate each branch, backtrack when stuck.
the game of 24 number is wild: gpt-4 with plain CoT solves like 4% of games,
with ToT it's 74%. deliberate search >>> single forward pass for hard planning
problems. obviously way more expensive per query though, lots of model calls.

reflexion (shinn et al 2023) - agent fails a task, then writes a short verbal
reflection about WHY it failed, stores it in an episodic memory buffer, and gets
the reflection back in context on the next attempt. "verbal reinforcement
learning", no weight updates. got ~91% pass@1 on humaneval with gpt-4. feels
related to the react loop - it's react agents underneath, plus memory.

random idea: the reflection buffer is basically a tiny second brain for the
agent itself?? notes -> memory -> better next attempt. same loop i'm trying to
build for myself here lol

still haven't found a good paper on long-term memory for agents beyond a single
episode. is that a real gap or am i just not searching right
