用 Karpathy 的 autoresearch 方法，将你的 Claude Skills 效果提升10倍

Author: Ole Lehmann Original link: How to 10x your Claude Skills (using Karpathy's autoresearch method)

The author Ole Lehmann (@itsolelehmann) is a highly influential AI content creator and educator, especially in applying AI to business automation and personal productivity. If you're interested in workflow automation, commercial deployment of AI agents, or increasing personal leverage, he is one of the top creators to follow on X right now. His style is extremely pragmatic—he rarely discusses abstract theories and focuses on "hands-on" guidance on configuring AI tools that can help you make money or save time.

The value of this article: you can take any skill, any workflow, any research task—as long as it's measurable and quantifiable—and upgrade it through the autoresearch method, continuously optimizing until its effectiveness becomes ten times what it was. The principle is dead simple: evaluate, optimize, evaluate, optimize… repeat endlessly. The author also provides an open-source link you can use directly. You can even have AI find Karpathy's autoresearch and adapt it.

Your Claude skill lies to you about 30% of the time—and your customer may realize it before you do.

Three months ago, I gave a client a landing‑page copy skill I'd built, swearing it was "absolutely reliable." They needed a launch copy for a new product, and I told them not to worry—I used this skill myself all the time.

That evening, the client sent me a screenshot: the CTA was "Learn More," the headline was "Transform Your Business."

I stared at my phone for three seconds, closed it, reopened it—same screenshot.

At that moment I realized I had no idea when the skill actually worked and when it was just phoning it in.

I built a method that can automatically iterate any skill on autopilot. This article teaches you how to run it yourself.

You start it, and the agent begins testing and refining the skill relentlessly—no manual intervention required.

My landing‑page copy skill went from a 56% quality‑check pass rate to 92%. Zero manual effort.

The agent just kept testing and tightening the prompt on its own.

Here's the complete method, plus the ready‑to‑use skill I built; you can take it and run with it:

P.S. Want more AI workflows like this every week? Follow me.

Where did this method come from?

Andrej Karpathy—co‑founder of OpenAI, former head of AI at Tesla, and the person who coined "vibe coding"—released a method called autoresearch.

The core idea is simple: instead of you making improvements manually, an AI agent does it in a loop for you.

It tries a small change. Checks if the result is better. If it is, keep it; if not, throw it away.

Then do it again. And again.

He originally used it on machine‑learning code, but the method works for anything you can measure and improve.

This brings up an uncomfortable truth:

With all those AI workflows you run every day, do you really know whether they’re "performing well" or just "outputting words"?

Most people can't tell the difference, because no one taught them how.

You’ve taken countless AI courses, learned tons of prompt techniques—but has anyone ever told you how to verify if the thing you built is actually working?

No. Everyone teaches you how to build; nobody teaches you how to test.

I couldn't tell either.

I spent a long time thinking about how this kind of failure happens.

What bothered me most wasn't that client screenshot. It was what happened afterwards—I started counting how many times I thought "it's fine" when it had already quietly gone off the rails.

There's one kind of failure you can’t feel at all. For things the prompt doesn’t explicitly forbid, the model slowly drifts toward "safety"—outputs get vaguer, more template‑like, each one passable but each one a little worse. By the time you notice, you have no idea which iteration started the problem.

Another kind is even worse: you only see the outputs that are "good enough"—you open them, use them, and close them. The ones that silently fail, where formatting slips or key elements go missing, you never know how frequent they are because you never go back and check.

The third kind is self‑deception. You spot a problem now and then, manually tweak that one output, and tell yourself "fixed." But you fixed the one output, not the skill itself. It will fail in the exact same spot next time.

I’ve done all three.

Including with the skills you build inside Claude.

I turned his method into a skill that runs inside Claude Code and Cowork. Whenever you want to use it, just run it on top of another skill.

Say "run autoresearch on my landing‑page skill," and it handles everything else.

How one cycle automatically improves your skill

Picture this.

You have a recipe that works seven times out of ten. The other three times, something goes wrong—maybe the sauce is bland, maybe the seasoning is off.

Instead of rewriting the whole recipe, you swap one ingredient and cook it ten times.

Better? Keep the change.

Worse? Switch back.

Then move to the next ingredient, cook ten times, keep or discard.

After 50 such rounds, your recipe succeeds 9.5 times out of ten.

That’s exactly what autoresearch does to your skill.

The "recipe" is your skill prompt. "Cooking" is running the skill. "Tasting" is scoring the outputs.

The only thing you need to do: provide the scoring criteria

Give the agent a checklist that defines "what good looks like." That’s literally your only job in the whole process.

Define it as a simple yes/no checklist.

Each question checks one specific aspect of the output. Pass or fail—that’s it.

The agent uses this checklist to score every output. That score tells it whether the most recent change is helping or hurting.

Think of a teacher grading with a checklist.

Not "rate the writing quality 1–10" (vague, and the results change every time), but clear yes/no items:

• Did the student state a thesis? Yes or no.
• Did every citation include a source? Yes or no.
• Is the paper within five pages? Yes or no. Grade 100 papers with this checklist and you’ll get consistent results every time.

Here’s what a landing‑page copy checklist might look like:

• "Does the headline contain a specific number or quantifiable result?" (Not "better copy"—but "get your ad spend back in 3 days.")
• "Does the opening sentence name a concrete, painful scenario?" (Not "many people have this problem"—but "you sent the email and got no reply.")
• "Does the CTA clearly tell the user what will happen after they click?" (Not "sign up now"—but "get your analysis report within 3 minutes of signing up.")
• "Does the copy contain any zero‑information words like 'disruptive,' 'industry‑leading,' 'optimal solution'?"
• "Within the first paragraph, does it mention a specific life change the user will experience after getting the result?" You don’t have to invent these yourself. When you launch autoresearch, the agent will guide you through the whole process.

It asks you what "good" means, helping you turn fuzzy feelings into checkable questions. If you have a style guide, just drop it in.

Three to six questions is the sweet spot. Don’t overdo it. I once tried ten, and the skill started gaming the checklist—the output actually got worse, like a student memorizing answers without understanding the material.

Now I have a personal rule: I don’t ship any skill that hasn’t been through autoresearch. Not because I’m a perfectionist, but because I’ve been burned before. I know that kind of confidence is really just ignorance.

How to run it

Step 1: Download the skill. Grab it from here and place it in your skills folder for Claude Code or Cowork.

Step 2: Pick a skill to improve. Say "run autoresearch on my [skill name] skill." Choose the one that gives you the most headaches—the one that’s sometimes brilliant and sometimes flakey.

Step 3: The agent asks you three things. Which skill to optimize, what test input to use (e.g., "write landing page copy for an AI productivity tool"), and what your checklist questions are.

Step 4: It runs your skill once and gives you a starting score.

That’s your baseline. My landing‑page skill started at 56%—vague headlines, buzzword overload, weak CTAs. More than half the checklist items failed.

Step 5: A live dashboard opens in your browser.

You’ll see the score curve, pass/fail for each checklist item, and a log of every change. It refreshes every 10 seconds.

Step 6: Walk away.

The agent begins its loop. It finds the weakest area, tweaks, tests, keeps the change if the score goes up, discards it if it drops.

Then it does it again. And again.

It runs until you tell it to stop, or until it scores above 95% three times in a row.

You can watch, or you can go grab a coffee. It doesn’t need you.

What happened with my landing page skill

I ran autoresearch on my own landing‑page copy skill. The result:

56% → 92%. Four changes, three kept, one reverted.

The agent actually made these changes to my skill prompt:

• Added an explicit rule for the most frequent failure: "Headline must include a specific number or result. Avoid vague promises like 'transform your business.'"

• Added a banned buzzword list: "Never use: revolutionary, cutting-edge, synergy, next-level, game-changing, leverage, unlock, transform."

• Inserted a concrete example of high‑quality landing page copy, highlighting where the pain‑point opener and CTA appear, so the skill could see what good looks like instead of guessing.

• Tried a stricter character limit, then reverted it because the copy became too thin and the CTA suffered. (The system recognized that a change that looked like an improvement in isolation actually hurt the overall output.) In the end I got:

• The improved skill, saved separately (the original stays intact, ready to be restored)

• A log of scores from every round

• A complete changelog: what each change was, why the agent made it, and the result

• A backup of the original skill That changelog might be the most valuable part. It’s a complete "lessons learned" for this skill—what works, what doesn’t, crystal clear.

When a stronger model comes along, feed it this changelog and it can pick up right where the last agent left off.

Honestly, after running autoresearch, the biggest shift wasn’t the score. It was the feeling.

Before, every time I delivered a skill, there was a little voice saying, “I hope this one works.” Now it’s different—I know under what conditions it works, when it’s likely to fail, and how to find the problem when it does.

From relying on luck to relying on a system. That’s the real value.

This method scales far beyond skills

Anything you can score, you can run through this method.

Website speed: Someone used it to optimize page load time. Change one thing, measure speed, keep or discard. After 67 cycles, it went from 1100ms to 67ms.

Cold outreach emails: Define your checklist: "Does it mention the prospect's company? Is it under 75 words? Does it end with a specific question?" Let the agent run 50 variants.

Newsletter openings: "Does the opener include a personal detail?" "Is there any cliché?" Let the agent polish your copy on autopilot.

Any prompt you use over and over.

If you can score it, you can autoresearch it.

Give it a spin

Pick your worst‑performing skill, launch autoresearch, and come back to something that’s genuinely solid and reliable.

Download the skill here (uploaded to Dropbox)

Or check out my GitHub

Have you ever been completely confident a skill was working, only to have a client or colleague spot a problem you completely missed?

Share your story in the comments—what went wrong? I read every single one.