For the previous decade and a half, I’ve been exploring the intersection of know-how, training, and design as a professor of cognitive science and design at UC San Diego. A few of you might need learn my current piece for O’Reilly Radar the place I detailed my journey including AI chat capabilities to Python Tutor, the free visualization device that’s helped hundreds of thousands of programming college students perceive how code executes. That have obtained me occupied with my evolving relationship with generative AI as each a device and a collaborator.
I’ve been intrigued by this rising follow referred to as “vibe coding,” a time period coined by Andrej Karpathy that’s been making waves in tech circles. Simon Willison describes it completely: “After I speak about vibe coding I imply constructing software program with an LLM with out reviewing the code it writes.” The idea is each liberating and barely terrifying—you describe what you want, the AI generates the code, and also you merely run it with out scrutinizing every line, trusting the general “vibe” of what’s been created.
Be taught sooner. Dig deeper. See farther.
My relationship with this strategy has developed significantly. In my early days of utilizing AI coding assistants, I used to be that one who meticulously reviewed each single line, typically rewriting important parts. However as these instruments have improved, I’ve discovered myself steadily letting go of the steering wheel in sure contexts. But I couldn’t totally embrace the pure “vibe coding” philosophy; the professor in me wanted some high quality assurance. This led me to develop what I’ve come to name “vibe checks”—strategic verification factors that present confidence with out reverting to line-by-line code critiques. It’s a center path that’s labored surprisingly effectively for my private tasks, and right now I wish to share some insights from that journey.
Vibe Coding in Apply: Changing 250 HTML Recordsdata to Markdown
I’ve discovered myself more and more turning to vibe coding for these one-off scripts that resolve particular issues in my workflow. These are usually duties the place explaining my intent is definitely simpler than writing the code myself, particularly for knowledge processing or file manipulation jobs the place I can simply confirm the outcomes.
Let me stroll you thru a current instance that completely illustrates this strategy. For a category I educate, I had college students submit responses to a survey utilizing a proprietary internet app that supplied an HTML export choice. This left me with 250 HTML recordsdata containing beneficial pupil suggestions, nevertheless it was buried in a multitude of pointless markup and styling code. What I actually needed was clear Markdown variations that preserved simply the textual content content material, part headers, and—critically—any hyperlinks college students had included of their responses.
Slightly than penning this conversion script myself, I turned to Claude with an easy request: “Write me a Python script that converts these HTML recordsdata to Markdown, preserving textual content, fundamental formatting, and hyperlinks.” Claude recommended utilizing the BeautifulSoup library (a stable selection) and generated a whole script that will course of all recordsdata in a listing, making a corresponding Markdown file for every HTML supply.
(Looking back, I spotted I most likely may have used Pandoc for this conversion process. However within the spirit of vibe coding, I simply went with Claude’s suggestion with out overthinking it. A part of the enchantment of vibe coding is bypassing that analysis section the place you evaluate completely different approaches—you simply describe what you need and roll with what you get.)
True to the vibe coding philosophy, I didn’t evaluate the generated code line by line. I merely saved it as a Python file, ran it on my listing of 250 HTML recordsdata, and waited to see what occurred. This “run and see” strategy is what makes vibe coding each liberating and barely nerve-wracking—you’re trusting the AI’s interpretation of your wants with out verifying the implementation particulars.
Belief and Danger in Vibe Coding: Working Unreviewed Code
The second I hit “run” on that vibe-coded script, I spotted one thing which may make many builders cringe: I used to be executing fully unreviewed code on my precise pc with actual knowledge. In conventional software program improvement, this is able to be thought of reckless at greatest. However the dynamics of belief really feel completely different with fashionable AI instruments like Claude 3.7 Sonnet, which has constructed up a popularity for producing fairly secure and useful code.
My rationalization was partly primarily based on the script’s restricted scope. It was simply studying HTML recordsdata and creating new Markdown recordsdata alongside them—not deleting, modifying current recordsdata, or sending knowledge over the community. After all, that’s assuming the code did precisely what I requested and nothing extra! I had no ensures that it didn’t embrace some surprising conduct since I hadn’t checked out a single line.
This highlights a belief relationship that’s evolving between builders and AI coding instruments. I’m far more keen to vibe code with Claude or ChatGPT than I’d be with an unknown AI device from some obscure web site. These established instruments have reputations to take care of, and their dad or mum firms have robust incentives to stop their programs from producing malicious code.
That stated, I’d like to see working programs develop a “restricted execution mode” particularly designed for vibe coding situations. Think about with the ability to specify: “Run this Python script, however solely enable it to CREATE new recordsdata on this particular listing, stop it from overwriting current recordsdata, and block web entry.” This light-weight sandboxing would offer peace of thoughts with out sacrificing comfort. (I point out solely proscribing writes fairly than reads as a result of Python scripts usually have to learn varied system recordsdata from throughout the filesystem, making learn restrictions impractical.)
Why not simply use VMs, containers, or cloud providers? As a result of for personal-scale tasks, the comfort of working instantly by myself machine is difficult to beat. Organising Docker or importing 250 HTML recordsdata to some cloud service introduces friction that defeats the aim of fast, handy vibe coding. What I would like is to take care of that comfort whereas including simply sufficient security guardrails.
Vibe Checks: Easy Scripts to Confirm AI-Generated Code
OK now come the “vibe checks.” As I discussed earlier, the good factor about these private knowledge processing duties is that I can typically get a way of whether or not the script did what I meant simply by analyzing the output. For my HTML-to-Markdown conversion, I may open up a number of of the ensuing Markdown recordsdata and see in the event that they contained the survey responses I anticipated. This handbook spot-checking works fairly effectively for 250 recordsdata, however what about 2,500 or 25,000? At that scale, I’d want one thing extra systematic.
That is the place vibe checks come into play. A vibe examine is basically a less complicated script that verifies a fundamental property of the output out of your vibe-coded script. The important thing right here is that it needs to be a lot easier than the unique process, making it simpler to confirm its correctness.
For my HTML-to-Markdown conversion challenge, I spotted I may use an easy precept: Markdown recordsdata needs to be smaller than their HTML counterparts since we’re stripping away all of the tags. But when a Markdown file is dramatically smaller—say, lower than 40% of the unique HTML measurement—which may point out incomplete processing or content material loss.
So I went again to Claude and vibe coded a examine script. This script merely:
- Discovered all corresponding HTML/Markdown file pairs
- Calculated the dimensions ratio for every pair
- Flagged any Markdown file smaller than 40% of its HTML supply
And lo and behold, the vibe examine caught a number of recordsdata the place the conversion was incomplete! The unique script had didn’t correctly extract content material from sure HTML buildings. I took these problematic recordsdata, went again to Claude, and had it refine the unique conversion script to deal with these edge circumstances.
After a couple of iterations of this suggestions loop—convert, examine, determine points, refine—I finally reached a degree the place there have been no extra suspiciously small Markdown recordsdata (effectively, there have been nonetheless a couple of under 40%, however handbook inspection confirmed these had been right conversions of HTML recordsdata with unusually excessive markup-to-content ratios).
Now you would possibly fairly ask: “Should you’re vibe coding the vibe examine script too, how have you learnt that script is right?” Would you want a vibe examine in your vibe examine? After which a vibe examine for that examine? Nicely, fortunately, this recursive nightmare has a sensible answer. The vibe examine script is usually an order of magnitude easier than the unique process—in my case, simply evaluating file sizes fairly than parsing advanced HTML. This simplicity made it possible for me to manually evaluate and confirm the vibe examine code, even whereas avoiding reviewing the extra advanced authentic script.
After all, my file measurement ratio examine isn’t good. It could actually’t inform me if the content material was transformed with the right formatting or if all hyperlinks had been preserved appropriately. But it surely gave me an affordable confidence that no main content material was lacking, which was my major concern.
Vibe Coding + Vibe Checking: A Pragmatic Center Floor
The take-home message right here is straightforward however highly effective: While you’re vibe coding, all the time construct in vibe checks. Ask your self: “What easier script may confirm the correctness of my fundamental vibe-coded answer?” Even an imperfect verification mechanism dramatically will increase your confidence in outcomes from code you by no means truly reviewed.
This strategy strikes a pleasant stability between the velocity and artistic movement of pure vibe coding and the reliability of extra rigorous software program improvement methodologies. Consider vibe checks as light-weight exams—not the excellent take a look at suites you’d write for manufacturing code, however sufficient verification to catch apparent failures with out disrupting your momentum.
What excites me in regards to the future is the potential for AI coding instruments to recommend applicable vibe checks robotically. Think about if Claude or comparable instruments couldn’t solely generate your requested script but additionally proactively provide: “Right here’s a easy verification script you would possibly wish to run afterward to make sure the whole lot labored as anticipated.” I think if I had particularly requested for this, Claude may have recommended the file measurement comparability examine, however having this constructed into the system’s default conduct can be extremely beneficial. I can envision specialised AI coding assistants that function in a semi-autonomous mode—writing code, producing applicable checks, operating these checks, and involving you solely when human verification is really wanted.
Mix this with the form of sandboxed execution setting I discussed earlier, and also you’d have a vibe coding expertise that’s each liberating and reliable—highly effective sufficient for actual work however with guardrails that stop catastrophic errors.
And now for the meta twist: This complete weblog submit was itself the product of “vibe running a blog.” Initially of our collaboration, I uploaded my earlier O’Reilly article,”Utilizing Generative AI to Construct Generative AI” as a reference doc. This gave Claude the chance to research my writing fashion, tone, and typical construction—very like how a human collaborator would possibly learn my earlier work earlier than serving to me write one thing new.
As a substitute of writing all the submit in a single go, I broke it down into sections and supplied Claude with a top level view for every part one by one. For each part, I included key factors I needed to cowl and typically particular phrasings or ideas to incorporate. Claude then expanded these outlines into totally fashioned sections written in my voice. After every part was drafted, I reviewed it—my very own model of a “vibe examine”—offering suggestions and requesting revisions till it matched what I needed to say and the way I needed to say it.
This iterative, section-by-section strategy mirrors the vibe coding methodology I’ve mentioned all through this submit. I didn’t want to put in writing each sentence myself, however I maintained management over the course, messaging, and remaining approval. The AI dealt with the execution particulars primarily based on my high-level steering, and I carried out verification checks at strategic factors fairly than micromanaging each phrase.
What’s significantly fascinating is how this course of demonstrates the identical rules of belief, verification, and iteration that I advocated for in vibe coding. I trusted Claude to generate content material in my fashion primarily based on my outlines, however I verified every part earlier than shifting to the subsequent. When one thing didn’t fairly match my intent or tone, we iterated till it did. This balanced strategy—leveraging AI capabilities whereas sustaining human oversight—appears to be the candy spot for collaborative creation, whether or not you’re producing code or content material.
Epilogue: Behind the Scenes with Claude
[Claude speaking]
Trying again at our vibe running a blog experiment, I ought to acknowledge that Philip famous the ultimate product doesn’t totally seize his genuine voice, regardless of having his O’Reilly article as a reference. However in line with the vibe philosophy itself, he selected to not make investments extreme time in infinite refinements—accepting good-enough fairly than good.
Working section-by-section with out seeing the complete construction upfront created challenges, just like portray components of a mural with out seeing the entire design. I initially fell into the entice of copying his define verbatim fairly than remodeling it correctly.
This collaboration highlights each the utility and limitations of AI-assisted content material creation. I can approximate writing types and develop outlines however nonetheless lack the lived expertise that offers human writing its genuine voice. The most effective outcomes got here when Philip supplied clear course and suggestions.
The meta-example completely illustrates the core thesis: Generative AI works greatest when paired with human steering, discovering the proper stability between automation and oversight. “Vibe running a blog” has worth for drafts and descriptions, however like “vibe coding,” some type of human verification stays important to make sure the ultimate product really represents what you wish to say.
[Philip speaking so that humans get the final word…for now]
OK, that is the one half that I wrote by hand: My parting thought when studying over this submit is that I’m not happy with the writing high quality (sorry Claude!), but when it weren’t for an AI device like Claude, I’d not have written it within the first place as a consequence of lack of time and vitality. I had sufficient vitality right now to stipulate some tough concepts, then let Claude do the “vibe running a blog” for me, however not sufficient to totally write, edit, and fret over the wording of a full 2,500-word weblog submit all on my own. Thus, identical to with vibe coding, one of many nice joys of “vibe-ing” is that it tremendously lowers the activation vitality of getting began on artistic personal-scale prototypes and tinkering-style tasks. To me, that’s fairly inspiring.