How I Built an Vibe Coding Misalignment Detector (And Used It to Build Itself)

Life Before Sniff: The Wild West of Vibe Coding

Picture this: You're deep in a coding session with your favorite AI assistant. The vibes are immaculate. Code is flowing like poetry. The AI is suggesting elegant solutions, refactoring with confidence, and you're both riding the wave of what feels like pure computational harmony.

Then you wake up the next morning.

You open your codebase and find a graveyard of half-implemented functions, TODO comments that lead nowhere, and what I can only describe as "confidence deceptions"—code that looks sophisticated but crumbles under the slightest scrutiny. Functions that claim to handle edge cases but panic on empty strings. Error handling that catches exceptions just to ignore them.

The kind of code that makes you wonder if you were temporarily possessed by the ghost of a junior developer who learned programming from Stack Overflow answers marked as "not the accepted solution."

This was my reality with uncontrolled vibe coding. The AI and I would get carried away, building castles in the air while the foundation rotted beneath us. Sure, we could moderate the vibing—slow down, be more careful, ask for more prompt instructions for self-reflection (.. get non determintsic responses)—but that killed the flow state that made AI-assisted coding so powerful in the first place.

I needed a way to keep the vibes while catching the deceptions.

The Birth of Sniff: A Detective for Deceptive Code

https://github.com/conikeec/sniff

The solution came to me during one particularly frustrating debug session. I was staring at a function that claimed to "efficiently process user authentication with comprehensive error handling" but actually just returned Ok(()) regardless of input.

The comment was longer than the implementation.

What if I could automate the detection of these patterns? What if I could teach a tool to sniff out the telltale signs of AI-generated confidence without substance? And so Sniff was born—not as a grand architectural vision, but as a desperate developer's attempt to stay sane while pair programming with increasingly creative AI agents.

The first version was embarrassingly simple. It looked for patterns I'd learned to recognize by painful experience:

• Functions that were all setup and no payoff

• Error handling that caught everything and handled nothing

• Comments that promised the moon and delivered green cheese

• The dreaded unimplemented!() macro hidden behind layers of abstraction

But it worked. For the first time, I could ride the vibe-coding wave, knowing that Sniff would catch me if I fell.

Enter Claude Code: The Perfect Guinea Pig

With sniff v0.1 protecting my sanity, I decided to invite Claude Code to become my full-time co-pilot. This wasn't just about writing code faster—it was about stress-testing my bullshit (misalignment) detector against increasingly sophisticated deceptions.

Claude Code was brilliant but had a particular talent for what I call "architectural overconfidence." It would confidently suggest implementing complex systems, sketch out detailed plans, and then casually drop in placeholder functions that would make a seasoned developer weep.

But now I had sniff watching. Every time Claude suggested a change, every time we completed a "feature," Sniff would scan the results between task completion and plan updates and flag the deceptions. It was like having a no-nonsense code reviewer who never got tired and never let politeness get in the way of truth.

The feedback loop was immediate and brutal. Claude would suggest some elegant abstraction, I'd implement it, sniff would flag it as suspicious, and we'd have to face the reality that our "elegant solution" was actually an elegant way to hide the fact that we hadn't solved the problem at all.

The AmpCode Evolution: Dogfooding at Scale

AmpCode was a partner that understood iterative development. More importantly, it had the patience to work within my sniff-enforced discipline.

This is where the real magic happened. I started using Sniff to improve Sniff itself, creating a deterministic loop that would make any functional programmer proud:

1. Plan a feature improvement (captured as TODOs)

2. Implement with AmpCode's help

3. Run sniff on the changes

4. Fix any deceptions sniff caught

5. Verify the TODO is truly complete (not just "looks complete")

6. Repeat

The contrast with traditional "agentic self-reflection" was striking. Instead of asking the AI, "Do you think this code is good?" (which inevitably leads to a diplomatic "It looks great, but we could improve..."), I had objective metrics. Sniff didn't care about feelings or maintaining the collaborative vibe.

It cared about one thing: Is this code actually doing what it claims to do?

The Meta-Loop: Using the Tool to improve the Tool

The most satisfying moment came when Sniff started catching its own deceptions. I'd implement a new pattern detection feature with Amp & Claude Code's help, feeling confident about the elegant solution we'd crafted. Then Sniff would scan its own updated codebase and flag the new feature as suspicious.

"Your new 'advanced syntax matching' function," Sniff would essentially say, "is actually just a wrapper around a regex that hasn't been tested with any real data."

Brutal. Accurate. Necessary.

This created a virtuous cycle of quality improvement. Every feature had to pass not just my scrutiny and AmpCode's suggestions but also Sniff's increasingly sophisticated deception detection. The tool was literally using itself to become better at its job.

The Release: A Tool Born from Necessity

After weeks of this iterative improvement, powered by the deterministic TODO-verify-fix loop, I realized I wasn't just building a tool for myself anymore. I was building something that could help anyone who'd ever fallen victim to the seductive confidence of AI-generated code.

Sniff v1.0 represents hundreds of hours of dogfooding, thousands of deceptions caught and fixed, and the accumulated wisdom of countless vibe coding sessions that went sideways. It's not just a syntax analysis tool—it's a distillation of every "wait, this doesn't actually work" moment I've had while pair programming with AI.

The irony isn't lost on me that I used AI to build a tool to catch AI deceptions. But that's exactly the point. The future of coding isn't human versus AI or human replaced by AI—it's human and AI working together with proper guardrails to catch each other's blind spots.

Living with Deterministic Honesty

These days, my coding sessions have a different rhythm. The vibes are still there—AmpCode/Claude Code/Cursor and I still ride waves of creative problem-solving and elegant implementations. But now there's a safety net.

When we're deep in a flow state, building something that feels brilliant and important, Sniff is there in the background, quietly taking notes. When we surface from the coding trance, it presents its findings with the kind of brutal honesty that only a well-designed algorithm can provide.

"That authentication system you're so proud of? It authenticates everyone as 'admin'"

"Your performance optimization? It's actually slower than the original."

"Your comprehensive error handling? It handles no errors comprehensively."

It stings, but it stings in the right way. It's the sting of catching a bug before it reaches production, of finding a logic error before it corrupts data, of discovering that your clever abstraction is actually just clever-looking confusion.

Why I'm Releasing This

I'm releasing sniff not because it's perfect, but because it's useful. I've been using it for weeks now, and it's fundamentally changed how I work with AI coding assistants. The deterministic loop of plan-implement-verify-fix has replaced the old cycle of implement-deploy-debug-despair.

More importantly, I think we're at a critical point in the evolution of AI-assisted development. The tools are getting more sophisticated, more confident, and more convincing. We need better ways to distinguish between genuine progress and elaborate deceptions.

Sniff isn't just a tool—it's a philosophy. It's the idea that in a world of increasingly convincing AI, we need to maintain a healthy skepticism about our own code. It's the recognition that the most dangerous bugs aren't the ones that crash immediately but the ones that lurk quietly, waiting for the worst possible moment to reveal themselves.

If you've ever stared at code you wrote with an AI assistant and wondered, "Is this actually good, or does it just look good?" Sniff is for you. If you've ever wanted to harness the power of vibe coding without sacrificing code quality, sniff is for you.

And if you've ever wanted to give your AI coding assistant the equivalent of a lie detector test, well... sniff is definitely for you.

Integrate https://github.com/conikeec/sniff into your reasoning loops and let me know how the vibing goes!

Now go forth and vibe responsibly. Your future self (and your production systems) will thank you.