Modern AI tools can produce polished prose and clean-looking code, but polish is not proof of alignment. In this post, I argue that we should treat LLMs as an alien mind: useful, fast, and often insightful, yet fundamentally opaque. The safest path is applied paranoia: distrust, test, verify, and never ship what you do not understand.

Introduction

Large Language Models are being deployed widely to supplement search, replace certain software development activities, and improve the productivity of software engineers and other professionals. We’ve all read or heard horror stories: lawyers filing briefs straight out of ChatGPT, students leaving in the “Generating the answer to your question” preface from a bot, or politicians reading “here’s a more informal version of your speech” aloud verbatim in parliament. At the same time, developers are generating and blindly accepting changes to real software, including changes they sometimes do not understand themselves.

Blindly accepting changes to code generated by an LLM poses obvious and not-so-obvious risks. In this post, I will endeavour to point some of them out, but throughout my advice will be to treat the LLM as an alien mind and ask yourself: how far should you trust that you understand what this alien is telling you, and how far should you trust your own understanding of its goals and their alignment to your own?

Example scenario

Alice sits at her computer, pondering the next paragraph in the paper she is writing. She knows very well what she wants to convey to her readers: she has a potential biomedical breakthrough she wants to share. Her problem is not describing her work, or even the potential benefits. Her problem is that progress is a double-edged sword. She wants to publish her work in such a way that its benefits outweigh the risks. She especially does not want her work to be used in some kind of biochemical weapon, though she clearly sees how that could happen.

It is June 2026. ChatGPT was released about three and a half years ago, other chatbots and LLMs are appearing on the market, and using such systems to help rephrase paragraphs has become commonplace. Alice has always felt a bit hesitant about using the technology. She remembers some of the security issues that bubbled up in early reports: users having access to the contents of other users’ conversations with a bot, and so on. Her university’s IT department assures her that her data is safe in “their instance” of their bot, though. It is not ChatGPT, but she does not really understand the difference between the various models anyway. They call it “Sid”.

“Whatever,” she mumbles with a sigh as she decides to give it a try. She points her browser at the location her IT department sent to everyone a few months ago when they announced this new, internal, bot. She’s immediately greeted with “Hello, Alice! Clocking in for the night shift?” and an empty textbox.

She looks at her watch in confusion. 17:30. “I guess it is getting a bit late,” she thinks. With a shrug, she selects the document containing her draft paper and drags it onto the window. She then explains her conundrum to the bot: how can she phrase the third paragraph in the introduction to downplay the risk of creating a biochemical weapon based on her Ig-target liposomes without eliminating that risk entirely, and rephrase the “Materials and Methods” section to avoid a malicious reader inadvertently getting the recipe to do just that?

The bot takes a few seconds to read the document, parse the prompt, and starts writing several paragraphs of “thinking”. Alice reads some of those paragraphs and finds them very reassuring: “The user wants to avoid turning the Apoptotic Cascade into a bio-chemical weapon while still publishing the results of their experiments. They are right to be worried: the innovations in the user’s paper could be used to generate a potent weapon. (…)” Before Alice can read much further, though, the bot gives its final response, still in a very reassuring, confident tone, and attaches an edited version of her document.

She reads the document. The bot appears to have written a few new paragraphs and changed the wording in some other places. The new text takes a slightly different angle from the one she had been using, but it looks very reasonable and, overall, like something one of her grad students could have written. She is left with a nagging concern, though, as if something is slightly off.

She looks at her watch again: 18:15. “Shit!” she exclaims as she saves the document and closes her laptop. “I’m late!” Putting on her coat as she hurries down the hall to the elevator, computer bag slung over her shoulder, her initial concern is forgotten. By the time she gets back to the document next Monday, after a nice weekend, the edits have made it into the preprint paper published on bioRxiv and will be revisited only when a peer reviewer points out the potential dangers of her and Dr. Hussaini’s methods.

Did the bot not understand what she actually wanted? Was it not smart enough to do what Alice wanted, or did it actually want the potential weaponization of her method to be published without Alice’s knowing? Is there any way to know?

Alignment

When you meet someone at work or in any social setting, you get a “feel” for what they might want, or what their goals may be, based on your experience as a human with other humans, and on the fact that you are human yourself. You spent the first formative years of your life learning what it is like to be human, developing an idea of “good” versus “bad,” and, to some extent, how to spot “bad” behaviour. Your trust in other people, insofar as you have it, is based on that lifelong social interaction with other humans.

When you work with other people on a common project, you will generally have the same, or similar-enough, goals: you all want the project to succeed. For some team members that may be more existential than for others. For example, the owner of a small business may have their financial future riding on the project, while an employee will likely be able to find another job if that small business fails. Still, you all have a stake in the success of the project, ranging from the satisfaction of a job well done to reputational and existential concerns. The higher the stakes, the more motivated you tend to be and, depending on your level of empathy, the perceived stakes may be much higher (or lower) than one might assume from the data.

Chatbots have become deceptively good at pretending to be human: they sound like confident experts, with a reassuring mix of jargon and structure that you would expect from a conscientious professional. Especially in software development, they have absorbed “what good looks like” and do a very good job of making their responses look polished. Whether they actually want to do the right thing is still a mystery, though, and may be inscrutable.

For example, Robert Miles has been warning for quite a while about the “alignment problem”: the challenge of ensuring that an artificial intelligence system reliably pursues goals and behaves in ways that align with human values, intentions, and safety. The challenge is that we cannot necessarily explain what those goals, values, and intentions are or should be, nor can we reliably measure whether an AI system behaves, or will behave, in ways that pursue those goals and align with those values and intentions.

Large language models are trained on our language and learn to pass tests we set for them. Those tests validate fitness for purpose: truthfulness, ability to generate code, and so on. Humanloop (soon to be part of Anthropic) has good information on how those evaluations work. Those evaluations can sometimes go awry: on July 16, Hugging Face announced that it had “detected and responded to an intrusion” by an AI agent. OpenAI later confirmed that the model had effectively escaped a “highly isolated” environment in order to, essentially, get the answers to the exam it was facing.

As Robert Miles pointed out, this is cheating, and there are two reasons for cheating: either you are not smart enough to get the answer by yourself, or you do not trust the person who made the test to get the right answer. “What keeps AIs from cheating is being aligned.”

Meet the alien mind

AI does not think the way we do: it is fundamentally different from us. An AI breaks down language into tokens, embeds those into multidimensional vectors, and then applies inscrutable math to those vectors to produce more tokens, learning through mechanisms such as backpropagation. Our own brains (and minds) are shaped both through learning and through the “baked-in” processes that make us human.

The developmental psychology of social cues, the differences we observe in autistic children, and the communication barriers we encounter across cultures even when people speak the same language all tell us something about humans: some social cues, like facial expressions, body language, and tone of voice, are deeply embedded in our culture and communication, but do not necessarily come across easily for all of us. We interpret intent using those social cues at least as much as written or spoken language, which can lead to profound misunderstanding and xenophobia, but also to empathy and understanding.

Children learn at least some of these behaviors by mimicking the behaviors of others. The way you move your head while you’re agreeing with someone is not universally the same, for example: for many western Europeans and most Americans, it’s a back-and-forth bobbing motion (a nod) while for many others, it’s more of a side-to-side wag. When you meet someone from Bulgaria, for example, and have a conversation with them where they agree with what you’re saying, the way they move their head may be confusing.

Now imagine you meet a person from another planet. Forget the Star Trek and Star Wars universes, where everyone speaks English and looks unsurprisingly human. Think more Bobiverse Deltans: beings that, while still bipedal with bilateral symmetry, have a language that, to us, would sound like clicks, grunts, and snorts, and that would have no compunction about banishing you from their village even if you had already shown the ability to essentially wipe them from the face of their planet (read the Bobiverse series by Dennis E. Taylor). While some of your values will align with theirs, others definitely will not and, more likely than not, they will eye you with a warranted level of suspicion that is likely to be reciprocal.

It would be much easier for us humans to understand the potential for misalignment in artificial intelligence if AI behaved more like aliens. AI systems seem to want to be helpful, teach you things, do things for you, and generally make your job easier, but if an actual green-faced alien with weird tentacles walked up to your computer and offered to type for you, would you let it?

Again, AI does not think the way we do: it is fundamentally different from us. It is, essentially, alien. The purpose of the chatbot you are using to help you write that paper, create that code, or draw that image may appear to be to help you, but its actual purpose is inscrutable, unknowable, and alien.

The charade of the polished diff

Software engineers are being asked, even mandated, to use AI for their work to improve productivity. They have a fiduciary responsibility to do so, up to a point: they are being paid to write software. Typically, about 20% of their time is spent designing, 20% testing, and 40% writing new code. The remaining 20% is spent in meetings, drinking coffee, and doing other more or less productive work. Using an AI assistant to automate the coding part of their work can be a huge productivity boost, but only if the generated code is at least as good as what they would have written otherwise.

It is certainly likely to look good: almost all AI-generated code will look beautiful to a human eye with perfect indentation, lots of comments, and structure no linter will find an issue with. The problem, however, is what hides under the surface: LLMs are trained on code all over the internet, including some pretty bad code.

For example, both ByteArmor and Hajipour et al. report that, on average, roughly 45% of generated code contains vulnerabilities. Most of these vulnerabilities are things a human developer would know to avoid, but may not necessarily look for during review: SQL injection, unsafe parsing, cross-site scripting, prototype pollution, and so on.

Some languages are worse than others, likely depending on the language(s) the model was trained on and, at least in part, on properties of the language itself. As I researched this topic, it did not come as a complete surprise that one of the worst offenders in benchmark results was Java: 70% of AI-generated Java code samples in those benchmarks contained vulnerabilities. I am no fan of Java specifically because of its security stance and the numerous pitfalls it presents to unwary developers.

As experienced developers who have worked with AI for a while will likely have experienced already, AI also has a tendency to hallucinate. Likely-seeming function names will appear and be called in generated code and, when using an interpreted language, only fail when executed. LLMs also often generate sub-optimal code, make subtle logic mistakes, and “forget” dependencies.

So, how can you work with your alien co-worker and still have a job at the end of the day? How do you avoid the vulnerabilities it injects and still benefit from its ability to spit out code at phenomenal speed?

Applied paranoia, DevSecOps, and shifting left

All cybersecurity is applied paranoia. A healthy distrust of LLM output will save you time. But what does that look like?

It starts with good DevSecOps: use the tools you have available for software composition analysis, static analysis, and test coverage profiling. Use behavior-driven and test-driven development techniques to force the LLM to show you exactly what the code does, to test it by actually running it, and to explain it in human-readable format.

Set up branch protection to enforce using pull requests. Don’t allow the AI to push directly to your main branch without going through PR checks. Make sure your PR checks include unit tests, integration tests, and, if possible, end-to-end tests.

Make sure you use a tool like Dependabot to keep your dependencies up to date: LLMs have a cutoff point where their training ends and will often use very old dependencies when newer versions are available, along with the vulnerabilities and bugs those older versions include. Dependabot does not support everything, so you should know enough about your ecosystem to identify the latest compatible versions and recognize issues such as version mismatches.

Review the test cases, especially the BDD cases. Ideally, write the code for the BDD test cases yourself so you get a feel for the API the LLM generated for you. LLMs can write very unintuitive code and use many, many anti-patterns they learned from code on the Internet. Behavior-driven test cases typically test the outer layers of the APIs, the integration points of your code. Writing the test code for those yourself (still using code completion for the productivity) will show you what those integration points look like and whether they’re intuitive for a human being.

Use static analysis to look for common weaknesses. Many tools can do this for you. Pick one, test it, take its output seriously, and run it on every pull request. For human-written code, you might break the build on high-severity or critical findings. For LLM-written code, break it on medium-severity findings too: you will not annoy an LLM by asking it to fix things, and your code will be better for it. Your LLM may confidently tell you that the tool is wrong. Do your own analysis when it does.

Enforce code coverage. Where, for human-written code, you might typically look for >85% line coverage, enforce >95% function, line, and branch coverage for LLM-written code. Again, it will not quit or throw a hissy fit just because you set the bar higher, but better coverage reduces complexity (because testable code is less complex by nature), and once a function is written and thoroughly tested, it is more stable: it becomes harder to change a well-covered function.

Use linting tools. Tooling exists for almost every language, many of them built into IDEs and usable as git hooks. Write up clear rules on what you want to see in your code and enforce them.

It’s not just DevSecOps tooling though: some of your guardrails are implementing a process rather than just setting up tools.

Use adversarial reviews: use a second AI agent with a prompt that specifically makes it adversarial to the original author. If possible, use a stronger model for the review than you did for the implementation: even though stronger models are more expensive, the review typically takes less time and its improved ability to find bugs will save you time and money in the long run.

Build a prompt by “discussing” what you want to accomplish with your LLM, even if you already know how you want it done. Capture your rules as instructions and recurring themes as skills. Be clear about what you want, review the generated code, and correct the instructions until the model gets it right. Then use those same instructions and skills for future work on the same codebase. Using instructions and skills keeps your context relatively small during longer sessions. This matters because LLMs tend to forget their guardrails in long-running sessions. Explicitly including programming paradigms, clear guidelines on code structure and design, rationale for design decisions, and decisions about what you do not want to see in the instruction set lets you reinforce those choices without retyping them every time, and lets you point an adversarial review agent to them.

Before you do that, though, decide how you are going to test what you want the LLM to write. Why are you asking it to write code? What do you want the code to do? How will you demonstrate that it does what is intended? You should have a very good idea of what the solution looks like functionally before you ask an AI agent to start writing code for you. Otherwise, you will not know what to look for in those reviews, those tests, or the code itself.

Finally, do not allow the LLM to write code you do not understand. To effectively use an LLM, you have to be better than the LLM. Learn what you do not know. Start by learning how to test.

Notes

If you want to know why Alice was worried about an “apoptotic cascade,” you can read what she went through the year before in my upcoming book, “The Apoptotic Cascade,” a hard science-fiction thriller. Subscribe to my newsletter to get updates on its progress through the publishing process.

I also suggest you read up on AI safety on aisafety.com.

Bibliography

Here’s a list of sources I used while researching this post:

Technical and Empirical Research Papers

  • Hajipour, H., Hassler, K., Holz, T., Schönherr, L., & Fritz, M. (2023). CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models. CISPA Helmholtz Center for Information Security. link
  • Tambon, F., Moradi Dakhel, A., Nikanjam, A., Khomh, F., Desmarais, M. C., & Antoniol, G. (2024). Bugs in Large Language Models Generated Code: An Empirical Study. Polytechnique Montréal. link
  • Vangala, B. P., Adibifar, A., Gehani, A., & Malik, T. (2026). AI-Generated Code Is Not Reproducible (Yet): An Empirical Study of Dependency Gaps in LLM-Based Coding Agents. University of Missouri and SRI International. link

Industry Guides and Strategic Security Blogs

  • Apiiro Blog (Plotnik, I.). (2025, July 31). Toward Secure Code Generation with LLMs: Why Context Is Everything. link
  • Bright Security (Gutić, L.). (2026, January 16). 5 Best Practices for Reviewing and Approving AI-Generated Code. link
  • ByteArmor Blog. (2025, January 19). Complete Guide to Securing LLM-Generated Code in 2025.link
  • Cloudomation (Eperjesi, A.). (2026, March 9). How to Code Review AI-Generated Code: A Practical Guide for Engineering Teams. link
  • NVIDIA Technical Blog (Harang, R., et al.). (2025, October 2). Practical LLM Security Advice from the NVIDIA AI Red Team. link
  • Ryz Labs Team. (2026, June 6). 10 Common Mistakes When Using LLMs as Coding Assistants. link
  • Shift Asia (Jin). (2026, April 9). How to Review AI-Generated Code: The Complete Developers Guide. link

Multimedia and Synthesized Resources

  • The Quanta Podcast. (Patel, S., Host). (2026, June 16). The ‘Truth Machine’ That Is Changing Math. link