What should it say when it shouldn’t answer?

An enterprise learning platform inside a large organisation — people in different roles, with different skills, and a standard the company has to hold everyone to. Courses go out, get completed, get measured. Assignment already runs through the department manager, who approves what each person takes, so the employee is clear on what to follow and what to finish. That leaves the training operations manager doing something narrower: answering the questions employees ask along the way.

Those questions arrive all day and most of them repeat. Why can’t I enrol in this course. Does the certificate I already hold count instead. What happens now the deadline has passed. The answers are already written down in the organisation’s own policy library. They just cost somebody’s time to go and find, every single time.

The assistant was never meant to replace that person. It was meant to stop them answering the same question for the fortieth time — so the time goes to the questions the policy library cannot answer.

The screens below are not images. Every one is live HTML I wrote, rebuilt from scratch with invented policy documents. The working prototype answers the same questions from the same invented set — nothing here is a screenshot of it, and no real organisation’s policy library appears anywhere on this page.

An assistant over a policy library is easy to make helpful and easy to make dangerous.

Two things make it hard. The same question has different correct answers depending on who asks — a junior’s mandatory training is not a senior’s, and a deadline set for an in-house department is not the deadline set for an overseas one. Same question, three right answers, and only the asker’s role tells them apart.

The second is worse, and it decides the whole design. A wrong answer is bad. An invented answer is worse. If the assistant states a rule that sounds official and no such rule exists, nobody goes back to check it — they just follow it.

The failure that costs you is not a wrong answer. It is an invented one.

The person doing this job today has the same failure modes. Someone answering the same question for the fortieth time, while three other things need attention, gets it wrong sometimes too. So the bar is not speed. The assistant has to be more consistent than the person it stands in for, and it can only manage that by refusing to say anything that isn’t written down.

I chose learning management because I had built three of them, and no two were alike. One was a cybersecurity training platform, where a section stayed locked until the one before it had been completed and verified. One served a chain of schools and training institutes, with far more roles in it than the first. One was built around governance policy, where the rules mattered more than the catalogue.

Different clients, different scale, different problems. The same shape underneath: the administrator’s day is repeated questions, and the answers are already written down somewhere.

Then the design started to fail, and it was my own doing. I built for five roles at once — training operations manager, LMS admin, manager, employee, compliance owner. Each role multiplied the conditions attached to every answer, and the product grew past its own centre. I could feel it stop being about anything.

I threw it out and started again with one primary user: the training operations manager, because they are the one absorbing the repeated questions. The other four are phases, not missing features. Each is the same assistant pointed at a different set of eligible documents — an employee sees their own policies, a compliance owner sees the audit set — and the permission decision in the next block is what makes that true. Adding a role is a mapping, not a rebuild.

The obvious way to handle permissions is to let the system read everything and strip out what the asker shouldn’t see from the finished answer. I rejected it, and the reason is not security.

Permission here is not secrecy. It is correctness. A junior and a senior asking what training is mandatory for me have two different right answers, and their role is what tells those answers apart. Once permission decides which policy applies, redaction after the fact stops making sense — you would assemble an answer out of senior policy and then cut holes in it, and what is left is not the junior’s answer. It is a damaged copy of someone else’s.

You cannot subtract your way to a different correct answer.

So the filter runs before retrieval. The role is resolved first, the eligible documents are selected, and the model never sees the rest.

That left one line to draw. Role does not govern what you may read about. It governs what you can be assigned under. A junior can be told what senior certification requires — that is career planning, and it is what the platform exists for. A junior cannot be treated as senior when the assistant works out which rule applies to them. Progression itself was never the assistant’s job; the portal already shows it as an unlock path.

The first release answers questions and does nothing else. It cannot assign a course or add a user — which, in all three platforms I had built, is most of an administrator’s day.

Assignment already belongs to the department manager. That approval step exists so somebody is accountable for what an employee is told to learn, and an assistant that assigns quietly removes the accountability without removing the responsibility.

The second reason matters more. An assistant that can act fixes things silently. Somebody asks about an overseas team that missed a deadline, it assigns the make-up course, done — and nobody ever finds out that no deadline rule was ever written for that department. Because it can only answer, it has to raise a ticket instead. Every ticket is a question the policy library could not answer. After a year, that list is the most useful thing the product has produced.

And the loop closes without anyone maintaining it. The policy owner drops the new or corrected document into the approved folder; the assistant re-indexes on a schedule and answers from that version from then on. No retraining, no manual upload step, nobody having to remember.

Submitted on time in March 2026, as the third assignment of a product management cohort. It was scored twice: 90 for covering the rubric, then 59 on a second pass that measured original thinking rather than completeness. The largest drop in the cohort, and the lowest revised score in it.

Both numbers were right. Running out of time, I had written by giving an AI pointers, letting it expand them, then trimming what came back. After enough rounds of that, the expansion was setting the reasoning instead of the other way round.

The brief asked for a document. It never mentioned a prototype. I built and shipped a working one anyway, a month later.

Write it in my own words and submit something shorter.

The scope problem and the writing problem were the same problem. Both times I added instead of deciding — five roles because I hadn’t chosen one, more prose because I hadn’t finished thinking. The document that scored 90 was not better than what I could have written myself. It was longer, and it covered every box.

There are still gaps in it I have not closed.