Your Documentation Might be Answering the Wrong Question. Here’s “Why.”

This weekend started with a small, well-scoped question: what’s the migration path from TrueNAS Core to SCALE that doesn’t cost me any data? I wasn’t looking to do the migration, just figure it out for planning purposes.

I thought it was a good Saturday morning question, something I can do quickly before everyone else wakes up. After all, I’ve run this box for years. I know what’s on it.

Except I went to confirm a couple of things before committing to a path, and it turned out several assumptions I was confident about — assumptions I’d been operating on for a long time — were just not true. And in the process of finding that out I turned up a few bugs I’d been living alongside without knowing it, quietly, for who knows how long.

Now, most normal, sane people would have just patched the assumptions and continued with the project. Except I’m neither normal nor sane. I am an Atlassian Admin, and this wake-up call made me realize viscerally that my documents were not up to snuff.

So, no. My action is to stop and ask what else I believe about this system that isn’t true. And then, since we’re already here: why stop at this one system? I have several after all. And there was the network. Also, I have a lot of unused disks, but are any of them worth repurposing? So many questions, so little time.

So the weekend stopped being a migration plan exercise and became an audit. Which is fine. Nothing was documented, but nothing was unknowable, either. The information was all there to ask. It was sitting right there in the systems, accurate and current, available to anyone who ran the right command. What was at risk was my model of why it was set up that way. The system could tell me its state. It could not tell me which of my beliefs about it had quietly expired. I’m great about asking for the why. Not so great about writing it down for next time. And because of that, I almost walked into a migration carrying assumptions I had no basis for.

I don’t think that’s a personal failing. I think it’s the default failure mode of technical documentation, and it shows up in Atlassian instances constantly. We are all answering the wrong question. Jira already answers what, better than any page you’ll write. The job of documentation was always to answer why.

Jira Documents the What. It Will Never Document the Why.

Here’s the thing that makes Atlassian documentation different from documenting almost anything else: your instance is already self-describing.

Here’s what I mean. You can go from the front page of Jira, and in three clicks, find the complete list of fields for your instance. It’s always current, and describes exactly the state of every field right now. That takes zero effort to document, it’s already there. Same for workflows, schemes, permissions, project types, and every app you’ve installed. The admin screens are a live, accurate, zero-maintenance inventory of your configuration. Any document you write that duplicates them starts rotting the moment you publish it.

However, I have been just as guilty of the same tendency. I’ll start every Atlassian documentation effort by doing exactly that, recording details on what the current state is. I’ll open a space, start transcribing the admin screen into Confluence. And it maybe takes one week (at best), and it’s already inaccurate. Six months later, and it doesn’t even resemble reality. Nobody trusts it, and it’s left to rot, slowly drifting more and more from anything anyone would recognize. Then it gets restarted eighteen months after that by someone new who is certain they’ll do it properly this time.

All the while, what I think we (and I emphasize again, I’m including “me” in that) miss is the most critical question. For this current engagement, I took a different approach. I started asking “why” more. I know, I know, seems late, but yeah. But time and time again, people cannot answer me because they never wrote it down. I can usually find it in some ticket, and Rovo helps surface those, but it’s still tedious work that should have just been written down.

But that’s the point. The information Jira cannot give you is intent. It will tell you that a field called “Business Justification” exists on four screens in two projects. If you’re lucky, that field will have a description that gives one sentence about how it will be used. If. It will not tell you that Legal asked for it in 2022 for an audit that concluded in 2023, that the person who requested it has since left, and that nobody has needed it since.

I did an environment assessment recently that turned up just over a thousand automation rules across two cloud sites. A thousand. Somebody had a reason for every single one of those on the day they built it. Not one of those reasons was written down anywhere. That is not a case of people being lazy. Those rules got built under deadline by admins solving real problems, and every one of them made sense at the time. The reasons just walked out the door with the people who held them.

And once intent is gone, you cannot get it back by looking harder at the config. You end up with an instance full of things nobody dares to delete, which is how a thousand rules becomes fifteen hundred.

The Next-Guy Test

One of the best pieces of advice I got during my time at university didn’t come from a professor or an instructor. It came from a pastor at the student ministry. But he told me to always work like someone else will be doing the job tomorrow. He had a few meanings behind that, obviously, but as someone aspiring to work in I.T., it seemed especially prudent advice.

So let’s review that advice against the current situation. If someone else were to take over my Jira instance tomorrow, they can tell me what’s on Jira. But if the admin screens cover the what, the only defensible reason for a page to exist in your documentation space is that it answers a question the instance itself can’t.

I’ve started running everything through what I think of as the next-guy test. Not “is this written down,” which is the question everyone asks and which produces the transcription problem above. The question is: could somebody who has never met me take over tomorrow?

That reframing does a lot of work. It kills the pages that duplicate the config, because the next guy can read the config. It kills the four-thousand-word onboarding doc nobody finishes, because the next guy is reading it at 2 a.m. during an incident and won’t get past the second heading. And it creates a small number of pages that turn out to matter enormously.

The next guy is usually you. Not you today, but you in eighteen months, with none of the context you’re currently holding and a great deal of misplaced confidence that you remember how this works. Programmers have a name for this. Eagleson’s Law: any code you haven’t looked at in six months might as well have been written by a stranger. It holds for configuration too, and I would argue it holds harder. Code you at least wrote in one sitting. A workflow you built across eleven separate changes over two years was never in your head all at once to begin with. That’s the version of yourself the documentation is actually for, and he is a considerably less reliable colleague than you’d like to think.

If it helps to have stakes attached, run it as the lottery test instead: you win Powerball tonight and never log in again. Not a bus, nothing grim. You just stop. What breaks, and how long does it take somebody to find out?

It also sets the standard for how much you write. The test isn’t whether the page is complete. It’s whether it’s sufficient. Three sentences that let somebody make a decision beat three pages that let them understand your reasoning in full.

And when I actually apply that test, almost everything I care about collapses into two documents.

Two Documents, Two Different Failures

This is the thing I find myself saying to junior admins more than anything else, usually while looking at something they’ve just built that works perfectly and that nobody will be able to touch in three years.

Every configuration of consequence needs a requirements document and a design document. Not one combined thing. Two, because they answer different questions and losing each one breaks something different.

The requirements document says why the thing needs to exist. What problem, whose problem, what happens if we don’t solve it, how we’ll know it’s solved. It’s written before the build and it’s mostly not about Jira at all.

The design document says why it was built specifically this way. Which approach we chose, what we rejected, what constrained us, what we knowingly traded away. It’s written during the build, while the constraints are still in front of you and still annoying you.

The split matters for a reason that took me years to say cleanly:

Without the requirements document, you can never delete anything. You don’t know what problem it solved, so you can’t determine whether that problem still exists. Everything becomes load-bearing by default. That’s the thousand-rules instance, exactly: not a hoarding problem, an epistemics problem. Nobody is keeping those rules because they’re attached to them. They’re keeping them because deletion requires knowledge nobody has.

Without the design document, you can never safely modify anything. You know why it exists but not why it’s shaped like that, so every change is a coin flip on whether you’re about to step on a constraint somebody hit in 2022 and solved deliberately. This is where you get the admin who won’t touch the workflow, and is right not to. Every one of those untouchable things in your instance is a design document that never got written. The constraint was real, somebody solved for it correctly, and the reasoning stayed in their head until it became folklore.

And you will answer for it forever. The next guy doesn’t just arrive unable to touch the thing. He arrives with a better idea. Why not just use a component for that? Without the design document you cannot answer him, only assert, so you relitigate a decision you already made correctly, from memory, against somebody with a fresh argument. Written down once, that conversation takes ninety seconds: we tried that, here’s what broke, here’s why we chose this.

Map those back to the next-guy test and it’s tidy. The requirements document is what lets the next guy decide whether to keep it. The design document is what lets him decide how to touch it. You need both, and most instances have neither.

And a decision log for everything after

Req and design cover the build. They go stale the first time somebody changes the thing, which will be about six weeks later.

So the third artifact is a running decision log: lightweight architecture decision records, appended every time a meaningful change lands. Not the formal RFC-with-a-template kind. That discipline dies in about three weeks. Five fields: what we decided, when, who signed off, what problem it solved, what we considered and rejected. Four minutes to write, and written the day the decision is made, while you still remember the alternative you dismissed and why.

The rejected options are the part people skip and the part that pays. When somebody proposes the same thing again in two years (and they will, confidently, as a fresh idea), the log is what turns a three-week rediscovery into a five-minute conversation.

Think of the decision log as the design document continuing to be written after the project closes. Because that’s what it is.

Every Audit Is a Search for Why

I get paid to walk into instances where nobody wrote any of this down.

An audit is not an inventory. If all I needed was a list of what exists, I’d export it and be done before lunch. What I’m actually doing in an assessment is trying to determine why: why this works, or more often why it doesn’t, and whether the thing in front of me is a deliberate choice I don’t understand yet or a mistake that has simply survived long enough to look intentional.

Those two look identical in the config. Completely identical. The only thing that distinguishes them is a document somebody wrote at the time.

When the req and design documents exist, an audit is reading comprehension. When they don’t, it’s archaeology, and archaeology produces a plausible story, not a true one. You reconstruct intent from artifacts, you’re usually close, and “usually close” is how you confidently recommend deleting the one thing that turns out to be load-bearing.

Which is the part of my weekend I’d rather not have relearned. I had the state of the system. I inferred the why. The inference was wrong, and it had been wrong for a long time without ever once announcing itself, because nothing in the system’s current state contradicts a wrong story about how it got there.

Where This Falls Apart

Here’s the thing – and it’s something my ADHD struggles with to. This is not a project you complete once and celebrate. This needs to be a continuing practice.

Documentation rots. That is not a character flaw in the people who wrote it, it’s just entropy. And it rots faster in Atlassian than in most places right now, because the pace of change alone will invalidate parts of your documentation before you’ve finished writing them. Every page you create is a maintenance liability you’re taking on. A wrong page is worse than a missing one, because a missing page makes people go ask somebody and a wrong page makes them confident.

The honest mitigation isn’t discipline, because discipline is exactly what runs out. It’s tying the writing to the moment. Decisions get logged when the decision is made, not in a quarterly documentation sprint that will not happen. If a page only stays current because somebody remembers to revisit it, that page is already stale and you just haven’t checked yet.

The Part That’s New

Everything above would have been true in 2016. It’s mostly good documentation practice with the Atlassian specifics filled in. But there’s a reason I think this got more urgent in the last eighteen months.

Rovo reads your Confluence.

That’s the entire pitch of the Teamwork Graph, and it’s a good pitch. Your organizational knowledge stops being a wiki people don’t visit and becomes context an agent can actually use. I’m not being cynical about it. This is the most interesting thing Atlassian is building right now.

But think about what it means if your documentation space is nothing but a transcription of the admin screens.

Somebody asks Rovo why a field exists. Rovo has read every page you have, and every page you have describes what. Here’s the thing worth remembering about how these models work: it produces the most probable continuation given what it has been fed. It is not looking something up, it is completing a pattern. The most probable continuation of a why-question is a reason, so you will get one. Specific, plausible, naming a business need you recognize, in complete sentences, with the confidence of something that has genuinely read all your documentation. And nothing about the way it arrives distinguishes it from a reason somebody actually had.

That isn’t the model malfunctioning. That’s it working exactly as designed on an input that never contained the answer. Feed it what, ask it why, and hallucination is the only move left. The old failure was a human saying “I don’t know, ask Rodney.” This one doesn’t say that.

Which is the archaeology problem again, except now it’s automated and it doesn’t hedge. When I reconstruct intent from artifacts I at least know I’m guessing, and I’ll tell you so in the findings. An agent reconstructing intent from artifacts produces the same guess in the register of an answer. It has no way to distinguish “the documentation says why” from “the documentation says what and I bridged the gap,” because from the inside those feel identical.

I made more or less this argument about certifications a few weeks back: every agent that touches your configuration is borrowing judgment from whoever set that configuration up. Documentation is the same borrowing, one layer out. An agent reasoning over your instance is reasoning over what you wrote down. Undocumented intent used to be a bus-factor problem, and bus-factor problems are slow. They surface when somebody leaves. This one surfaces every time anybody asks a question, and it surfaces as an answer rather than as a gap.

I don’t think most organizations have connected these yet. They’re evaluating Rovo as a capability question (can it do the thing) when a lot of the answer is going to come down to what’s sitting in Confluence when they turn it on.

Wrapping Up

Your instance already tells anyone who looks exactly what it does. It will never tell them why, and no amount of transcribing the admin screens into Confluence will change that. The why has to be written by the person who had it, on the day they had it, or it does not survive.

So here’s the thing to actually go do, and it takes about twenty minutes.

Open your automation rules. Sort by last modified, oldest first. Take the five at the top and, for each one, try to write a single sentence explaining why it exists and what would break if you turned it off. Not what it does. You can read what it does. Why it’s there.

However many of the five you can’t answer is your real documentation debt, expressed as a fraction. Mine was worse than I’d have guessed, and I’m the guy who writes about governance for a living.

Then, going forward, the discipline is smaller than it sounds. Nothing of consequence gets built without a requirements document saying why it needs to exist and a design document saying why it’s shaped that way, and every meaningful change after that gets a four-minute entry in a decision log. That’s it. That’s the whole program, and it’s the thing I harp on to every junior admin who’ll sit still long enough, because I have never once regretted having those documents and I have lost entire weeks to not having them.

The admin worth keeping isn’t the one whose instance is documented. It’s the one whose instance can be handed to somebody else — and increasingly, to something else — without the reasoning walking out the door alongside them. That used to be a courtesy you extended to your successor. It’s turning into a correctness requirement for every agent you point at your data.

I’d like to know what your five-out-of-five looks like, because I suspect the number varies wildly by how long the instance has been running and how many admins it’s outlived. And if there’s a page in your space that has genuinely earned its place by answering a why, I want to hear about it.

Until then, this is Rodney, asking: have you updated your Jira issues work items today?

---

Enjoyed this one? These posts are free and always will be — but if this saved you a headache or taught you something worth keeping, you can drop a tip in the Ko-fi jar. No paywall, no subscription, no catch. Just a thanks if the writing earned it.

→ Support The Jira Guy on Ko-fi