Do You Know That You Don't Know?

A call, a shared screen, a client’s development environment. On it, PHP old enough to vote, still holding up numbers that people downstream trust. The engineer on the other end has been hunting a memory corruption for days. He is not a beginner. He has more years on that codebase than I have, and he has found something.

He shows me a string he pulled off the internet. USE_ZEND_ALLOC. Set it to zero and PHP stops using its own memory manager and falls back on the system’s malloc.

Then he says the sentence this whole article is about.

“On php-cli it works. On php-fpm, seems not. I asked the sysadmins, they put it in the www.conf, and still nothing. So I dunno.”

Hold those last three words. We are coming back for them.

Which one of them was the incompetent one?

Take the story apart and go looking for the fool in it.

The developer? He found the flag. Nobody handed it to him. He read the crash, he searched, he came back with a real key, the exact string that appears in the PHP source. Then he tried it in the place where things get configured, because he is a developer, and configuration lives in the config file.

The sysadmins? They took a request from a developer, opened www.conf, and put the setting where settings go in a pool config. That is their file. That is their mandate. Ask them to know that this particular variable is special, and what are you asking for? A systems administrator who knows the internals of a language runtime. A sysadmin specialised in PHP.

Does that person exist? Has anyone ever met one? A company that runs computers needs sysadmins. A company that writes PHP needs PHP developers. Which of them has the seam in their job description?

Everyone on that call executed their role correctly. The corruption is still there.

So what were the roles measuring?

Wasn’t there a word for this?

There is. Someone noticed the wall, and named it, and the name was supposed to dissolve it.

DevOps. Two role names glued together. Does gluing two words produce a person who knows both jobs?

Watch what companies did when they were handed the word. They hired a DevOps guy. A wall with a job title, and three boundaries where there had been one.

And look at that call again, because the thing the word promises had already happened in it. The sysadmins took a developer’s request and edited an application setting on his behalf, same day, no ticket war, no blame. The wall came down. That is the ceremony, performed correctly, in good faith, by people who were not fighting each other.

The memory corruption did not attend.

Why did it work in one place and not the other?

This was the part that read as mystery. Same flag, same server, same PHP. It works from the command line. It does nothing under php-fpm. Two behaviours, so there must be two rules, so someone must have misconfigured something.

There are not two rules.

In the PHP source, zend_startup() calls start_memory_manager(). That calls a small constructor, and the constructor does this:

tmp = getenv("USE_ZEND_ALLOC");
if (tmp && !ZEND_ATOL(tmp)) {
    ...
    mm_heap->use_custom_heap = ZEND_MM_CUSTOM_HEAP_STD;

getenv. Inside startup. The memory manager reads that variable while the engine is waking up, and builds the heap according to what it finds there. php-fpm applies the env[] lines from www.conf to the pool afterwards, once the engine is already standing. The key arrives at a door that has been opened and closed.

On the command line you type USE_ZEND_ALLOC=0 php script.php, and the variable is in the process environment before the binary exists. Under php-fpm the process is a daemon, and it was born hours earlier, before anyone in this story sat down.

It was never two behaviours. It was one fact, seen from two sides of a wall.

To set it, you have to touch the file that starts the daemon. That file belongs to the sysadmins. That setting means something only to a developer. So where is the boundary between the two roles, exactly? It is not a concept, and it is not on the org chart. It is a file. Whose is it?

So who on that call was the senior one?

Somebody on that call solved it. Not with PHP knowledge.

Whoever solved it did not know what USE_ZEND_ALLOC was either. Had never used it. Had no idea the memory manager read it at startup. Searched for the same string, and the same pages came back.

One fact did all the work. Environment variables do not go in a config file.

That is the entire distance between “so I dunno” and a solved bug. Not a deeper knowledge of PHP. Not more years on that codebase, because the developer had those. One small fact about how a variable reaches a process. Junk knowledge, never used on that project or the one before it, sitting there for years earning nothing, waiting for a Tuesday that might never have come.

He found the key. Somebody else knew it was a key, and that a key wants a door, and that this was not the door.

So how much of what you know is that? Not the skills on your CV. The debris. The stray facts you could not sell to anyone, that you never chose to learn, that came off some other stack in some other year.

Which of those is going to save you next?

What did the word actually promise?

Senior. We say it as though it certifies mastery. Senior PHP developer. The title names a language. It certifies nothing whatsoever about the thing the language runs inside.

So the org chart says the gap belongs to nobody. The CV agrees with the org chart. The two documents are in perfect harmony, and the runtime has read neither.

And the word itself? Senior is Latin. Comparative of senex. It means older. Age. Time elapsed. That is the entire content.

Now put that one useless fact next to it. What had actually accumulated in the person who solved it? Not mastery. Just enough time in the trade to have collected debris. A stray fact from another year, another stack, another problem with nothing to do with this one.

Older. The word had been telling the truth the whole time.

So who added the promise? Which of us reads expert into a word that only ever said has been here a while?

What was the flag going to fix?

Here is the part nobody on that call reached.

USE_ZEND_ALLOC=0 is a debugging instrument. You set it so that a memory debugger can produce a readable trace, so that the corruption becomes visible. It does not repair anything. Bypass the memory manager and the bug simply lands somewhere else, in memory nobody happens to be watching, and the symptom goes quiet.

So suppose it had gone perfectly. Right file, right team, first try. The crash stops. Does the bug?

And from the outside, what would that have looked like? A room, three job titles, several days, and a thermometer being administered as medicine. It would have looked exactly like a fix. It would have been written up as one.

How many of the fixes in your repository are thermometers?

Are those two sentences the same sentence?

Now take back those three words.

“So I dunno.”

There is a more famous version of that admission. I know that I know nothing. It is the most quoted line in the history of asking questions, and it sounds identical. Same content, same humility, same shrug.

One of them is where an inquiry begins. The other is where one stops.

He was not short of knowledge. He said “I don’t know” and put a full stop after it.

Everyone on that call said “I don’t know” about USE_ZEND_ALLOC. The difference was where the sentence pointed. One version can see the shape of the thing it is missing, so it goes looking. The other cannot see that there is a shape, so it sits down.

Nothing in that story turns on what anybody knew. It turns on what each of them knew about the edges of what they didn’t.

So when someone says “I don’t know,” how do you hear which one they mean? Which one did you mean, the last time you said it?

Where is your next bug?

Not in the code you find difficult. You know where that is. You are careful there.

Your next bug is sitting exactly where you don’t know that you don’t know. It has to be. That is what the words mean. You cannot go and check the place, because checking it would require knowing there is a place.

The developer on that call could not have found his own gap. He was not lazy and he was not a fool. He had a map, and the map ended, and there was no mark at the border saying the world continues past here. That is not a defect in him. Every map ends. Yours ends. Every other map on that call ended too, just somewhere else, and it was luck, not seniority, that the edges did not all end in the same place.

So the bug does not live in the gap between the roles. It lives at the ragged edge of whichever map is nearest to it.

And your next promotion will arrive for a fact you picked up years ago and never needed, on a day you cannot predict, for a problem you have not met yet.

They will call it seniority. The word will only be claiming that you were there a while, and the word will be right.

So, once more, in the oldest question there is. Do you know that you don’t know?

And if you do… where?