I killed the worker, watched it die, and moved on. Twenty-five minutes later a batch I thought I had stopped had quietly finished on its own.
An orphaned child process is a worker whose parent keeps running after the worker is killed, so the parent’s remaining work proceeds as if nothing was cancelled.
Table of Contents
The problem, in numbers
The setup was ordinary: a small shell script that ran two jobs in sequence — first one batch, then another. Something looked wrong partway through the first batch, so I stopped it the way reflex says to: find the worker, kill it, done.
The worker died. I confirmed it was gone. I turned to something else.
What I had not accounted for is that the worker was a child of the shell script, and the shell script was still very much alive. From its point of view, the child had finished — that is all a killed subprocess looks like to its parent. So the script did what it was written to do: it moved to its next line and launched the second batch.
That batch ran for 25 minutes and completed about two dozen units of work before it reached its own natural end. I found out only when I went looking for the half-finished job and found it done.
A killed subprocess and a finished subprocess look identical to the parent that launched it.
Why it stayed invisible
Three things conspired to hide it, and each is worth naming because they recur.
First, the kill worked. There was no error, no zombie, no warning — the thing I aimed at died exactly as asked. Success on the wrong target feels the same as success.
Second, the parent was silent. A wrapper script that just calls two commands prints nothing of its own between them. There was no “starting batch two” line to make me realize the sequence was still marching forward.
Third, I killed by name, not by lineage. Reaching for the worker by what it was — its program name — rather than by where it sat in the process tree meant I never considered that something had launched it and was waiting on it.
The uncomfortable part is that none of this is exotic. It is the default behavior of every shell that runs one command after another.
Success on the wrong target feels exactly like success on the right one.
Proving it was the wrapper, not a restart
The first explanation that came to mind was the wrong one: that something had restarted the job — a scheduler, a retry, a watcher. That would have been a different and worse bug.
The logs ruled it out cleanly. The wrapper’s own output showed the two batches as one continuous run: a header for the first job, then — with no gap, no second invocation, no restart banner — a header for the second, and a completion line 25 minutes later. One process, two phases, exactly as written. The second batch had not been relaunched; it had never been prevented from starting.
That distinction matters for the fix. A restart bug is fixed by finding what restarts. A run-on bug is fixed by killing the right thing in the first place.
The fix, cheapest first
Kill the job, not the leaf. The moment you have a wrapper, “stop this” means stop the wrapper — or the whole process group — not the one worker you can see. On a POSIX shell, launching the job in its own process group and signalling that group takes the parent and every child with it. The worker-by-name habit is the trap.
Make the wrapper announce its PID. The cheapest durable defense costs one line at the top of the script: write its own process id to a known file. Now “stop the job” has a single unambiguous target, and stopping it means killing the thing that starts the next phase, not the current one.
#!/usr/bin/env bash
echo $$ > run.pid # the job identity, not the worker
run_first_batch
run_second_batch
rm -f run.pidCode language: PHP (php)Give the parent a voice. A one-line “starting phase two” between jobs turns an invisible run-on into something you would have caught in the first ten seconds. Silence is what let 25 minutes pass.
When in doubt, check the tree, not the list. Before concluding a job is stopped, look at what is still running and what launched it — a live parent with a dead child is the signature of exactly this mistake.
Stop the thing that starts the next job, or the next job starts without you.
What the fix cannot do
A PID file and a process group make the job killable; they do not make it safe to kill at any moment. If a worker is halfway through something that must complete as a unit, killing its group mid-step can leave that unit half-done — which is a separate discipline: make each unit of work resumable — so an interrupted job costs one item, not a corrupt state, the same discipline that lets you reconcile before you recreate on the retry. The process-group fix ensures you can stop the whole job; idempotent, resumable work is what makes stopping it cheap. You want both, and they solve different halves of the problem — the same process-hygiene instinct behind giving each parallel agent its own git worktree so concurrent jobs never collide.
The short version
A shell wrapper ran two batches in sequence. Killing the first batch’s worker by process name stopped that worker but not the wrapper, which returned from the call, advanced to its next line, and ran the second batch unattended for 25 minutes. The logs proved it was a single continuous run, not a restart. The lesson is to operate on the job rather than the process: launch it in its own process group and signal the group, write the wrapper’s PID to a file so “stop” has one target, and give the parent a line of output between phases so a run-on is visible. A PID file makes the job killable; resumable work is what makes killing it cheap.
Frequently Asked Questions
Why does killing a child process not stop the parent script?
Because to the parent, a killed child is indistinguishable from a child that finished normally — both just mean “the subprocess I was waiting on has ended.” The shell then continues to the next command. Nothing about the kill propagates upward; signals travel to the process you target, not to whatever launched it.
How do I kill an entire job and all its subprocesses at once?
Launch the job in its own process group and send the signal to the group rather than to a single PID. On a POSIX shell that means signalling the negative of the group id, which reaches the leader and every descendant. Killing by program name reaches only the processes that happen to share that name, which is rarely the whole job.
What is a PID file and why write one?
A PID file is a small file into which a script writes its own process id when it starts. It gives operators and other scripts a single, unambiguous answer to “which process is this job?” — so stopping the job means acting on the identity in that file, not guessing from a process list where a wrapper and its worker look like unrelated entries.
How can I tell an orphaned run-on from a restart?
Read the launcher’s own output. A run-on appears as one continuous log — one start banner, phases following each other with no re-invocation. A restart shows a second start banner, often from a scheduler or watcher. The two have opposite fixes, so distinguishing them before you act saves you from hunting a restarter that does not exist.
Is this only a shell scripting problem?
No. Any parent-child arrangement has it: a process manager and its workers, a test runner spawning subprocesses, a CI step that backgrounds a task. Whenever one thing launches another and then continues, killing the launched thing leaves the launcher free to proceed. The shell wrapper is just the most common place to meet it.
Should the parent process trap signals to clean up?
It helps. A parent that traps termination signals can remove its PID file, stop in-flight children, and refuse to start the next phase — turning an abrupt kill into an orderly stop. It is not a substitute for targeting the right process, but it makes the right target behave well when you hit it.
