For a long time I could not reconcile two things. The progress reports were good. My son was not.
I assumed I was reading the reports wrong, or that I was the problem, or that I simply lacked the training to see what the professionals could see. It took me far too long to understand what was actually going on, and when I did, it was not that the reports were missing something.
The reports were measuring the thing that was hurting him, and scoring it as progress.
What the reports actually measured
When I finally read them properly, rather than skimming for the word “progress,” here is what they contained. How compliant he was with their goals. How many stars he was collecting on their charts by doing what he was told.
That was it. That was the content.
And here is what he looked like to me across the same period:
It looked like he was dead inside. Just following directions because he has to.
I want to be careful here, because that is a hard sentence to read and it was a hard sentence to write. But it is what I saw, and softening it would misrepresent what those months were like.
Why no report could have warned me
This is the part that took me the longest to work out, and it is the reason I now read every progress report differently.
The report was not failing to capture his distress. Compliance was the metric and compliance was the damage. They were the same variable. A report showing he was doing well and a report showing what was happening to him could not both exist, because the thing being scored as success was the thing I was frightened by.
There is no version of that document that could have told me. It was working exactly as designed.
It was never a skill
People sometimes describe this as a generalisation problem. The child masters a skill in the clinic and cannot use it at home, and the fix is more practice in more settings.
That was not our experience, and I think the framing is wrong. It was not a skill that failed to transfer. It was compliance. He learned what that particular room expected of him, and that knowledge did not apply anywhere outside it, because there was nothing transferable in it to begin with. He was not learning how to do something. He was learning what they wanted.
Something in the research literature is at least consistent with this. The 2023 BMJ review known as Project AIM pooled 252 studies covering 13,304 children. When the analysts excluded outcomes rated by caregivers and teachers and studies at high risk of detection bias, most of the reported benefits did not survive. Effects that shrink when you change who is doing the rating are effects tied to a particular observer in a particular setting.
I am not claiming that study proves my point about my son. It is one finding, and mine is one family. But it stopped me assuming the measurement was more solid than what I could see with my own eyes.
Why it could never have carried over
I tried running the same kind of charts at home, because that was what was recommended. It did not make sense, and it took me a while to understand why it did not make sense.
You cannot use a chart all day long. He went to the park. He went to school. He had after-school classes. He was at other people’s houses. I could not control a reward chart across all of those places, and nobody else was going to run one for me.
Which means the approach was not failing to generalise through bad luck or insufficient practice. It could not generalise. A token system needs a controlled environment to operate in, and a child’s actual life is not a controlled environment. What was holding the behaviour up was the scaffolding, and the scaffolding only exists where someone is standing there holding it.
That is the same point as the one above, arrived at from the other direction. It was not a skill that stayed behind in the clinic. It was compliance with a system, and the system did not come with us to the park.
When I raised it, the answer was more hours
I did tell them. I said he was distressed, that he had started refusing to get in the car, that he was inventing reasons to go anywhere else.
They said this was normal. Then they asked me to go back to the insurance company and request additional hours, because he needed it.
That is the response that eventually told me everything. The system had one reading of a suffering child, and that reading was insufficient dose. There was no mechanism for the answer to be less, or different, or stop.
It is worth knowing that the evidence does not support that reflex. A 2024 meta-analysis in JAMA Pediatrics, covering 144 studies and 9,038 children, tested whether more intervention produces better outcomes across three separate measures of dosage. None showed a significant positive association. Their conclusion was that the findings “do not support the assertion that intervention effects increase with increasing amounts of intervention.”
Who was actually in the room
There was nobody at the clinic to talk to about any of this. The people delivering the sessions were very young technicians. The clinical director was never there.
I used to think that was a failing of the particular clinic we had chosen. Having since read the credentialing requirements, I think it is closer to the design.
The Registered Behavior Technician handbook, published by the certifying body itself, sets out what an RBT needs: to be at least 18, a high school level education, a 40 hour training, a competency assessment, and a background check. Once certified, they must receive ongoing supervision for “a minimum of 5% of the hours you spend providing behavior-analytic services each calendar month.”
Five percent is the floor, not a description of what every provider does, and plenty exceed it. But do the arithmetic on the floor. At the 20 plus hours a week my son was receiving, that is one hour of supervision. For the other ninety-five percent, the person in the room with my child was the person in the room with my child.
I am not blaming those technicians. Most were kind, and several were barely out of school themselves and had been handed a caseload. The point is that when I asked to speak to someone about my son’s distress, the structure did not really contain anyone whose job that was.
What I ask now
I still read progress reports. I read them differently.
- What is being counted here, in plain words? If the answer is compliance with your goals, say so.
- What are you measuring that could tell us this is not working? If every measure is a progress measure, the report cannot fail.
- Who collected this, and how much supervision did those hours have?
- What does my child look like an hour after a session, and who is recording that?
- If I told you he is distressed, what would you change? If the only available answer is more hours, that is the answer to a different question.
- What would you recommend if coverage were not a factor?
That last one has become my favourite question to ask any provider, whether they take insurance or not. It surfaces, quickly and usually honestly, who the plan is being designed around.
What I am not saying
I am not saying every ABA provider works this way, or that the people we met were acting in bad faith. Most of them were doing what they were trained to do inside a structure that rewarded exactly what it measured.
I am also not saying my experience is evidence. It is one child, one clinic, one family. The research I have linked here is real and worth reading, but it does not prove what happened to my son, and I would not want anyone to read it that way.
What I am saying is narrower. If the paperwork says progress and the child in front of you says otherwise, you are not confused and you are not failing to understand the data. It is worth asking exactly what that number is counting. In our case the answer was: how well he did what he was told.