Linda Lombardi
There was a time when formal shelter behavior evaluations for dogs seemed like an important breakthrough. But while an objective way to assess behavior seems like a great idea, it’s increasingly clear that these batteries of tests aren’t doing the job it was hoped they’d do.
In 2016, Gary Patronek, DVM, and Janis Bradley published a paper analyzing the literature on canine behavior evaluations, concluding that they were “no better than flipping a coin.”
This did not prompt immediate abandonment of these tests and there were some understandable questions about their conclusion. How was it possible for their review of the literature to conclude that these tests were not valid and predictive, when individual papers said that they were?
In a new paper they address those questions, delving into statistical issues, the difference between colloquial and technical uses of words like “valid,” “predictive,” and “reliable,” and the kind of evidence needed to justify using a test for real life–and sometimes life and death–decisions.
Testing the Tests
Standards exist for judging whether a test has real-world applicability. “This is not something we need to reinvent,” Bradley says. “There is robust human diagnostic test literature that tells us how to do this and what acceptable standards are, and behavior evaluations in dogs–at least the ones that we looked at, that were intended to have application to dogs in shelters–simply don’t meet any reasonable standard.”
One problem with research on predictive ability of behavior evaluations is that most of it has been done on owned dogs, not dogs in shelters.
“Epidemiologists will tell you there’s no such thing as saying a test is valid,” Bradley says. “All you can ever say is that we’ve demonstrated validity with this population, in this context, with this exact instrument. If you change any of those things–and likely all of those things are going to change in any application in a shelter–any claims you make for validity go completely out the window.”
Many shelters modify protocols, and it’s impossible to perform a test as consistently in the shelter environment as in a lab. Tiny details can change results. One study showed that factors like the evaluator’s height and how much they leaned over changed the outcome. “Try to imagine the world where you’d have the kind of consistency that would eliminate confounds like that in a shelter environment,” Bradley says. “It could never happen.”
The context is also different, and in an important way. The ability to identify aggression is of greatest concern in these evaluations, and results can vary by environment.
“The most common underlying motivations for aggression are fear and anxiety because aggression is distance-increasing behavior,” says veterinary behaviorist Wailani Sung, DVM, of the San Francisco SPCA. “Dogs in the shelter are already lower in tolerance because their overall stress level is so high. So they could test more negatively in a shelter environment versus in a home where they have more stable relationships and a more consistent environment.”
Another problem is that serious aggression is rare in dogs, and those dogs probably don’t get evaluated at shelters. “Most shelters eliminate the tail end of the bell curve without ever subjecting them to a behavior evaluation,” says Bradley. “If everyone’s afraid to go into the kennel with the dog, they’re not going to take it out and do a behavior evaluation.” For statistical reasons, it’s more difficult to design a reliable test for a rare behavior if there’s a concern about false positives. In this instance, false positives–a result that claims to show a dog is aggressive when he isn’t really–can result in unnecessary euthanasia.
“This entire enterprise to a certain extent appears to be based on an idea that there’s some elevated risk with regard to shelter dogs,” Bradley says. “There’s no evidence that this is the case.” Shelter dogs are no more or less likely to someday show aggression.
What’s the Alternative?
Although there’s a considerable body of research questioning the validity of behavior assessments, it takes time for that to trickle down to the real world. “There’s a huge discrepancy between the academic world that has been studying these things versus the practical world that’s out there on a day-to-day basis that still has the pressure of putting safe animals out into the world,” says Jeannine Berger, DVM, vice president of rescue and welfare at the San Francisco SPCA.
Change is hard, and this is a hard kind of change. “Sometimes you don’t want to be shown that what you’re doing is not helpful and may be harmful. That’s a bitter pill to swallow,” says Dr. Sung.
Dr. Berger says discussion about behavior assessments has increased since the ASPCA came out with a position statement last year referring to this literature. “People actually in the field are now starting to question if this is the right approach to making those decisions,” she says.
Amy Marder, VMD, who spent much of her career working on these types of assessments, now believes the best approach is for staff to observe dogs in day-to-day interactions and keep good records. This doesn’t have to involve more work or resources, since it happens during physical exams and walks, which need to be done anyway. “You can get a lot of information from doing routine procedures rather than a formal evaluation,” she says.
This is what San Francisco SPCA does. “We use the five freedoms model to assess welfare, and everyone is trained on that,” Berger says. “We are looking at, are they eating, are they drinking, are they comfortable in their environment, are they fear free and distress free, are they performing normal behaviors, are they free of illness and disease,” she says. “We use that to assess each animal every day, and anything out of the norm is reported and is addressed.”
This article was reviewed/edited by board-certified veterinary behaviorist Dr. Kenneth Martin and/or veterinary technician specialist in behavior Debbie Martin, LVT.
