Talking to Judah in our reading club helped me to crystallize some of my thoughts about Ali Alkhatib & Michael Bernstein’s Street-Level Algorithms paper. This post explores these.
Thinking about this paper again is making me appreciate it more. As a reminder, the PDF can be found here.
On real-life bureaucrats
One of the challenges I had reading the paper was that a recurring example of street-level bureaucrats were police officers.
It’s true! They are. They absolutely fill the role of manifesting institutional policy. But reading this paper in 2021, after a summer of police brutality being front and center, meant that I have a different perspective than I would have two years ago.
Consider this quote comparing the reflexivity of algorithms and bureaucrats:
The algorithm is always behind the curve: at best, it gets feedback or negative rewards only after it has executed its decision, in this case when YouTubers appeal or gather media attention. By contrast, police reflexively construct an interpretation of the situation as soon as they encounter it, rather than merely match on learned patterns.
This is, again, true. Police reflexively react to new situations. Yet I kept thinking “but they do such a terrible job at it so often!”
Are we so confident that humans reacting reflexively is always a good thing? In the case of police, it might not be. (On the other hand, what would be better?)
On regulation
Street-level bureaucrats have often been agents of discrimination and unfairness, historically speaking. They might enforce discriminatory policies (redlining) or use discretion in a discriminatory way (traffic cops stopping drivers of color).
If we’re comparing street-level algorithms to street-level bureaucrats, then we should also consider the institutions and policies that have let street-level bureaucrats be agents of discrimination.
I wrote this yesterday:
Put otherwise, we shouldn’t expect to oversee algorithmic systems in the same way that we oversee bureaucratic systems. Existing regulation still allows bureaucracy to discriminate or cause harm. Algorithms can do this too—faster and at a greater scale.
We should rethink the entire system. Not just the algorithms and the bureaucrats, but the entire decision-making system, along with the oversight and regulatory apparatus.
On generalization and robustness
Another subject that Judah and I tossed back and forth was the generalizability and robustness of algorithms. We posited the idea that this paper was ultimately talking about the need for algorithms to be more generalizable and robust.
Upon reflection, I think there’s more. I think the authors try to ground ideas that computer scientists have been talking about for a while—generalizability and robustness—and ground them in the human tasks of bureaucracy.
Humans generalize every day without even having to think about it. Humans can “fill in the gaps” of new situations; this is one of our best features! This paper reframes abstract, algorithmic concepts by grounding them in the human language of street-level bureaucrats.
On thinking bigger
Finally, I’ve been thinking a lot lately about bigger picture problems in sociotechnical systems. One of the questions raised in Safiya Noble’s Algorithms of Oppression, which I’m reading now, is whether or not we even need Google search—whether the goal of cataloguing everything on the internet is a valid one, and whether or not that goal causes harm.
I’d never thought about search in this way. Search, to me, was always something that exists; and all of the discourse surrounding it would focus on how to make it better. Her book has helped me to rethink it, and other systems, by asking “should this exist?”
I experienced a similar feeling reading this paper. The authors discuss street-level algorithms as a modern-day version of street-level bureaucrats. But on reflection, I questioned why we should expect algorithms to replace humans in this way.
Shouldn’t we instead be taking a step back and rethinking all of the decisions we’re asking an agent—human or algorithm—to make? Instead of just accepting that algorithms will replace bureaucrats, reconsider the whole design.
The paper’s third case study on bail recommendations included a case of a street-level algorithm interacting with a street-level bureaucrat. I would have loved to see this interaction discussed further. I think human-in-the-loop systems where algorithms and humans are working alongside each other will be even more common in the future.
Closing thoughts
Talking to someone else about this paper helped me to appreciate it in more depth. Grounding the discussion of street-level algorithms in the human ability to fill in the gaps is a complex idea, and I think it’ll take a while before I fully appreciate it.
I think that’s common with papers that reframe concepts in a new way. After some time, I expect to start thinking about more algorithmic systems in terms of their street-level capabilities. And so I appreciate this paper for expanding my frame of reference in this way.