Flock says its new tool will help identify police abuse, but hasn’t explained how it works

Surveillance technology startup Flock announced a series of new policies and tools on Thursday, claiming they will help curb abuse of its automated license plate camera reader systems and help keep its law enforcement customers accountable.

Flock is now recommending police departments retain data for seven days rather than the previous 30 days. Along with this change, the company introduced what it calls “Evidence Mode,” which can be turned on when customers need to retain data for longer periods in “exceptional cases” and requires a specific case number. Customers will also now be able to set limits on the data they share with other customers based on the type of offense that’s under investigated. 

The company also highlighted a tool called “Audit Assistance” as one of the key changes. Flock launched the tool in April. Since then, Flock said “more than one-third” of its customers have turned it on, and the startup is now requiring all customers to enable it by the end of the year.

Flock said Audit Assistance “detects abnormal activity and flags it for Administrator review,” according to a blog posted released Thursday. If that happens, the tool can automatically lock out users who have been flagged until an administrator intervenes. According to Flock, the tool has already helped customers spot and stop abuse.

Several executives have touted the capabilities of Audit Assistance. But despite its stated importance, Flock has not explained in detail how Audit Assistance actually works.

Some official comments provide some understanding of what Audit Assistance can do; it gets vague when trying to determine how it does it.

For instance, Flock’s head of trust and compliance Ashley Haber said in a video that the Audit Assistance can flag “what may look like an odd search history from a specific user,” helping customers understand how their users are operating the system. In another video, Flock co-founder Paige Todd said “Audit Assistance actually flags unusual patterns early. Think a user searching the same plate repetitively for more than 30 days.”

And Major Patrick Krieg of the Dunwoody, Georgia Police Department, a Flock customer, described the feature in a press release as “an algorithm that has notified us of any type of bias.”

Flock describes Audit Assistance as a feature that can “surface atypical activity early and review it quickly with a clear, documented workflow.”

What’s unclear — and TechCrunch has asked for answers about — is how this “algorithm” actually works, what data is it trained on, and if it was developed using an large language model. TechCrunch also asked what kind of patterns is it designed to surface and what exactly gets flagged as “abnormal activity”? 

According to Flock, Audit Assistance is not based on [Machine Learning] or AI. The company told TechCrunch that “It is a data tool that flags atypical search patterns for further review. One example that would trigger a flag is if a user searches for the same plate using multiple different case codes, as this could potentially indicate abuse. A flag does not mean that it is definitely abuse — it only means it warrants further investigation.” 

Numerous questions remain unanswered, however, including whether Flock has any statistics or data that shows how effective the tool is, what its false positive and flase negative rates are, and if it has been audited by a third party?

Privacy experts and Flock critics are not impressed by Audit Assistance. 

The American Civil Liberties Union senior policy counsel Chad Marlow wrote in response to Flock’s announcement that “there is no evidence that the tool works consistently” to address abuse. 

“While dozens of officers have recently been arrested, fired, or otherwise disciplined for misusing Flock for personal reasons, Flock claims these arrests are proof its auditing tool works,” Marlow wrote. “However, unless we know the number of officers misusing the system, we cannot conclude if Flock and its auditing tools are catching 95% of violators or 5%. Flock needs to have its auditing tool analyzed by an independent evaluator to determine its actual effectiveness. Until then, we don’t know if the tool is a real security measure or just window dressing.”

Cooper Quintin, a security researcher and senior public interest technologist at the Electronic Frontier Foundation, told TechCrunch that it may not even matter whether Audit Assistance works. 

“If it does work, law enforcement will figure out ways to get around it,” he said, adding that what matters in the end is whether abusers are caught and held accountable. “If there are no consequences for abuse and whoever [the tool] reports to is the same police agency that is doing the abuse, it’s just a fig leaf for Flock. The best way to ensure accountability is to pass laws constraining the use of this technology and requiring a warrant for its use.”

Rachel Levinson-Waldman, the director of the Brennan Center’s Liberty and National Security Program, told TechCrunch that making Audit Assistance mandatory is a step in the right direction, and that “in theory” the tool is “a great idea.” But she also said the tool itself “needs to be audited” because there’s not enough information about how it works in practice.

Flock’s announcements come after years of reported incidents of its customers repeatedly abusing its technology. Earlier this week, three former Georgia sheriff’s deputies were arrested and accused of using Flock cameras to stalk people with whom they had a “personal relationship at the time,” as the Bibb County Sheriff’s Office said in a press release. In this case, the Bibb County sheriff David Davis specifically called out Flock’s new tool Audit Assistance for uncovering the misuse.