Tracked: How Colleges Use AI To Monitor Student Protests

The pitch was enticing and easy.

For a number of thousand {dollars} a yr, Social Sentinel supplied colleges throughout the nation refined know-how to scan social media posts from college students prone to harming themselves or others. Used appropriately, the device may assist save lives, the corporate stated.

For some faculties that purchased the service, it additionally served a unique objective — permitting campus police to surveil scholar protests.

Throughout demonstrations over a Accomplice statue at UNC-Chapel Hill, a Social Sentinel worker entered key phrases into the corporate’s monitoring device to seek out posts associated to the protests. At Kennesaw State College in Georgia 5 years in the past, authorities used the service to trace protesters at a city corridor with a U.S. senator, data present. And at North Carolina A&T, a campus official advised a Social Sentinel worker to enter key phrases to seek out posts associated to a cheerleader’s allegation that the college mishandled her rape criticism.

An investigation by The Dallas Morning Information and the Investigative Reporting Program at UC Berkeley’s Graduate College of Journalism reveals for the primary time that as extra college students have embraced social media as a digital city sq. to specific opinions and arrange demonstrations, many school police departments have been utilizing taxpayer {dollars} to pay for Social Sentinel’s providers to observe what they are saying. A minimum of 37 faculties, together with 4 in North Texas, collectively educating tons of of hundreds of scholars, have used Social Sentinel since 2015.

As a nonprofit journalism group, we rely in your help to fund greater than 170 reporting tasks yearly on essential international and native points. Donate any quantity immediately to turn out to be a Pulitzer Middle Champion and obtain unique advantages!

The true variety of faculties that used the device could possibly be far greater. In an e mail to a UT Dallas police lieutenant, the corporate’s co-founder, Gary Margolis, stated it was utilized by “tons of of faculties and universities in 36 states.” Margolis declined to touch upon this story.

The Information examined hundreds of pages of emails, contracts and advertising materials from faculties across the nation, and spoke to highschool officers, campus police, activists and consultants. The investigation exhibits that, regardless of publicly saying its service was not a surveillance device, Social Sentinel representatives promoted the device to universities for “mitigating” and “forestalling” protests. The paperwork additionally present the corporate has been shifting in a brand new and doubtlessly extra invasive path — permitting colleges to observe scholar emails on college accounts.

For faculties struggling to reply to high-profile faculty shootings and a worsening campus psychological well being disaster, Social Sentinel’s low-cost device can appear to be a great deal. Along with the handfuls of faculties that use the service, a Information investigation final yr revealed that no less than 52 faculty districts in Texas have adopted Social Sentinel as a further safety measure since 2015, together with Uvalde CISD the place a gunman killed 19 kids and two lecturers in Might. The corporate’s present CEO J.P. Guilbault additionally stated their providers are utilized by one in 4 Okay-12 colleges within the nation.

Some consultants stated AI instruments like Social Sentinel are untested, and even when they’re adopted for a worthwhile objective, they’ve the potential to be abused.

For public faculties, the usage of the service units up a further battle between defending college students’ Constitutional rights of free speech and privateness and colleges’ obligation to maintain them protected on campus, stated Andrew Guthrie Ferguson, a regulation professor at American College’s Washington School of Regulation.

“What the know-how lets you do is determine people who’re related collectively or are related to a spot or location,” stated Ferguson. “That’s clearly considerably chilling for First Modification freedoms of people that consider in a proper to protest and dissent.”

Navigate360, the personal Ohio-based firm that acquired Social Sentinel in 2020, referred to as The Information’ investigation “inaccurate, speculative or by opinion in lots of cases and considerably outdated.” The corporate additionally modified the identify of the service from Social Sentinel to Navigate360 Detect earlier this yr.

Picture by Ari Sen/The Dallas Morning Information.

In an interview in February, Guilbault stated the service doesn’t have the power to observe protests.

However paperwork from Kennesaw State present campus police tracked demonstrators’ on-line exercise for days with Social Sentinel earlier than a contentious 2017 city corridor.

Brandy White, a felony intelligence analyst in KSU’s police division, was in command of the monitoring. On instruction from her supervisors, White entered details about demonstrators and protest teams into Social Sentinel’s monitoring device and arrange searches to seek out posts concerning the occasion, emails present.

White additionally obtained a KSU police intelligence briefing from a colleague concerning the occasion. The doc, obtained by The Information in response to a public data request, singled out one progressive activist group, the liberal grassroots community Indivisible, and cited conservative conspiracy theories that George Soros funded the protesters.

The report additionally recognized a younger progressive activist named Marisa Pyle.

On the time, Pyle was a 19-year-old scholar at a close-by school. She advised The Information that she had no thought about KSU’s monitoring.

“There isn’t any model of that the place I wanted any type of particular consideration or surveillance by Kennesaw State.”

“An ear to the social panorama”

The thought for Social Sentinel got here out of a police briefing within the mid-2000s.

A sergeant named Jimmy had extra to say than common.

He advised Margolis, then the police chief on the College of Vermont in Burlington, a couple of sequence of petty thefts, a sexual assault and a protest deliberate on the government workplaces of the college.

“Jimmy, the place did we get this data?” he requested.

The sergeant despatched Margolis to see an in a single day dispatcher, who confirmed him the supply — new on-line websites, together with a web site referred to as The Fb.

It was in that second, Margolis later stated, the seed for Social Sentinel was shaped.

“It grew to become clear that we would have liked a digital beat cop – ‘somebody’ with an ear to the social panorama,” he wrote in a 2015 weblog put up on the Social Sentinel web site.

In 2008, in his remaining yr as police chief, Margolis merged his consulting service with one other began by Steven Healy, a former army police officer who beforehand held prime security roles at Princeton and Wellesley. 4 years later, the 2 introduced Campus Sentinel, an app that aggregated crime stats on school campuses.

It could take one other two years earlier than Margolis may notice his imaginative and prescient for an app that saved tabs on on-line exercise. In 2014, with help from a Boston-based personal fairness agency and a New York-based enterprise capital firm, the duo rebranded the service as Social Sentinel, a social media risk alert service.

The College of North Texas was an early adopter.

The varsity’s police chief, Ed Reynolds, advised The Information final yr that the college scheduled a presentation with the corporate after listening to success tales from different chiefs at a convention.

The presenters billed Social Sentinel as an AI device that would detect threatening tweets concerning the UNT campus, and confirmed Reynolds the way it could possibly be used to seek out posts.

After contemplating different competing providers, the UNT police chief signed a $15,000-a-year contract in December 2015.

In November of that yr, a Social Sentinel worker despatched Reynolds recommendations for tips on how to use the service, emails obtained by The Information present. In a single part, the corporate proposed the college undertake a rule stopping them from monitoring protests, besides once they threatened public security.

It’s unclear whether or not UNT adopted the advised coverage — no signed copy exists within the data offered to The Information. Except for the proposed procedures despatched to the North Texas faculty, Social Sentinel appears to have supplied faculties few guardrails for tips on how to use its service. The Information reached out to each college identified to make use of the service — not one of the faculties that responded stated that they had guidelines particularly governing Social Sentinel.

Reynolds stated UNT stopped utilizing the service in 2018, after figuring out they might deal with the monitoring in home. That very same yr, Collin School signed a contract with Social Sentinel and UT Dallas renewed its contract for an additional three years.

A minimum of one school in North Texas nonetheless makes use of the monitoring device. Nicole Shaw, the police chief at North Central Texas School, stated the college’s newest contract for the service price $12,750 a yr.

Shaw stated she knew of just one occasion the place the service alerted to a doubtlessly troubling state of affairs. In that case, the division was already conscious of the scholar who wrote the put up and the scholar was not an imminent risk to himself.

Probably suicidal college students aren’t the one factor the faculty is monitoring for.

“Protests, that’s one of many buzzwords,” she stated.

“We do not take sides, we simply need to ensure that everybody has a protected expertise.”

The whitepaper

Social Sentinel has lengthy stated its service can’t be used for monitoring protests.

However emails in 2017 between campus police officers at Kennesaw State present the college used the service no less than thrice for that objective.

Along with the city corridor assembly, the college monitored protests in opposition to President Trump’s 2017 ban on vacationers from a number of Muslim-majority nations and demonstrations in opposition to the college’s former chancellor, Sam Olens.

A number of spokespeople from Kennesaw State declined repeated requests for interviews for this story.

Paperwork present the service was a key device for the college’s police division in 2017. Members of the employees reported what number of tweets have been flagged of their month-to-month conferences that yr. The service confirmed up in a 2017 job description outlining the each day duties of a felony intelligence analyst, saying the position included monitoring Social Sentinel for “crime, civil unrest, protests” and extra.

Guilbault stated the service doesn’t have the power to observe protests, however declined to touch upon any actions earlier than their acquisition of the corporate in 2020.

“The phrase protest isn’t even in our engines,” he stated, referring to the know-how the corporate says it makes use of to investigate social media posts.

Paperwork The Information obtained from colleges across the nation present firm representatives repeatedly promoted Social Sentinel’s capability to observe protests.

In 2017, a marketer working with the corporate, David Smitherman, reached out to a UNC-Charlotte administrator, Christine Davis, to tout the service.

Learn the entire doc right here. Picture courtesy of The Dallas Morning Information.

Smitherman had contacted Davis no less than two different instances saying the service could possibly be used for forestalling doubtlessly unstable protests, emails present. In a November 2016 message to a different administrator, he wrote that it could possibly be used for mitigating the impacts of “disruptive demonstrations & protests.” Across the similar time, representatives used the identical language of “mitigating” and “forestalling” protests in messages to Stephen F. Austin State College and UNC-Asheville.

This time he added one thing totally different — a whitepaper from the corporate titled “Demonstrations and Protests: Utilizing Social Media to Collect Intelligence and Reply to Campus Crowds.”

The doc outlines how the service can be utilized to observe earlier than and through demonstrations, and likewise mentions that Michigan State College used the service to observe protests. Emails from 2015 and 2016 obtained by The Information additionally present the whitepaper was despatched to no less than three different faculties — UT Dallas, Appalachian State and UNC-Asheville — contemplating the service.

“Throughout an occasion, risk alerts can present vital perception concerning the leaders or agitators who might desire a confrontation with regulation enforcement, the overall local weather of the gang and the potential for crowd development,” the doc says.

Davis advised The Information that she didn’t comply with up with Smitherman and the college didn’t search a service to observe protests. Smitherman declined to remark for this story.

“Monitoring is Icky”

From the start, Social Sentinel’s co-founder, Margolis, stated publicly that the service couldn’t be used to surveil particular person individuals.

In emails to shoppers, the corporate had a unique message.

“I hear that you’re inquisitive about importing usernames, consumer accounts, and many others to comply with identified threats,” one worker wrote to former North Carolina State College Police Chief Jack Moorman in August 2015. “We not too long ago launched that characteristic, and now you can add a listing of Twitter authors.”

“I admire the knowledge on the characteristic of importing the accounts of people that we need to comply with,” Moorman wrote again. “This needs to be an excellent profit to ongoing risk evaluation circumstances.”

NC State declined to touch upon Moorman’s actions, however stated Social Sentinel was not used to focus on people or teams. Moorman didn’t reply to requests for remark.

Paperwork present colleges might have used one other characteristic of the service referred to as the Native+ Listing to focus on protesters and particular person college students. Of the 37 faculties recognized by The Information as utilizing the service, solely two, Oregon State and the College of Connecticut, offered their Native+ Listing in response to public data requests.

A minimum of one North Texas school — UT Dallas — may have used the characteristic to observe people. Vanessa Balderrama, a venture coordinator for the college, stated the college couldn’t launch its Native+ Listing as a result of it contained college students’ names.

North Carolina A&T, a traditionally Black school in Greensboro, might have used the checklist characteristic in a method that will reveal posts essential of the college’s management after a cheerleader stated her coaches mishandled her rape criticism. Emails from the college in February 2019 present a Social Sentinel worker suggesting a lengthy checklist of key phrases and phrases associated to the protests to the college’s social media supervisor.

“Sure, please add these in,” the supervisor wrote in response.

North Carolina A&T Affiliate Vice Chancellor Todd Simmons denied utilizing the service to observe the demonstrations, saying the e-mail was “an effort to upsell us to the Native Plus product.” In its investigation, The Information discovered no proof that the Native+ Listing price colleges extra money.

The Native+ Listing resembles one other device common with regulation enforcement across the similar time — the CIA-funded service Geofeedia.

The North Carolina faculties have been aware of the competing device — no less than two of the faculties’ police departments have been a part of a regulation enforcement group run by North Carolina’s State Bureau of Investigation, which had a contract with Geofeedia.

Paperwork obtained by The Information present UNC-Chapel Hill’s police division requested the state bureau to surveil professional and anti-abortion protesters in October 2015. The investigative company rapidly obliged. Emails between the companies present SBI tracked key phrases and hashtags together with “Feminist College students United,” “College students for Reproductive Justice,” “#feminist,” “#studentsforlife” and even “#unc.”

In 2016, the ACLU of Northern California disclosed that regulation enforcement companies had used Geofeedia to observe Black Lives Matter demonstrators in Ferguson. Quickly after, Twitter, Fb and Instagram introduced they’d droop or restrict Geofeedia’s entry to their service. A month later, Geofeedia laid off greater than half of its employees.

The ACLU revelations raised alarms at Social Sentinel. One former worker advised The Information that executives determined to cease promoting the service to native police departments after they realized their service was additionally used to observe Black Lives Matter protesters.

In a 2017 webinar, Margolis referenced the issues with Geofeedia and stated the corporate shared the ACLU’s fears.

“Our workforce at Social Sentinel had precisely those self same issues again in 2015, and we put a stake within the floor and created the risk alert method for precisely the explanations the ACLU cited,” Margolis advised the webinar’s attendees. “Focusing on and surveilling people or teams isn’t OK.”

“Monitoring is icky,” Margolis added.

Demonstrators at UNC-Chapel Hill protest the Silent Sam statue in August 2018. Pictures by Ari Sen. United States, 2018.

Demonstrators at UNC-Chapel Hill protest the Silent Sam statue in August 2018. Pictures by Ari Sen. United States, 2018.

After the disclosures about Geofeedia, UNC-Chapel Hill sought one other social media monitoring service, and finally signed a $24,500 yearly contract with Social Sentinel in November 2016.

Lower than a yr later, the college surveilled scholar protesters once more.

NBC Information reported that the faculty collaborated with an FBI Joint Terrorism Process Pressure to surveil the social media posts of individuals protesting a Accomplice statue on campus in 2017. It’s unclear which monitoring service the college utilized in that case.

Throughout one other protest of the Accomplice monument a yr later, campus police once more monitored posts, this time utilizing Social Sentinel’s checklist characteristic, emails present.

The Information obtained no documentation from the faculty displaying UNC used the service for something however monitoring protests. The previous campus police chief, David Perry, and Chancellor Kevin Guskiewicz stated in 2020 that they weren’t conscious of any cases when the service had helped to stop violence on the campus.

“The entire haystack”

Some college officers stated they weren’t bothered by the potential privateness implications of Social Sentinel as a result of they stated they solely used the device to observe public social media posts.

“I do not assume it was ever a priority of mine,” Reynolds, the UNT Chief, advised The Information final yr. “The knowledge that we have been taking a look at was public data.”

“My understanding of the product is they will solely draw these feedback from public posts on social media,” stated David Thomasee, government director of operations at Gulf Coast State School. “I am undecided that the expectation of privateness is there if anybody on the planet can see it.”

However even amassing publicly posted data might hurt college students’ privateness and speech rights, consultants say. Jake Wiener, a lawyer on the Digital Privateness Data Middle additionally stated social media monitoring providers might prolong the attain of regulation enforcement companies past what they have been beforehand in a position to monitor.

“One thing like a social media surveillance program goes to comb in a lot knowledge,” Wiener stated. “An officer goes to be much more more likely to examine, and presumably arrest, people who find themselves not really the kinds of targets they’re searching for.”

Ferguson, the regulation professor at American who additionally authored The Rise of Huge Information Policing, stated the usage of social media monitoring instruments could possibly be challenged beneath an interpretation of a 2017 Supreme Court docket Case, Carpenter v. U.S. In that case, the courtroom dominated that the federal government couldn’t seize a suspected robber’s cellphone data, together with location knowledge from cell towers, with out a warrant.

The courtroom’s choice might mirror a shift within the interpretation of privateness regulation to a brand new concentrate on how a lot data was collected somewhat than simply what data was collected.

“You are searching for a needle within the haystack however you have to acquire the entire haystack to seek out the needle,” Ferguson stated.

Universities might also have extra incentives to undertake surveillance know-how than native police.

“We positively hear directors speaking rather a lot like ‘the college is sort of a household and we’re right here to guard you from something unhealthy that may occur on campus,’’’ stated Alex Morey, a lawyer on the Basis for Particular person Rights and Expression.

“That type of mentality would result in a heightened willingness to see these surveillance programs as a great factor.”

E-mail Monitoring

4 months after a gunman killed two individuals and injured 4 others at UNC-Charlotte in 2019, Judy Stanwyck seen one thing unusual.

Stanwyck, a senior purchaser for the college, was dealing with UNC-Charlotte’s buy of Social Sentinel. She wasn’t aware of the know-how so she seemed up the corporate’s web site and began studying.

One characteristic jumped out — the corporate was promoting the power to observe college students’ college e mail accounts.

“I discovered this,” Stanwyck wrote in an e mail to her colleague, together with the hyperlink to the web site. “I might simply affirm precisely what they’re buying.”

Stanwyck didn’t reply to requests for remark.

Within the firm’s proposed contract with UNC-Charlotte, it stated the e-mail monitoring service was solely out there for Okay-12 colleges. However in an e mail to the college’s affiliate vice chancellor for security and safety, an worker stated they might add that service in the event that they wished to.

The corporate initially stated it solely monitored public data on social media. However since no less than 2019, data present it has supplied faculties the power to observe personal scholar emails.

It’s unclear whether or not a school has taken the corporate up on its provide — The Information was unable to discover a college that had used the e-mail monitoring service, although the apply is frequent in Okay-12 colleges.

Privateness consultants stated they have been appalled by the service’s capability to observe personal scholar messages.

“The thought of including surveillance software program to the campus infrastructure the place college students are speculated to study is simply surprising,” Dave Maass, the director of investigations on the Digital Frontier Basis, stated. “It is simply an abomination to the thought of scholastic freedom.”

UNC-Charlotte in the end didn’t signal the contract, citing finances limitations. On the similar time, the corporate was selling the e-mail monitoring service to a different North Carolina school — UNC-Asheville.

In 2019 emails to the college, the co-founder Margolis promoted Social Sentinel’s capability to work with the Google suite of providers and promised they’d work with Microsoft Outlook within the close to future.

One other firm worker marketed the brand new Gmail integration within the signature of each e mail she despatched to the college that yr.

The Navigate360 CEO Guilbault stated some faculties use the service to observe emails. However they will not be stopping there.

In emails to Gulf Coast and Palm Seashore state faculties in 2019, the corporate touted its capability to observe Google Docs, Google Hangouts chats and even Fb Messages.