China developing AI tool to predict who could pose political risk

A Chinese company has been trying to develop artificial intelligence-powered technology that would enable authoritarian governments to not just monitor dissidents but also potentially predict who could become one in the future. 

 

The work, which appears to be in the research stage, is ripped out of dystopian science fiction, offering a glimpse of a world in which an authoritarian state is able to move against its citizens before they begin any public dissent. 

 

The Chinese company, Geedge Networks, sells a commercial version of the Great Firewall, the surveillance and censorship software that China uses to control online activity. 

 

Those tools allow governments to monitor internet traffic and flag when someone tries to get around traditional internet censorship. 

 

Such technology, if perfected, would give authoritarian governments a powerful tool to use against perceived enemies. The idea that an authoritarian government would use artificial intelligence to suppress dissent is troubling enough. 

 

But the use of AI to predict dissent well before a person has taken action has become a nightmare scenario, according to some involved in the industry. 

 

“This is what happens when mass surveillance meets AI,” said Brett J Gold- stein, the director of the Wicked Problems Lab at Van- derbilt’s Institute of National Security. 

 

The Vanderbilt researchers found that Geedge, working with its government-supported research arm, Mesa Lab, was developing technology that would generate profiles of Chinese citizens and then use AI to highlight who may pose a political risk. But the company’s progress appears to have been hampered by Biden-era export controls on US- designed computer chips that power artificial intelligence. 

 

That suggests that US restrictions may have slowed China’s development of the next generation of surveillance technology. Today, Geedge has access to enough graphic processing units for its current products, according to US officials. But to carry out the most ambitious version of its predictive technology, the company would probably need more advanced chips than China can acquire, the officials said. 

 

The Trump administration has weakened some of the Biden-era export controls and kept restrictions on the most advanced processors from Nvidia, the US com- pany that designs the world’s most powerful artificial intelligence chips. 

 

During President Trump’s recent trip to Beijing, US officials said that China would have access to a more advanced version of the Nvidia chips. But China is trying to wean its artificial intelligence companies off US-designed chips, so that export controls can no longer constrain their ambitions. 

 

The new information was gleaned from a trove of data, including 100,000 documents from Geedge that were originally posted last September. 

 

Working with the documents, Wired and other publications have outlined how Geedge exported its network security software to countries including Ethiopia, Kazakhstan, Myanmar and Pakistan, enabling them to conduct mass surveillance on mobile networks. 

 

In the first months of 2024, according to the Geedge documents, the company’s researchers were working to develop behavioral profiles of people based on telecommunications, social media and location data. 

 

AI models were used to classify people and to “detect harmful information,” often a euphemism used by the Chinese Community Party to identify political dissent or other material the government wants suppressed. 

 

Geedge’s research was in some ways similar to work by another Chinese company with ties to the state, GoLaxy. Last year, Vanderbilt and The New York Times outlined GoLaxy’s efforts to develop artificial intelligence-powered software that would push targeted propaganda — themes supported by the Chinese government and against views that Beijing opposed. 

 

But as the Geedge documents show, artificial intelligence models have sped the development of such tools. 

 

But officials acknowledged that Chinese firms were working toward such technology to fine-tune their surveillance state. And China’s Public Security Bureaus have been racing to use DeepSeek to pursue other predictive policing technology, according to experts and government officials. 

 

But predictive tools, particularly if the company eventually incorporates intercepted phone calls or surveillance video, could come up against constraints on computing power. If the company tried to tap into those kinds of surveillance streams to predict security threats, it would more quickly come up against the limits of China’s current computing power, American officials said.