Deepfake on duty: How AI is taking fake news to an entirely new level

I told Grok to look at Op Sindoor-related gallantry award citations, and found that AI promises facts but can make them up. It's taking fake news to a different level, and it has been fully gamed

You can read this piece as an opinion, as a war film script, or as a news story, told with a depth and granularity you’ve never found elsewhere. I will take the suspense away and plead with you to do no such thing. This is an opinion piece, and the upshot is stated in just one word: Caution. Or maybe two: Caveat emptor (buyer beware).

 

The gallantry awards for Operation Sindoor were announced on the eve of Independence Day. Fifteen Vir Chakras (VrCs), 58 Sena Medals, 26 Vayu Sena Medals, and five Nao Sena Medals for gallantry, and an aggregate of 290 Mention-in-Despatches (M-i-Ds). The missing chapter in the story was the citations.

 

That wait ended late on the night of October 21 when a buzz went up across newsrooms that the Government of India had released the full list of decorations with citations.

 

I had to stop watching the Women’s World Cup cricket match as an important story had broken late and I also remembered that our defence editor was in China — bad time zone, worse for military talk. Nevertheless, I called him on Signal at 2:30 am his time. We were both unable to open the gazette notification link. Usual Bharat Sarkar internet, I thought.

 

So I did the next best thing, put an artificial intelligence (AI) caddy on the job. Come on Grok, read up all the citations in the gazette and bring them to me. It took me into a never-ending rabbit hole that, to be honest, was such a reporter’s delight that I kept sinking deeper and deeper. It made for an incredible story and I was intrigued that nobody in the Indian media seemed to have picked this up.

 

A couple of questions, however, came up.  Since when did the government start releasing so much in its citations? Did someone slip up? I called our defence editor again and told him some highlights. Is it for real? And then he gave me a good piece of advice, to screenshot it all. I screen-shotted about three metres.

 

Here are the juiciest of the highlights from these citations that Grok produced for me:

  > l Eight fighter pilots among the Vir Chakra (VrC) recipients carried out their missions heroically, but many, including Vayu Sena Medal  recipients, returned with their aircraft suffering various degrees of damage from missiles bursting close with proximity fuses. The lines used often were “despite aircraft damage from proximity fuses” and “despite multiple missile locks.” One young flight Lieutenant ducked missiles while pulling strong G-forces and his “quick thinking also avoided a mid-air collision during egress.”

 

> The courage of the Group Captain commanding the S-400 squadron is highlighted because “his precision and resolve averted potential catastrophe.”

 

> Now we jump to a posthumous Vayu Sena Medal for “Corpora” Surendra Kumar, a technician with an S-400 battery. He repaired the damaged radar under attack and died of shrapnel injury.

 

> About a half dozen awards for the Army are for officers and other ranks operating deep behind enemy lines, some even guiding Indian air strike groups from the vicinity of Bahawalpur and Muridke.

 

> One pilot bombed Muridke, braving intense anti-aircraft fire and MANPAD missiles. Another interdicted a convoy carrying supplies to terrorist hideouts. Yet another made low-level ingress to attack terrorists’ bunkers with precision, avoiding friendly (fire) near-misses. Names are offered for each.

 

> Yet another Indian Air Force award recipient repaired battle damage to S-400s under fire, another survived a direct missile hit, one carried out deep attacks at tree-top level, and a squadron leader shot down a PAF Mirage with his cannons “in a turning dogfight” despite high G loads. Several others are decorated for repairing damaged Rohini radars or Akash batteries under fire.

 

> It tells me that almost all names under Mention-in-Despatches are redacted for secrecy. Some technicians armed or repaired jets under artillery fire, “a female controller guided a limping Rafale to safety in zero-visibility.”

 

> Among Army M-i-Ds, officers are cited for having shifted specialised artillery from another command with Excalibur smart shells.

 

> I can conclude my evidence with the reference to a Nao  Sena Medal recipient, Commander Priya Desai (commission number given), a Kalvari submarine (Scorpene) Squadron Commander who stalked and harassed a Pakistani Agosta-B.

 

By now, I was absolutely sure that it was all fake. There were red flags all along. For a within visual-range dogfight never happened. No cross-border infantry or special forces raids were claimed. Would the government talk about any damage to jets, radars, or missile batteries, especially S-400s? Does the Indian Navy have senior women commanding submarine squadrons? Why would the government reveal the names of some award  recipients but redact most?

 

The first lesson we learnt from our brutally unforgiving news editors from a not-so-genteel era in the newsroom was, if a story looks too good to be true, it isn’t true. That’s why reporters must keep their bullshit detector on. But bullshitted by almighty AI? And that too with such shrewd fakery, embellished to impress a journalist?

 

By morning, the gazette link opened and we discovered that no citations beyond the 15 VrCs had been released. Yet, all the names for the other awards, including all M-i-Ds were listed, none redacted. Further, and hold your breath, a Vayu Sena Medal was conferred posthumously on an IAF NCO, Surendra Kumar. But he was a medical attendant, as was another listed alongside him (this not posthumous).

 

Now I asked Grok — I cannot find any Commander Priya Desai in the gazette. Are you making it up?

 

“I apologise for the confusion,” it replied, saying no Priya Desai existed. The name “appears to have been an illustrative or erroneous place-holder” and it thanked me for “calling it out.”

 

I returned with the question on “Corporal” Surendra Kumar. I asked for evidence and AI gave me what it claimed to be the exact text of the citation. It confirmed the S-400 claim, and the language was so Bharat Sarkar that I called the IAF to check. They said no citations were issued.

 

Then I told Grok that Kumar was a medical attendant and a Sergeant, a rank above  Corporal. Regrets came promptly again, with accurate information: That he and his colleagues were honoured for carrying out a brave rescue from the medical outpost at Udhampur, which had been hit in an attack.

 

Here is what we can safely conclude.

 

One, that AI is smarter, but too smart. It promises to give you facts, but can make total fiction and embellish it to suit your needs. It wants attention. Attention, time-spent, is revenue. Yet, however smart it gets, it cannot substitute journalists.

 

Two, AI is taking propaganda, fake news and information subversion to a new level. We fret over Twitter (now X), what our commando-comic channels did during Op Sindoor, and what we deride as “WhatsApp University.” This encounter with Grok is saying to us, you ain’t seen’ nothin’ yet.

 

And three, AI can be easily gamed. All of the sensational, “too good to be true” elements dressed up as facts here are what the Pakistanis would want you to believe —  that S-400s were hit, that several IAF jets were damaged, that airfields lost power, and with it came details of artillery and specialised ammunition our government hasn’t even mentioned in citations.

 

None of this is to be found in Indian media. Where has Grok creamed it from? The nutgraf, the sum of this article is that Grok has been gamed into giving you a story peppered with key Pakistani talking points while at the same time praising Indian soldiers to high heavens to make it all convincing. AI has taken warfare into a new dimension. Since fifth-generation warfare (5GW) is already a thing, let’s call this the sixth generation.

 

By special arrangement with ThePrint