In January 2013, Paul Kolbe was at a hotel in downtown Mexico City for a meeting of oil and gas company BP’s global security team. He had joined the British multinational five years earlier, after serving 25 years in the CIA’s directorate of operations. Suddenly, his phone began pinging with calls and emails from BP headquarters.
Just before sunrise, a group of armed men had attacked a gas plant in In Amenas, an Algerian town 6,000 miles (9,650 km) away. At the time, BP jointly operated the plant with Libya’s and Norway’s state oil companies, and the attackers, led by an Algerian called Mokhtar Belmokhtar, had tied workers to piping and storage tanks using explosive detonating cord. They were using those workers’ phones to issue demands back to BP HQ.
Kolbe recently told Freethink that his head immediately began swirling after hearing the news: “Who’s doing this? Who do we talk to? Who do we need to engage in the US, UK, and other governments? How is this going to play out? How are the Algerians going to react?”
The site was secured, and a committee set up by BP executives began working to resolve the crisis. A few days later, the Algerian army attacked the gunmen as they tried to leave the plant with foreign hostages. In the ensuing gunfight, 40 workers and 29 attackers were killed.
The In Amenas hostage crisis became a watershed moment for corporate security and the in-house intelligence teams tasked with anticipating and avoiding such tragedies, a reminder of the scope and scale of the threats they had to manage.
“Did we have the fidelity to be able to understand that this group in the deserts of Mali was planning an attack on this facility in Algeria? No,” Kolbe told Freethink.
“It was like something from the Wild West.”
Michael Ard
The history of private sector intelligence is as old as commerce itself.
As intercontinental trade grew in the 17th and 18th centuries, so did the information networks that could give companies a competitive advantage. Traders at the British East India Company and Lloyd’s of London, an insurance market, built sophisticated intelligence-gathering networks, with the latter occasionally outperforming even the Royal Navy in the speed and accuracy of its maritime incident reports.
As electric telegraph cables replaced horse couriers and boats, multinational companies began to receive snippets of near real-time information on the conflicts and catastrophes that could upend their operations. However, it was the early computers created in the depths of World War II that truly brought “mechanization” to intelligence gathering for the first time, Calder Walton, a historian of intelligence at Harvard University’s Belfer Center, told Freethink.
While the Cold War rumbled on through the second half of the 20th century, it was government agencies, not private companies, who held the computational “crown jewels” of intelligence gathering, according to Walton, although they increasingly depended on firms like IBM to help build state-of-the-art systems to collect and mine their ever-growing piles of secret intelligence.
As global supply chains became increasingly intricate, so did companies’ demands for the fine-grained intelligence needed to enter new markets, vet partners, and safely send executives into unstable regions. Alongside modernized in-house teams, the 1970s and ‘80s saw a blooming of risk consultancies and investigation firms that could provide in-depth reporting on opaque regions or arm corporate lawyers for litigation.
Cyberspace opened up a new battlefront for industrial espionage.
With hefty budgets and sprawling operations across high-risk regions, oil companies have often been pioneers in deploying tech for private intelligence gathering—decades before the In Amenas attack, they were using drones in Algeria for protective surveillance, Michael Ard, a former US government intelligence analyst and senior security representative for Marathon Oil, told Freethink.
The birth of the public internet around the turn of the millennium presented previously unimaginable opportunities for private intelligence. Companies and consultants could now monitor local news, coordinate informants, and even breach rivals’ computer systems from the comfort of their desks.
Although cyberspace opened up a new battlefront for industrial espionage, demand remained for more traditional work. In the 2010s, Ard worked on a project for an extractive industry client that involved a consultant who specialized in sweeping office light fixtures and landlines for eavesdropping devices.
“It was like something from the Wild West,” he told Freethink. “The guy shows up with this briefcase, opens it up, and there’s this special machine. He pulls it out and sets it up, and we were just walking around with him, going through everything.”
“Companies are realizing the geopolitical landscape is becoming existentially risky, so they need serious intel teams to help them navigate this stuff.”
Lewis Sage-Passant
By the 2010s, smartphones, social media, and the tumbling cost of satellite imagery brought forth a diverse but dizzying wealth of open-source data.
“The explosion of data, openly available, in the early 2000s onwards, that’s the big, big fundamental dividing line for private sector intelligence compared to the past because it’s not necessarily secret, and one can often acquire just as valuable intelligence through open sources,” Walton told Freethink.
The value of open-source data was proven during the Arab Spring, when private intelligence practitioners were able to correctly anticipate unrest by monitoring social media in Libya and Egypt—several of these practitioners were called into meetings with US government agencies in the aftermath so that their public counterparts could learn how they had done it, according to Lewis Sage-Passant’s Beyond States and Spies, a recently published exploration of private intelligence.
The first years of the 2020s—which included the COVID-19 pandemic, Black Lives Matter protests in the US, and Russia’s invasion of Ukraine—have driven a surge in demand for private intelligence as executive suites work to parse epidemiological data, negotiate civil unrest, offset volatile energy costs, and navigate the potential for sabotage, sanctions, and disrupted international shipping.
“Companies are realizing the geopolitical landscape is becoming existentially risky, so they need serious intel teams to help them navigate this stuff,” Sage-Passant, global head of intelligence for a major pharmaceutical company and an adjunct professor of intelligence at Sciences Po in Paris, told Freethink, adding that geopolitical risk has now become a “core function” for many multinationals.
Software applications can pull in myriad data sources, helping map the links between companies, people, or devices of interest.
The external intelligence vendors catering to this growing appetite vary in size, specialty, and services. Recent research by the Belfer Center’s Maria Robson-Morrow and colleagues found 70 vendors operating in more than 170 countries and offering 265 distinct geopolitical and security services.
“There are many firms that are providing intelligence products and services, but also providing more of what we’d consider security services or regulatory support,” Robson-Morrow told Freethink.
Some of the biggest players are huge consultancies, such as Control Risks, or corporate investigation firms, such as Kroll, with thousands of staff members and offices worldwide. Others, such as the consultancy Sibylline and investigative outfit K2 Integrity, have more modest headcounts but similar reach.
Although each has a distinct emphasis, all the major firms offer counsel on compliance with local laws, risk management, and forensic investigation. They’ll even embed staff with regional or technical expertise within client companies.
Vendor teams might use totally different suites of specialist software, depending on what they are doing for clients. Those working on investigations often turn to Analyst’s Notebook or Maltego, for example. Both applications can pull in myriad data sources, helping map the links between companies, people, or devices of interest, creating a digital version of the corkboard, polaroids, and colored string wheeled out for the finale of a detective thriller.
A crowded market of platforms offer security teams a literal map, allowing them to layer on real-time feeds of crime, civil unrest, or local social media mentions. These dashboards can help plan executive travel and alert staff to an incident near a manufacturing site or retail outlet as it happens. Some early movers, such as Factal and Dataminr, have managed to maintain the accuracy of their alerts in an environment increasingly awash with disinformation, Sage-Passant told Freethink.
“Private sector organizations can be much more nimble and can develop niche capabilities.”
Maria Robson-Morrow
Intelligence is a data business, so it makes sense that many vendors also boast of their artificial intelligence (AI) prowess, and private practitioners, with their looser regulations and more limited means, have a stronger incentive to try AI than their public sector colleagues.
“Private sector organizations can be much more nimble and can develop niche capabilities,” said Robson-Morrow.
“Even 10, 20 years ago, you would have still been confident to say that NSA or GCHQ held capabilities that surpassed private sector capabilities, and that is emphatically no longer the case,” Walton told Freethink, referring to the US and UK’s main signals intelligence agencies.
Strider Technologies, based out of Salt Lake City, Utah, has developed exceptional capabilities by applying AI to the burdensome task of risk profiling, the process of determining who might want to steal company secrets and how they typically do it so that a firm can better protect its most vulnerable assets and staff.
“I know two people, veteran CIA officers with deep expertise both in Russia and in China, who have been given demonstrations by Strider’s team about what they can do, and both have just been blown away: ‘If we had this capability in the CIA, it would be a gamechanger,’” Walton said.
“We’re building, literally, a digital twin of the global economy, down to the individual level: all of the corporate actors, financial flows, trade flows,” Greg Levesque, Strider’s CEO and co-founder, told Freethink. “We can spin up a digital twin of any organization in the entire world, process all of that, and then run all our analytics through their organization to identify risk and opportunities, without the company giving us any data or access to their network.”
“It was taking them 10 months…We built a tool to do it in 10 hours.”
Greg Levesque
To avoid its AI cooking up imaginary threats based on probabilities alone, Strider’s model uses an approach called document validation. It corroborates an AI model’s response using the original source material, which is also served to the analyst for verification.
“Frankly, I don’t really care if so-and-so went for coffee with so-and-so,” Levesque, who worked in public sector intelligence before founding Strider in 2019, told Freethink. “What I do care [about] is did they get cash? Did they sign a contract? I want something tangible that, within a rule of law-based system, which we all live in and which all of our corporate customers live in, they can act on.”
Levesque said his firm’s capabilities are only possible because of the digitization of organizations over the past decade, a process that was accelerated when most were forced into remote work by the COVID-19 pandemic. The services built out of the resulting AI models are valuable enough that Strider now counts seven of the 10 largest US companies by revenue among its clients, according to Levesque.
He says Strider is also helping to vet every applicant to the US Department of Defense’s fund for small business research, which pays out more than $1 billion a year: “It was taking them 10 months to do a vetting of just one cohort in the program. We built a tool to do it in 10 hours.”
“As the AI gets better, the solutions get better.”
Gareth Westwood
In a world awash with data, it is effectively managing, serving, and using such data, rather than gaining access to it, that gives analysts, vendors, and their clients an advantage, according to Chitra Sivanandam, a technology strategist with 25 years’ experience in intelligence innovation, including for In-Q-Tel, the CIA’s venture capital arm.
She told Freethink that, as in other industries, AI can free up analysts’ time and attention for critical work: “The analyst has more freedom to legitimately ignore some parts of the equation that can be more automated and focus on the pieces that could be the showstopping thing that could take a business down.”
Gareth Westwood, head of global intelligence for the risk consultancy Sibylline, said the firm is using AI monitoring to let analysts know that they may need to update their regional risk scores, which take into account factors related to politics, security, criminality, and more.
“We did do it manually for a while,” he said, noting that the task had been a full-time job, “but with AI, we’ve been able to expand our risk matrix and almost constantly monitor these things…As the AI gets better, the solutions get better, and the folks that are using it effectively, with the right data sources, are really winning in the game.”
“Everyone’s looking to become ‘one pane of glass’…and sometimes that can really dilute the quality.”
Gareth Westwood
AI could become a force multiplier, but it has its limits, including patchy sources and poor design. Westwood has already seen third-party open source aggregators fall victim to “technical shocks” when their social or traditional media inputs have shut off data export, reformatted metadata, or put up paywalls. Large language models based on keywords alone can also make embarrassing mistakes, he said, like reporting an urgent incident in the country of Georgia as happening in the US state.
Westwood and others caution against an overdependence on AI, and Sivanandam told Freethink that the current wealth of open sources means a lazy analyst can always find a few pieces of “evidence” to fit an otherwise flimsy hypothesis. Vendors of intelligence technology can also fall victim to mission creep, gradually expanding the scope of what they offer.
“The big thing I’m seeing at the minute with technology is that everyone’s looking to become ‘one pane of glass’ [offering to] do A, B, C, and D, and sometimes that can really dilute the quality,” said Westwood.
For firms that scour cyberspace for human threats, the growing availability of end-to-end encryption also began shrouding vistas even as they opened up through the 2010s.
“A few years ago, there was a lot of open chatter that would involve, for example, threats to executives. Most of that chatter was openly available, and a lot of that has now migrated,” said Robson-Morrow, citing Telegram as a popular refuge.
In some places, like the US, Canada, and Australia, those who carry out investigations need a license, but in many ways, the private intelligence sector remains very lightly regulated despite digitization dramatically increasing its capabilities. Firms and operatives are bound by the same restrictions on personal data and computer use as any other company or citizen, but unlike their public counterparts, there is little in the way of dedicated regulation.
Some are pushing for more. The watchdog group Privacy International, for example, has called on the Information Commissioner’s Office, which serves as the UK’s data protection regulator, to “clarify current obligations [of the private surveillance industry] and areas that currently lack sufficient protections,” including social media monitoring
For the time being, though, ethical boundaries are being defined by the sector itself, through conversations within professional bodies like the Association of International Risk Intelligence Professionals and the Strategic Consortium of Intelligence Professionals, Robson-Morrow told Freethink. She says in-house teams and vendors share concerns and ideas about threats and practices “constantly”—their openness is only limited by a duty to protect clients’ interests, like planned investments in a region—and through these conversations, they set ethical norms and identify bad actors.
“The lack of clarity over regulation internationally might be either exploited, or it’s a hole people are falling through without knowing or meaning to.”
Sam Newbery
Each firm Freethink spoke to has its own red lines.
Levesque told Freethink that Strider will never sell its counterintelligence services to hostile governments or firms based in “adversary nation states,” even if they approach the company through a US subsidiary. To ensure it doesn’t do so inadvertently, he said the company uses its investigative capabilities to look into potential clients before signing contracts with them: “We eat our own dog food. All of the tools we sell, we use in-house.”
For Sibylline, one red line is not using adtech data, which can include internet users’ locations, purchases, and internet browsing history. That data is harvested en masse by seemingly innocuous apps or websites, packaged up, and sold in bulk a country at a time, often for hundreds of thousands of dollars.
“We don’t have a need for it, and I’m sure if we did have a need, we’d follow very strict ethical guidelines because it’s extremely intrusive,” said Westwood.
For clients willing to gamble their reputation and more, though, there are vendors based in countries where the rule of law is weaker and who will cross the line into illegality for a fee.
“I would expect that competition would be making them push boundaries, but I would also say that the international dimension and the lack of clarity over regulation internationally might be either exploited, or it’s a hole people are falling through without knowing or meaning to,” said Sam Newbery, an intelligence historian at the University of Salford in the UK.
There’s also an ongoing debate in the community over whether private operatives can misrepresent themselves to infiltrate groups threatening a client’s interests, said Robson-Morrow. The use of disguises goes back to the earliest days of private intelligence, when plant collector Robert Fortune dressed up as a Qing nobleman to steal the secrets of tea cultivation for the East India Company. Such subterfuge is far easier and less risky in an era of digital anonymity, particularly with the help of generative AI.
“How do you manage information overload?”
Paul Kolbe
For Kolbe, advances made in intelligence since 2013’s In Amenas attack are attributable not just to better intelligence-gathering tools, but also to improvements in how companies use the information.
“We evaluated in the aftermath and added resources, added focus on making sure that when there’s indications of threat, you’ve got a conveyor belt to decision-making on how you act to mitigate that threat and respond to it,” he told Freethink. “You can’t eliminate it, but how do you make sure it doesn’t just be a report somewhere sitting on someone’s desk?”
The December 4 assassination of United Healthcare CEO Brian Thompson demonstrates how something as diffuse as public sentiment can crystalize into disaster. Kolbe said AI tools have “absolutely” empowered security teams, but that the fundamentals of intelligence remain as difficult as ever.
“The power any given analyst has at their fingertips is certainly greater than it was, but that also comes at the cost of how do you manage information overload and how do you best separate the chaff?” he told Freethink. “That’s a never-ending dynamic.”
We’d love to hear from you! If you have a comment about this article or if you have a tip for a future Freethink story, please email us at tips@freethink.com.
