Frontier AI companies don’t grow their own coffee beans. Like every other company, they make constant decisions about what to do in-house and what to buy from someone else. They already outsource significant AI-related work. Surge AI generated $1.2 billion in revenue in 2024, almost entirely from selling data labelling and RLHF services to most frontier labs. METR, Apollo Research, and the UK AI Security Institute run evaluations that now feature prominently in frontier model system cards.

I think this pattern will and should extend further – to safeguards: technical measures that reduce risks from AI systems, such as content classifiers, jailbreak detectors, monitoring tools, and know-your-customer checks. This would be both in AI companies’ interest and good for the world. 

Concretely, I think: 

• If you’re currently working on safeguards inside a frontier AI company, you should seriously consider whether you’d have more impact offering that service as a third party to the whole industry. 
• Funders should support a thriving safeguards market by providing organisations with startup capital and, where needed, longer-term funding that allows them to prioritise pro-social outcomes over purely commercial incentives.
• Evaluation organisations should consider expanding into safeguard development; they likely have a lot of the necessary skills. 
• Frontier AI companies should work more closely with third-party safeguard developers – sending clear demand signals and working closely enough with them to enable rapid iteration, including sharing data on safeguard performance. ​