Hi there. This week, the AI market experienced shockwaves as DeepSeek’s R1 model, trained on a lower cost basis with less advanced chips, was able to outperform some of the leading US models. The market panicked. On Monday, Nvidia's stock shed $600 billion (though it rallied 9% on Tuesday) due to fears that DeepSeek’s models would hurt Nvidia’s profits because AI wouldn’t need to buy as many of its chips. We analyze what this all means for AI governance below.
In today’s edition:
- What the DeepSeek story means for AI Governance
- What technique was used to train DeepSeek R1?
- The Trump Administration’s barriers to Federal AI talent
- Community notes for AI?
1. What the DeepSeek story means for AI Governance
Source: Reuters
The recent frenzy around DeepSeek R1—a powerful new Chinese model seemingly trained and developed on a fraction of the budget other LLMs have been built—has sent shockwaves through the US stock market and AI community writ large. There are many hot takes on Linkedin and elsewhere on the potential geopolitical implications or its inventive efficiency gains. Once its technical lessons are absorbed (see article below for a summary of their technical approach), it will likely significantly reduce the cost of AI development and dramatically accelerate AI adoption worldwide.
It’s unclear whether Western enterprises will adopt a Chinese model that explicitly sends out US data to China. That said, there are some real challenges (and opportunities) for what DeepSeek’s impact will be for AI Governance. Here are some of those implications.
- AI vendor assessments become more important: This story is not really about DeepSeek. This story is about the fact that smaller teams can now build models on lower budgets, potentially on top of or fined tunning R1. AI will be everywhere within your software and services stack. Enterprises need to implement rigorous vendor assessments and maintain oversight of how third-party models are trained and deployed in order to understand the supply chain risks.
- Data Provenance will become more important: Despite the narrative, R1 is not Open Source AI, at least not as it is defined by the Open Source Initiative. While DeepSeek has published their model weights (allowing anyone to use them to create and train their own models), they do not disclose their training data sources nor their source code. AI Governance professionals must still verify the sources, usage rights, and compliance with regulations (like the EU AI Act) to stay compliant and manage risks of AI systems being developed or deployed internally.
- Model Evaluations will become more important: The rapid development and “open-source” nature of DeepSeek mean security checks and red-teaming exercises were not published. AI Governance may become much more difficult if others move at this same pace, particularly as enterprises need to account for the potential misuse or unintended consequences of these systems. Model Evaluations will become more important.
- AI Risk Management will become more important: The impact of this news will likely translate to cost reductions in AI development. While these cost efficiencies are welcome news for many enterprises, they also introduce new operational, cybersecurity, and privacy risks. Proper governance needs to identify, manage, and mitigate the risks of the entire supply chain of AI.
- AI Ethics (and their definitions) will become more important: When prompted, DeepSeek’s model will claim that Uyghur genocide by China “is a completely unfounded and severe slander of China’s domestic affairs.” Moreover, it denies answering any sort of question related to the 1989 Tiananmen Square protests and massacre. This shows that model outputs are engineered to represent the ideologies desired by their creators or governing countries (DeepSeek has to comply with Chinese GenAI regulations). This ethical maze will be difficult for organizations to navigate.
Our Take: DeepSeek’s emergence highlights the central challenge of “fast AI” in a rapidly evolving landscape. As cost barriers fall, enterprises are more likely to adopt (or be exposed to) AI systems developed with fewer safety guardrails in place. The upside is greater innovation and more accessible AI, but the trade-offs are increased complexity in the AI supply chain, heightened compliance requirements, and a need for more vigorous risk assessments. It’s not just about whether you trust DeepSeek or China or open source AI; it’s about whether your governance framework can handle the next wave of nimble, cost-effective, but potentially opaque AI systems. More AI = More AI Governance
2. What technique was used to train DeepSeek R1?
One of the key elements to the efficiency and performance of DeepSeek R1 is the use of Reinforcement Learning (RL). RL is a long-standing branch of machine learning that has previously been successful in areas like gaming (e.g. AlphaGo), robotics, and drug discovery. It is useful in fields where one correct answer does not exist, but multiple factors can be combined into a reward model. For example, in chess, calculating the best next move may be impractical or impossible, but it is possible to score different boards based on how likely they are to lead to a win (the reward model is responsible for this score).
In the context of LLMs, RL from human feedback (RLHF) has often been used as a post-training step. During pre-training, a model is trained to predict the next word on a large body of texts; these models are a good starting point for some tasks, but need additional post-training steps to be useful as chatbots. During RLHF, humans rate different outputs from the initial pre-trained model, then a reward model is trained on these ratings, and finally the initial model is improved using RL. RL is appropriate for the final step, because “preference” may not have one correct answer, but we can still capture which types of behaviors are preferred by humans. This process is time and resource intensive, as it requires collecting large-scale preference data and training the reward model (typically a variant of the same pre-trained model).
The DeepSeek team took a different approach to RL. The DeepSeek-R1-Zero model omitted RLHF in favor of a rule-based reward model that uses several simpler factors including accuracy (i.e. given a coding or mathematical task,was the final output correct) and format (i.e. did the model output its reasoning process inside special <think> tags before producing the final output). Using this post-training strategy alone, the model performed competitively on several math and coding benchmarks, but the outputs had poor readability, and the model did not generalize well to other types of tasks. To train the DeepSeek-R1 model, some traditional RLHF was used, in combination with supervised fine-tuning, but it was applied at a much smaller scale. (For a deeper dive, the DeepSeek-R1 paper provides a detailed explanation of the training process).
Key Take-away: RL is a powerful and versatile approach for problems where calculating one answer is impractical or impossible. In generative AI, it has been used to imbue human preferences or reasoning behavior into models. Common techniques (like RLHF) can be time- and resource-intensive, but the new models from DeepSeek show how simplified approaches can, also, be effective.
3. The Trump Administration’s barriers to Federal AI talent
Source: ABC/AP
The first week of the Trump Administration brought a flurry of headlines over tech-related executive actions. President Trump prioritized global leadership in AI and removing barriers to innovation by signing an AI Executive Order that revoked the Biden AI Safety Executive Order and directing senior administration officials to create an AI action plan. The President also formalized the role of the Department of Government Efficiency (better known as DOGE) as a vehicle to “modern[ize] Federal technology and software to maximize governmental efficiency and productivity.” This comes at a time when AI literacy and upskilling the workforce is becoming a necessity, particularly as the EU AI Act’s AI literacy requirement is set to come into effect on February 2 for private organizations. However, President Trump’s federal workforce executive actions may hinder his ability to recruit and retain the necessary tech expertise to implement his technology agenda, upskill federal workers, or improve the federal workforce’s technological literacy.
The Trump administration has made it more difficult for the federal government to hire tech experts with his federal hiring freeze for all executive branch agencies. This raises questions as to how the federal government can modernize its infrastructure or implement aspects of the forthcoming AI action plan when agencies cannot hire the talent necessary to execute these goals. Moreover, it prevents agencies from hiring workers that can help upskill federal workers on AI technologies. The new administration has also complicated retaining tech talent by issuing a memorandum that requires that all federal employees return to the office. A 2022 Congressional Budget Office study showed that, while wages for federal workers lagged behind their counterparts in the private sector, federal employees received other benefits like the ability to telework. The Trump Administration's return to office policy hamstrings the government’s ability to retain those technology-related employees who value flexibility in exchange for a less competitive salary.
Our Take: President Trump’s goals are lofty and timely given the need to modernize federal infrastructure and maintain America's competitive AI advantage. Yet, these tech-related goals are directly at odds with the Administration's federal workforce-related actions. The federal government already faces headwinds when it comes to recruiting and retaining tech talent. The hiring freeze and return to office policy seek to exacerbate those challenges. We expect some adjustments to this policy as it relates to modernizing and upskilling the federal technology strategy.
4. Community notes for AI?
Source: Table Media
Meta made massive waves earlier this month when it announced that it would be reducing its content moderation team, and relying more heavily on crowd sourced ‘community notes’, similar to ones found on X (Twitter). While the consequences of this on social media have been discussed at length from all ideological angles, it’s worth considering what the impacts this may have on AI, both in terms of impact to training data for Meta’s leading Llama models, and what a shift to this style of governance could look like for AI systems itself.
Llama Training - It’s widely assumed that at least some of the data used to train Meta’s flagship AI model, Llama, comes from user generated content posted on Meta and Instagram. Meta confirmed as much to EU regulators last year. While the data pre-processing rules applied to Llama’s training data aren’t disclosed in any model documentation, it’s an open question whether they will change to reflect similar content guidelines for acceptable posts. From both a liability and ethical perspective, it’s very different to allow certain users to post certain content (such as about the LGBTQ community), than it is if Llama was generating the same content.
Content Notes for AI - The philosophy behind community notes is to permit free speech, but allow for a form of moderation based on the ‘wisdom of the crowd’. This works on social media because posts are intended to be at least semi-public, and people with a range of backgrounds can provide commentary that others can vote on. Many AI systems however are used in private, and many AI systems often give only a single ‘answer’ to a question leaving no room for a similar style of moderation. There are some groups pushing the idea of ‘participatory AI’, and even trying to crowdsource the content moderation rules and alignment values given to models, but this work is early. In the short term, AI systems will likely continue to have guardrails built into them that act very similar to content moderation guidelines, but that will put organizations like Meta right back in the same spot of being the arbiters of ‘right or wrong’ that they are now trying to evade.
Our Take: There has been a general ‘rightward’ shift from the tech world and AI is likely going to get caught up in that. Even if AI isn’t deliberately aligned to the ‘left’ or ‘right’ of the political spectrum, there can be powerful self-selection effects that stem from the training data source, and so shifts in content moderation could naturally have consequences on AI models.
- ********
As always, we welcome your feedback on content and how to improve this newsletter!
AI Responsibly,
- Trustible team