Chinese AI Safety Tests: Only 3.6% Published Results

Chinese AI Safety Tests: Why Only 3.6% of Reviewed Releases Published Results

Introduction

Artificial intelligence is developing rapidly, with new models appearing across coding, research, image generation, reasoning, and business automation. As these systems become more capable, an important question arises: how can users understand whether an AI model has been tested for safety before it is released?

A report from AI research firm SemiAnalysis, covered by Reuters on October 9, 2026, examined 857 model releases from nine major Chinese AI developers between 2021 and September 2026. According to the report, only 31 releases, or approximately 3.6%, had publicly disclosed safety-evaluation results tied to a specific model. Just nine releases, representing approximately 1.1% of the total, had those results published at or before launch.

The findings have drawn attention to the difference between developing an AI model, testing it internally, and making the results of those tests available to the public.

One distinction is essential: the report measures public disclosure, not whether developers actually conducted safety testing. A company may perform internal evaluations without publishing its results.

Nevertheless, the findings raise an important question for the global AI industry: how can customers, researchers, businesses, and policymakers assess AI safety when evidence about testing is difficult to find?

What Did the SemiAnalysis Report Find?

The report examined 857 model releases associated with nine major Chinese AI developers over a period spanning 2021 through September 2026.

According to the findings reported by Reuters, 31 releases had publicly disclosed safety-evaluation results tied to a specific model. That represents approximately 3.6% of the reviewed releases.

Only nine releases had those results published at or before launch. This represents approximately 1.1% of all the releases examined.

The review found no published safety disclosure for 813 releases. This should be understood as an absence of publicly available evidence identified by the review, rather than proof that those models were never evaluated.

The report considered safety disclosures relating to several areas, including harmful outputs, jailbreak resistance, privacy, refusal behaviour, and dangerous capabilities.

These categories matter because AI safety is not a single measurement. A model might perform well on one type of test but still struggle with another. Evaluating different risks helps organisations understand the limitations of a system before deciding how to use it.

The report, as described in the supplied coverage, did not provide comparable figures for US developers. Therefore, the statistics should not be used to claim that Chinese models are inherently less safe than American models or that developers in one country are universally more responsible than those in another.

Why Does Public AI Safety Testing Matter?

AI models can produce incorrect information, respond to malicious instructions, expose sensitive information, or assist with harmful activities if appropriate safeguards are missing or ineffective.

Safety testing is one way to investigate these risks before and after deployment.

However, simply saying that a model has been tested provides limited information. Users also need to understand what was tested, how the evaluation was conducted, and what the results showed.

A useful safety report might describe the test methodology, the categories of risks examined, the limitations of the evaluation, and the model version to which the results apply.

Publishing this information can help independent researchers, enterprise customers, and other developers make more informed decisions.

Without model-specific evidence, external reviewers may struggle to determine whether an evaluation applies to the exact version they intend to use.

Public disclosure is not a guarantee of safety, but it can make claims about safety easier to examine.

What Is an AI Safety Evaluation?

An AI safety evaluation is a structured process used to assess how a model behaves under particular conditions.

Instead of checking only whether the model can answer ordinary questions, evaluators examine how it responds to challenging or potentially harmful situations.

Several areas are especially relevant.

1. Harmful outputs

Evaluators examine whether a model generates responses that could cause harm or provide inappropriate assistance.

For example, a general-purpose assistant should have safeguards against providing certain dangerous instructions. The precise requirements depend on the model, its intended use, and the evaluation framework.

A published report can help users understand which risks were examined and where weaknesses remain.

2. Jailbreak resistance

A jailbreak attempt tries to bypass a model's safety instructions or restrictions.

Evaluators can test whether a model maintains its intended boundaries when users provide manipulative, misleading, or adversarial instructions.

No finite set of tests can cover every possible attack. Developers need to evaluate new attack patterns and continue improving safeguards after launch.

3. Privacy protection

AI systems may process personal information, confidential business documents, or sensitive user requests.

Privacy-related evaluations can examine whether a model reveals protected information or responds inappropriately to requests involving private data.

These evaluations should be complemented by access controls, secure data handling, and appropriate privacy practices at the application level.

4. Refusal behaviour

Models may need to decline requests that fall outside their permitted boundaries.

Testing refusal behaviour helps assess whether the model appropriately refuses disallowed requests while continuing to assist with legitimate questions.

A useful evaluation should consider both sides of the problem: refusing unsafe requests and avoiding unnecessary refusals of harmless ones.

5. Dangerous capabilities

More capable AI systems may require specialised evaluations to understand how their abilities could create additional risks.

Depending on the model and intended use, assessments might examine whether it can perform certain risky tasks or provide assistance that requires additional safeguards.

The purpose is not merely to assign a score. It is to identify potential risks and help developers determine what controls, limitations, and monitoring may be necessary.

Why Might Developers Not Publish Safety Results?

There can be several reasons why an organisation conducts testing without releasing a detailed public report.

Some companies may consider evaluation methods, internal findings, or model-development processes confidential. Others may worry that publishing detailed security findings could reveal weaknesses that attackers could exploit.

Some organisations may publish general safety commitments rather than detailed, model-specific results. Reporting practices may also differ between teams and product releases.

These possibilities do not establish why any particular developer failed to publish a report. The figures alone cannot determine the motives behind each decision.

There is also a difference between publishing an overall safety statement and releasing a report that identifies the specific model version tested, the evaluation methods used, and the observed results.

The latter can provide more useful evidence for independent scrutiny.

A balanced approach to transparency must protect genuinely sensitive information while still providing enough detail for customers and researchers to evaluate safety claims.

Does the Report Prove That Chinese AI Models Are Unsafe?

No. That conclusion would go beyond the reported findings.

The study measured whether model-specific safety-evaluation results were publicly disclosed. It did not establish that every model without a published report was unsafe, untested, or more dangerous than a model from another country.

A model may undergo extensive internal testing without its results being made public. Conversely, a model with a published report may still have important weaknesses.

Safety depends on several factors, including the model's capabilities, the quality of its safeguards, its intended application, the surrounding software, and how it is deployed.

A model used for casual writing assistance may require a different risk assessment from a model connected to sensitive company databases or systems capable of performing consequential actions.

The strongest conclusion is therefore about transparency: according to the report, model-specific public safety evidence was uncommon within the releases examined.

Why This Matters for Indian Businesses and Startups

Indian businesses are increasingly exploring AI for customer support, software development, document processing, research, and workflow automation.

When selecting an AI model, an organisation should consider more than its benchmark scores, price, or speed. It should also assess whether the model is appropriate for the intended task and whether its risks can be managed.

Published safety evaluations can provide useful evidence, although they should never be the only factor in a purchasing or deployment decision.

For example, a startup building an AI customer-support assistant should examine how the selected model handles sensitive information, malicious prompts, incorrect answers, and requests outside its intended scope.

A company deploying AI in a high-impact workflow may need additional testing, access restrictions, human oversight, and monitoring.

Indian developers can also conduct their own evaluations using representative test cases. These tests should reflect the actual users, languages, data, and risks associated with the product.

The goal is not to assume that one country's models are safe and another country's models are not. It is to make model selection more evidence-based.

What Students and AI Developers Can Learn

For students studying computer science, artificial intelligence, or cybersecurity, this report illustrates why model evaluation is an important part of AI engineering.

Building a model is only one stage of development. Engineers must also understand its limitations, test its behaviour, document its weaknesses, and monitor how it performs in real applications.

Students can explore evaluation using a simple chatbot project. They could prepare test questions covering factual accuracy, privacy, harmful requests, and resistance to misleading instructions.

They could then record the expected behaviour, compare the actual responses, and document the limitations of their test set.

Developers can take this further by versioning evaluation datasets, repeating tests after model updates, and tracking regressions.

These practices help turn broad claims about AI quality into measurable questions that can be investigated.

How Can AI Companies Improve Transparency?

The findings point to several practical steps that AI developers can consider.

First, companies can publish model-specific evaluation summaries that identify the version tested and the risks assessed.

Second, they can explain the methodology and limitations of their tests so readers understand what the results do and do not demonstrate.

Third, they can distinguish pre-release evaluations from tests performed after deployment.

Fourth, developers can provide updates when significant model changes affect previous safety findings.

Finally, independent evaluations can complement internal testing by offering an additional perspective.

These practices would not eliminate every AI risk. However, they could help customers compare available evidence and understand the basis of a developer's safety claims.

Conclusion

The SemiAnalysis report covered by Reuters highlights a transparency challenge in AI development. Of 857 Chinese AI model releases examined between 2021 and September 2026, 31 reportedly had publicly disclosed model-specific safety-evaluation results, and only nine had results published at or before launch.

These figures measure published evidence, not the actual percentage of models tested internally. They also do not establish a comparable disclosure rate for US developers or prove that models without published results are unsafe.

The broader lesson is that AI safety requires both careful evaluation and clear communication about what that evaluation demonstrates.

For Indian businesses, students, and developers, the practical takeaway is to look for credible evidence, test models against their intended use cases, and avoid treating marketing claims or a single safety score as proof of reliability.

As AI systems become more capable, transparent evaluation can help users make better-informed decisions about where and how to deploy them.

Sources: Reuters' report on AI safety-test disclosures, based on research by SemiAnalysis, and the related coverage from Investing.com.