Exploring DeepSeek: A Survey on Advances, Applications, Challenges and Future Directions

I. Introduction

THE rapid advancement of large models has significantly accelerated the creation of intelligent systems. large models are designed to support or substitute human efforts in producing diverse, personalized, and high-quality content more efficiently and cost-effectively, tailored to user needs and requests [1], [2]. large models cover a wide array of synthetic content, such as text [3], images [4], audio [5], video [6], and interactive 3D elements [7]. The progress in GPU technology and the rise in computational capacity have enabled the training of large-scale deep neural networks with vast numbers of parameters. This allows these models to capture more complex information, leading to improved performance across a wide range of downstream tasks. Since 2022, a diverse range of large models has been created and progressively enhanced by prominent technology companies around the globe, such as OpenAI model series [8] and Claude model series [9]. However, their closed-source nature limits transparency regarding model architecture, training data, and optimization strategies. This lack of openness hinders the broader large model community from understanding the key challenges and breakthroughs in model development, leading to high trial-and-error costs and substantial training expenses for other AI companies.

Open-source AI models [6], [10] play a critical role in democratizing AI technology by lowering barriers to entry and accelerating innovation across diverse sectors. They foster collaborative research and development, enabling the global community to build upon and refine the existing models. Additionally, open-source models enhance transparency and reproducibility, which are essential for ensuring ethical AI deployment and addressing societal concerns related to AI-generated content. Despite these benefits, achieving high performance at a lower cost remains a significant challenge for open-source developers, who often have limited resources compared to large technology companies [11].

In the late 2024, DeepSeek introduced two open-source models, DeepSeek-V3 [12] and DeepSeek-R1 [13], which quickly garnered global attention. These models demonstrated performance slightly surpassing that of GPT-4o [14] and GPT-o1 [15], while achieving this at computation costs an order of magnitude lower. This breakthrough not only showcased the potential of cost-efficient AI development but also highlighted the growing competitiveness of open-source alternatives in the landscape of large models. The success of DeepSeek models can be attributed to several key factors, including innovative architectural designs, optimized training pipelines, and strategic use of computational resources. These advancements enable developers to train large-scale models more efficiently, reducing both financial and environmental costs associated with AI development.

Moreover, the open-source nature of DeepSeek family models has fostered a collaborative ecosystem, where researchers and developers worldwide can experiment with, improve, and adapt these models for various applications. This collective effort has accelerated the adoption of DeepSeek models across industries, such as healthcare [35], finance [36], [37], and education [38]. However, the widespread deployment of these models also raises critical issues about safety [39], fairness [40], and ethical use [35], necessitating comprehensive assessments of their impact on society.

This paper aims to present a concise overview of the DeepSeek family of models, exploring their current status, core innovations, practical applications, and safety considerations. By analyzing these aspects, the survey seeks to provide insights into the factors that enable cost-effective AI development, highlight the advantages and limitations of open-source approaches, and identify future research directions. Ultimately, this work aims to support the responsible development and deployment of AI technologies that benefit both industry and society as a whole.

Our main contributions are summarized as follows:

● We provide a comprehensive review of the entire DeepSeek family models, summarizing the core innovations in their development processes, including data processing, training, and infrastructure, and comparing them with traditional counterparts.

● We analyze how the existing DeepSeek models are applied to various downstream tasks, highlighting their practical use cases and performance.

● We examine the potential security, privacy, and ethical concerns associated with current DeepSeek models, considering both technical and ethical implications.

● We outline potential future directions for the development of DeepSeek family models, offering insights to guide further research and applications in the field.

This paper is organized as follows. Section II introduces the overview of DeepSeek. Section III depicts the core innovation of DeepSeek. Section IV demonstrates the application of the DeepSeek family models. We discuss the security, privacy, and ethical issues in Section V. Lastly, we take an insight into Section VI to explain the future development.

II. DeepSeek Overview

This section presents a comparison between DeepSeek and its competitors, offering a high-level overview of the DeepSeek model family, their architectural evolution, structured development process, and summarized technical innovations.

A DeepSeek vs. the Competition

DeepSeek has emerged as a major player in the large model landscape, warranting a detailed investigation into its capabilities and innovations. As shown in Table II (data source from [41]), both DeepSeek-V3 and DeepSeek-R1 utilize the DeepSeekMoE architecture [18], featuring 671B parameters and a 128 K context length. Mixture-of-Experts (MoE) [49]-[51] is a sparse model architecture that activates only a subset of expert networks per input, enabling large-scale models to achieve high performance with greater computational efficiency. Their API token costs are notably competitive, with DeepSeek-V3 priced at $1.1 per million tokens and DeepSeek-R1 at$2.19 per million tokens. These cost structures position them among the most cost-effective options compared to other leading MoE-based proprietary models.

Table  II.  Comparison of the DeepSeek-V3 & DeepSeek-R1 and Other LLM. Bold, underline and $\underline{\underline{\rm{double\ underline}}}$ Mean the Best, Second and Third Performance,respectively. Token Price, Code Index, Math Index and Intelligence Index are provided by [41]

Model Series	Model Parameters	Context Length	Base Model Architecture	Input Token Price	Output Token Price	Coding Index	Math Index	Intelligence Index
DeepSeek-V3	671B	128K	DeepSeekMoE	$0.27	$1.1	36	$\underline{\underline{57}}$	46
DeepSeek-R1	671B	128K	DeepSeekMoE	$0.55	$2.19	49	82	$\underline{\underline{60}}$
Qwen-2.5	72B	1M	MoE	$1.60	$6.40	35	53	40
Mixstral 8x22B	8x22B	65K	MoE	$2.00	$6.00	17	27	26
Falcon	180B	2048	Transformer	/	/	/	/	/
Gemma 2	27B	128K	Transformer	$0.80	$0.80	17	32	38
Llama-3.1	405B	128K	Transformer	$3.50	$3.50	30	46	40
GPT-4o	1.8T	128K	MoE	$2.50	$10.00	32	45	41
GPT-o1	1.8T	200K	MoE	$15.00	$60.00	52	85	62
Claude-3.7	250B+	128K	MoE	$3.00	$15.00	$\underline{\underline{44}}$	54	57
Grok-3	2.7T	128K	MoE	$5.00	$15.00	/	/	66
/: the item is not available; 　　 indicates closed-source models; Token Price represents the official API cost per million tokens (e.g. OpenAI for o1) or the median across providers where a first-party API is not available (e.g. Llama-3.1); Token Price, Code Index, Math Index and Intelligence Index are provided by [41]; Coding Index is to evaluate the coding performance on LiveCodeBench [42] and SciCode [43]; Math Index is to assess the math performance on AIME [44] and MATH-500 [45]. Intelligence Index is a combination metric covering MMLU-Pro [46], GPQA Diamond [47], Humanity’s Last Exam [48], LiveCodeBench [42], SciCode [43], AIME [44], MATH-500 [45];

 | Show Table

DownLoad: CSV

In contrast, models, such as GPT-4o [14], GPT-o1 [15], and Claude-3.7 [52], also adopt MoE architectures but have significantly higher API costs, ranging from $10 to$60 per million tokens. Among them, GPT-o1 is the most expensive, priced at $60 per million output tokens. Qwen-2.5, despite having a smaller parameter size (72B), offers a much longer 1M token context window, albeit at a higher cost of$6.40 per million tokens. Meanwhile, Llama-3.1, which employs a transformer-based architecture with 405 billion parameters, shows relatively lower performance across key evaluation metrics.

From a performance perspective, GPT-o1 achieves the highest Coding Index (52) and Math Index (85), suggesting superior reasoning and problem-solving capabilities. Grok-3 [53] achieves the highest Intelligence Index (66). DeepSeek-R1 demonstrates a strong balance across multiple dimensions, achieving the second-highest Coding Index (49) and Intelligence Index (60), while maintaining a significantly lower token price than its competitors.

These comparisons highlight DeepSeek’s strong positioning within the open-source model ecosystem, where it delivers state-of-the-art performance while maintaining remarkable cost-efficiency and accessibility. Its open availability, combined with leading scores in coding and intelligence, makes it a standout choice for developers and researchers seeking high-performing models without the high costs or restrictions of proprietary systems.

At the same time, DeepSeek also holds its ground among top proprietary models, outperforming or closely matching the capabilities of well-known closed-source systems such as GPT-o1, GPT-4o, and Claude-3.7 often at a fraction of their cost. Given this rare balance of openness, efficiency, and raw capability, it is essential to further explore the methodologies behind DeepSeek’s architecture, its practical applications, and its potential influence on the future of the AI landscape. This survey aims to provide a comprehensive analysis of DeepSeek in that broader context.

B DeepSeek Family Models

As shown in Table I, the DeepSeek family models constitute an extensive suite designed for a range of tasks, including language comprehension, code generation, mathematical reasoning, and multimodal applications. This encompasses a range of series: DeepSeek, which targets language understanding, DeepSeek-Coder, dedicated to coding, DeepSeek-Math/DeepSeek-Prover, designed for mathematical reasoning, DeepSeek-VL, intended for vision understanding, and Janus, which facilitates both vision understanding and visual generation.

Table  I.  Overview of the DeepSeek Model Family and Their Capabilities

Model Series	Model Name	Release Date	Architecture Source	Main Data Source	Model Parameter	Language Understanding	Reasoning		Visual Understanding	Visual Generation
							Math	Coding
DeepSeek	DeepSeek-LLM [16]	Jan.2024	LLama [17]	Diverse Text data	7B/67B	●	◑	◑	○	○
	DeepSeek-V2 [12]	May2024	DeepMoE [18]	Internet Data	16B/236B	●	◑	◑	○	○
	DeepSeek-V3 [19]	Dec.2024	DeepMoE [18]	Math/Coding/Multilingual	671B	●	●	●	○	○
	DeepSeek-R1 [13]	Jan.2025	DeepSeek-V3 [19]	Long COT data/Distillation	671B	●	●	●	○	○
DeepSeek-Math	DeepSeek-Math [20]	Feb.2024	DeepSeek-Coder-V1 [21]	DeepSeekMath Corpus [20]	7B	◑	●	◑	○	○
DeepSeek-Coder	DeepSeek-Coder-V1 [21]	Jan.2024	LLama [17]	Github	1.3B/6.7B/33B	◑	◑	●	○	○
	DeepSeek-Coder-V2 [22]	Jun.2024	DeepMoE [18]	Common Crawl [23]	16B/236B	◑	◑	●	○	○
DeepSeek-Prover	DeepSeek-Prover-V1 [24]	May2024	DeepSeek-Math [20]	Synthetic Lean 4 proof data	7B	◑	●	◑	○	○
	DeepSeek-Prover-V1.5 [25]	Aug.2024	DeepSeek-Prover-V1 [24]	Synthetic Lean 4 proof data	7B	◑	●	◑	○	○
DeepSeek-VL	DeepSeek-VL [26]	Mar.2024	DeepSeek-LLM [16] + hybrid vision encoder [27], [28]	Multi-place Text-image Data	1.5B/7B	◑	○	○	●	○
	DeepSeek-VL2 [29]	Dec.2024	DeepMoE [18] + hybrid vision encoder [27], [28]	Multi-stage Text-image Data	3B/16B/27B	◑	○	○	●	○
Janus	Janus [30]	Oct.2024	LLama + independent vision encoder [31], [32]	Image-text Paired Data	1.5B	◑	○	○	●	●
	JanusFlow [33]	Nov.2024	LLama + independent vision encoder [31], [32]	Image-text Paired Data	1.5B	◑	○	○	●	●
	JanusPro [34]	Jan.2025	LLama + independent vision encoder [31], [32]	Large-scale Data	1.5B/7B	◑	○	○	●	●
○: the item is not considered; ◑: the item ‘is partially considered; ●: the item is considered;

 | Show Table

DownLoad: CSV

These architectures of the DeepSeek family models can be broadly categorized into two main structures. The first is a decoder-only architecture, represented by Llama [32], which remains the foundational structure for models such as Janus [30], [33], [34] and earlier DeepSeek series. The second is the DeepSeekMoE architecture [18], an improved version of Google’s Mixture-of-Experts (MoE) framework, which has been adopted starting from DeepSeek-V2 [12]. We observe that the DeepSeek family models tend to transition towards using DeepSeekMoE in their later models. This shift to DeepSeekMoE results in notable improvements in performance and efficiency, especially for large-scale and complex tasks [18]. For multimodal models, although Janus continues to build upon the LLaMA foundation, DeepSeek-VL2 [29] integrates DeepSeekMoE with specialized visual encoders, enabling advanced cross-modal reasoning and enhancing text-vision integration capabilities.

The data sources of the DeepSeek family models are highly diverse and tailored to each series’ specialized tasks. These include domain-specific corpora [20]-[22], synthetic datasets [13], [24], [25], large-scale vision-language datasets [26], [29], [30], [33], [34], and curated mathematical reasoning datasets [20].

As shown in Fig. 1, the DeepSeek family models showcase a systematic evolution of multimodal and large language models through continuous improvements in data diversity, algorithmic advancements, and architectural innovations.

Figure  1.  The Evolution Overview of the DeepSeek family models. represents the data-related improvment on the training datasets. represents the algorithm/architecture-related improvement on the training processes.

DownLoad: Full-Size Img PowerPoint

C DeepSeek Workflow

As illustrated in Fig. 2, the DeepSeek workflow consists of three primary stages: Data Processing, Training, and Inference.

Figure  2.  The Overview of DeepSeek Workflow. It consists of three stages: data processing, training, and inference.

DownLoad: Full-Size Img PowerPoint

In the Data Processing stage, data from diverse sources—including text, images, and structured datasets—is collected and transformed into a unified format suitable for downstream tasks. Unlike conventional LLMs, DeepSeek-R1 [13] employs advanced data distillation techniques to extract high-quality knowledge from more powerful models (e.g., Qwen [54]). This approach enhances dataset richness and quality, improving the training process by emphasizing the most informative patterns, thereby boosting both efficiency and effectiveness in downstream applications.

To ensure a smooth and logical progression, the output from the data processing stage serves as the input to the training stage, and similarly, the trained models are used in the final inference phase.

The Training stage is divided into three key phases:

● Pre-training. DeepSeek family models undergo large-scale training on a diverse corpus to learn fundamental patterns and representations through next-token prediction tasks.

● Supervised Fine-tuning. The models are further refined using domain-specific data, enabling them to adapt to specialized tasks with improved accuracy and relevance.

● Reinforced Fine-tuning. Reinforcement learning techniques, such as reward-based optimization, help align the model’s outputs with desired behaviors, enhancing overall performance and user satisfaction.

Finally, in the Inference stage, trained DeepSeek models interact with users, generating responses tailored to queries.

D Summarization of DeepSeek Innovation

We summarize the core innovation of DeepSeek family models, and divide them into three parts, model architecture, algorithm, and infrastructure, as illustrated in Fig. 3.

Figure  3.  The Summarization of DeepSeek Innovation.

DownLoad: Full-Size Img PowerPoint

Model Architecture & System Pipeline primarily aims to optimize the structural design and computational efficiency of the DeepSeek family of models. Detailed in Section III-A. The model architecture innovations are summarized as follows.

● Language Model Architecture outlines the design and optimization of decoder-only models, highlighting DeepSeekMoE and Multi-head Latent Attention (MLA).

● Multimodal Model Architecture integrates specialized vision encoders, with DeepSeek-VL [26] using a hybrid encoder for enhanced understanding and Janus [30] employing dual encoders for both comprehension and generation.

The system pipeline innovations are summarized as follows.

● Data Collection Pipeline designs efficient data collection and storage pipelines, ensuring seamless acquisition and processing of multimodal and multilingual data to expand the model’s knowledge coverage.

● COT & Distillation applies Chain-of-Thought (CoT) reasoning to improve the model’s ability to handle complex tasks, while data distillation techniques efficiently extract large model knowledge into smaller models with similar capabilities.

Algorithm focuses on improving model training efficiency, generalization ability, and task adaptability, covering various aspects from data processing to optimization strategies. Detailed in Section III-B.

● Reinforcement Learning leverages reinforcement learning to optimize model behavior, particularly in applications like dialogue generation and code generation, guiding the model to produce more desirable outputs through reward signals.

● Training Loss designs task-specific and data-driven loss functions to enhance model training efficiency and performance.

● Multi-token Prediction extends traditional next-token prediction by requiring the model to predict multiple future tokens simultaneously, enhancing training efficiency and long-term coherence in generated sequences.

● Fill-in-the-middle ensures the construction of high-quality large-scale pretraining datasets with proper data transformation.

Infrastructure primarily aims to ensure hardware effectiveness and efficiency of the DeepSeek family of models. Specifically, it includes Network Co-design, HaiScale, HFReduce, 3FS distribution File System, HAI Platform. Detailed in Section III-C.

● Network Co-design. Large-scale model training typically requires hundreds to thousands of GPUs [55], and the distribution and configuration of these GPUs significantly impact training efficiency and data transmission performance. DeepSeek adopts an innovative network co-design approach that integrates both hardware and software requirements, optimizing communication bandwidth, reducing latency, and improving parallel scalability for large language models.

● HFReduce. In Large-scale model training, allreduce is a critical communication operation used to aggregate gradients across multiple GPUs during distributed training [56]. DeepSeek considered the data communication overhead during training and utilized HFReduce to optimize the communication process.

● HaiScale. HaiScale further optimizes the parallelism method (e.g., Data Paralleism [57], Pipeline Parallelism [58], Tensor Parallelism [59] and Experts Parallelism [60].)

● 3FS Distributed File System. The 3FS Distributed File System mitigates I/O bottlenecks in large-scale AI tasks, leveraging optimized communication and network tuning to minimize congestion in both storage and computation processes.

● HAI Platform. HAI Platform provides task scheduling, fault tolerance, and disaster recovery, improving resource utilization and lowering costs. It serves as an open-box solution tailored for deep learning researchers.

III. DeepSeek Technical Insights

In this section, we delve into the core technical innovations behind the DeepSeek family of models, highlighting advancements in model architecture & system pipeline (Section III-A), training algorithm (Section III-B), and infrastructure (Section III-C).

A Model Architecture & System Pipeline

Language Models Architecture. The architectural improvements of the DeepSeek family models are mainly reflected in two aspects: DeepSeek Mixture of Experts (MoE) and Multi-head latent attention.

DeepSeekMoE [18] builds upon conventional Mixture of Experts (MoE) architectures [49]-[51], which are widely used in transformer-based language models. As illustrated on the left side of Fig. 5, a standard transformer language model consists of L stacked layers of transformer blocks. In MoE-based architectures, the Feed-Forward Network (FFN) within each block is replaced with an MoE module, where multiple experts process different parts of the input. A router dynamically selects a subset of experts for each token during both training and inference, activating only a fraction of the total model parameters in a forward pass. This dynamic routing mechanism enhances parameter efficiency while maintaining computational efficiency.

Figure  5.  Overview of Model Architecture Improvement on Language Models. RoPE stands for Rotary Position Embedding [62].

DownLoad: Full-Size Img PowerPoint

However, conventional MoE architectures face two key challenges. First, individual experts tend to specialize in vastly different types of knowledge, leading to fragmented learning. Second, common knowledge is often redundantly learned across multiple experts, resulting in inefficiencies.

To address these issues, DeepSeekMoE introduces two novel mechanisms: Fine-Grained Expert Segmentation and Shared Expert Isolation. Fine-Grained Expert Segmentation further divides each expert into smaller, more specialized sub-experts, each focusing on a specific domain of knowledge. This segmentation enhances performance on specialized tasks while mitigating knowledge conflicts among experts. Shared Expert Isolation tackles the redundancy problem by introducing dedicated shared experts that focus exclusively on learning and storing common knowledge. This allows domain-specific experts to concentrate on their respective domains without unnecessary duplication of shared information. By integrating these two mechanisms, DeepSeekMoE achieves more efficient expert routing and knowledge organization. As a result, DeepSeekMoE-2B outperforms GShard-3B (a conventional MoE model) while utilizing only 30% of its computational cost [18].

In summary, the motivation for DeepSeekMoE lies in optimizing expert efficiency, reducing redundancy, and achieving better performance with fewer resources. Its architectural refinements reflect a thoughtful response to the known limitations of prior MoE frameworks, making it well-suited for building large-scale, cost-effective, and high-performing language models.

Multi-Head Latent Attention (MLA) introduces an innovative approach to attention mechanisms by leveraging low-rank key-value joint compression, significantly reducing computational and memory overhead. Traditional Multi-Head Attention (MHA) mechanisms [61] operate independently over full-rank key-value representations, requiring the caching of $2n_h d_h \ell$ elements per token during inference, where $n_h$ is the number of attention heads, $d_h$ is the dimension per head, and $\ell$ represents the number of layers. MLA mitigates this bottleneck by compressing key and value representations into a compact latent space, reducing the KV cache to $(d_c + d_h^R)\ell \approx \frac{9}{2}d_h \ell$, where $d_c$ is the KV compression dimension and $d_h^R$ represents the per-head dimension of the decoupled queries and key in MLA.

The core mechanism of MLA (Fig. 5) is low-rank joint compression for keys and values, which reduces the KV cache size. Specifically, the compression process is formulated as:

where $c_t^{KV} \in \mathbb{R}^{d_c}$ represents the compressed latent vector for keys and values, with $d_c (\ll d_h n_h)$ as the KV compression dimension. The down-projection matrix $W^{DKV} \in \mathbb{R}^{d_c \times d}$ reduces dimensionality, while $W^{UK}, W^{UV} \in \mathbb{R}^{d_h n_h \times d_c}$ restore the keys and values. During inference, MLA caches only $c_t^{KV}$, requiring just $d_c \ell$ storage elements across $\ell$ layers. Additionally, since $W^{UK}$ and $W^{UV}$ can be absorbed into $W^Q$ and $W^O$, explicit computation of keys and values becomes unnecessary.

To further save activation memory during training, MLA also compresses queries:

where $c_t^Q \in \mathbb{R}^{d_c'}$ is the compressed query representation with $d_c' (\ll d_h n_h)$ as the query compression dimension. The matrices $W^{DQ} \in \mathbb{R}^{d_c' \times d}$ and $W^{UQ} \in \mathbb{R}^{d_h n_h \times d_c'}$ handle down-projection and up-projection, respectively. Although this query compression does not reduce the KV cache, it helps lower memory consumption during training.

Furthermore, MLA employs a Rotary Position Embedding (RoPE) strategy [62] to preserve rotational invariance, which enhances the model’s ability to generalize across different positions. However, RoPE is inherently position-sensitive for both keys and queries, making it incompatible with low-rank KV compression. To overcome this limitation, a decoupled RoPE strategy is introduced. This approach separates the positional encoding process by utilizing additional multi-head queries and a shared key dedicated to carrying RoPE information. By decoupling RoPE from the compressed KV representations, the strategy allows for more efficient inference while maintaining positional awareness.

Multimodal Model Architecture. As mentioned in Table I, the multimodal models of DeepSeek are divided into two series: DeepSeek-VL [26], [29] and Janus [30], [33], [34]. Their base architecture is built upon the DeepSeek base language models, with different types of vision encoders integrated into the architecture (See Fig. 4). DeepSeek-VL series models are primarily designed for visual understanding tasks, whereas Janus series models support both visual understanding and visual generation.

Figure  4.  Overview of Model Architecture Improvement on Multimodal Models. This figure compares conventional models with the DeepSeek family, highlighting the architectural enhancements in vision encoders and model integration.

DownLoad: Full-Size Img PowerPoint

DeepSeek-VL differs from conventional vision-language models by incorporating a hybrid vision encoder, which includes SAM-B [27] and SigLIP-L [28]. Unlike conventional models that typically rely on a single vision encoder with an adaptor to align vision and language features, DeepSeek-VL enhances feature extraction and representation learning by integrating multiple specialized encoders. This hybrid approach enables better generalization and performance on various vision-language understanding tasks.

On the other hand, DeepSeek Janus expands beyond conventional vision understanding and generation models [63], [64] by introducing distinct vision encoders for different tasks: an understanding vision encoder and a generation vision encoder. This dual-encoder structure, combined with DeepSeek-LLM, allows Janus to excel in both visual comprehension and image generation. In contrast, traditional models often rely on a shared visual encoder for both tasks, which can limit their effectiveness in handling diverse multimodal challenges. In this section, we demonstrate the technical innovation of DeepSeek family models, and comparison with conventional open-source counterparts from three workflow stages, data processing, training/inference, and infrastructure.

The data collection pipeline. As illustrated in Fig. 6, the pipeline highlights key differences compared to conventional domain-specific data collection methods. Unlike traditional approaches, the DeepSeek pipeline employs an iterative process to systematically assemble a large-scale domain corpus from diverse data sources [20]. The process begins with a seed corpus representing the target domain, from which an initial domain classifier is trained, albeit with limited domain diversity. This initial classifier is then used to collect the first subset of domain-specific data. Following the initial iteration of data collection, the pipeline continues by identifying additional domain-relevant data sources to iteratively refine and retrain the domain classifier. Additionally, the DeepSeek pipeline offers significant advantages in domain-specific data collection. It excels in discovering diverse data sources, enabling the system to expand the range of domain-relevant data beyond predefined static repositories. Furthermore, the iterative refinement process gradually enhances data quality by retraining the domain classifier after each collection cycle, ensuring higher relevance and purity of the dataset. These features collectively make DeepSeek highly efficient and versatile for large-scale, high-quality data collection in various domain-specific applications.

Figure  6.  The data collection pipeline of DeepSeek Family Models.

DownLoad: Full-Size Img PowerPoint

Chain-of-Thought (COT) dataset & distillation. Reasoning data plays a critical role in enhancing model utility. To obtain high-quality reasoning data, DeepSeek employs two primary approaches (as illustrated in Fig. 7): (1) the curation of Chain-of-Thought (CoT) data across multiple domains and varying levels of complexity, and (2) the distillation of outputs from other advanced models (e.g., QWEN [54]). The current CoT data curation includes three formats: COT via text reasoning [65], CoT via Coding Reasoning [66], and COT via tool-integrated reasoning [67]. These reasonings are maintained within the CoT data curation process. Additionally, to acquire extended CoT datasets, DeepSeek leverages distillation techniques from high-performing models, such as DeepSeek-V3 and distilled versions of DeepSeek-R1. Notably, it has been observed that distillation significantly accelerates improvements in model performance.

Figure  7.  Overview of Chain-of-Thought (COT) and data distillation.

DownLoad: Full-Size Img PowerPoint

Takeaway 1: DeepSeek achieves enhanced model performance and efficiency through key architectural and optimized data strategies. These include Fine-Grained Expert Segmentation and Shared Expert Isolation in the DeepSeekMoE framework, MLA for reduced computational overhead, and hybrid and dual-encoder designs in multimodal models. Additionally, DeepSeek ensures efficient data expansion and quality enhancement through iterative domain-specific data collection and Chain-of-Thought (CoT) data distillation, enabling superior performance in complex reasoning tasks.

B Training Algorithm

Reinforcement Learning on DeepSeek. Reinforcement learning plays a crucial role in fine-tuning large language/multimodal models, aiming to optimize the quality of generated outputs. Compared to standard supervised fine-tuning (SFT), reinforcement learning exhibits notable advantages, particularly in terms of generalization. Recent research [68] demonstrates that RL-based methods significantly outperform traditional SFT when handling out-of-distribution data. Specifically, as shown in Fig. 1 of [68], RL methods consistently enhance performance on out-of-distribution tasks with increasing training computation, whereas traditional SFT approaches experience performance degradation under similar conditions. This indicates that reinforcement learning not only improves the quality of generated outputs within the training distribution but also substantially strengthens the model’s robustness and generalization capabilities when encountering unseen or diverse scenarios.

Existing reinforcement fine-tuning approaches primarily include Proximal Policy Optimization (PPO) [69] and the recently proposed Group Relative Policy Optimization (GRPO) introduced by DeepSeek [20]. As shown in Fig. 8, PPO [69] has been widely used for this purpose. PPO is a policy gradient method that ensures stable and efficient training by constraining the magnitude of policy updates. In this process, the policy model receives a query and generates a response, which is then evaluated by the reward model to assign a reward signal. The value model estimates the value of the generated response, which is used to compute the advantage function, helping the model assess how much better or worse an action is compared to the average. Additionally, a reference model serves as a fixed baseline to measure the divergence between the current policy and a stable reference, ensuring that policy updates are not excessively large. The core of PPO lies in utilizing Generalized Advantage Estimation (GAE) [70] to reduce variance, coupled with an actor-critic framework that balances policy improvement and value function optimization.

Figure  8.  Overview of Reinforced Fine-tuning. Unlike PPO, GRPO does not require a Value Model to compute advantages.

DownLoad: Full-Size Img PowerPoint

In contrast, DeepSeek employs Group Relative Policy Optimization (GRPO), a variant of PPO designed to leverage comparative feedback from multiple candidate responses generated for the same query. Unlike PPO, which focuses on the absolute quality of a single response, GRPO introduces a group module that ranks multiple responses within a query. The reward model assigns scores to each response, and the policy model is updated based on their relative quality rather than absolute scores. By comparing responses within the same group, GRPO provides a more informative reward signal, enabling the model to learn nuanced preferences and generate outputs that better align with the desired quality. The reference model remains crucial in this process, acting as a baseline to prevent excessive divergence from the original policy. Overall, GRPO enhances the reinforcement learning process by incorporating relative comparisons, leading to more effective fine-tuning of the language model.

One of the key advantages of GRPO over traditional PPO is that it not only reduces computational costs but also maintains/improves competitive performance. In PPO, the value model is a trainable component that needs to be continuously updated during reinforcement fine-tuning. This process can be computationally expensive, as it requires additional forward and backward passes to estimate the value function accurately. Moreover, training the value model can sometimes introduce instability, especially if the value estimates are biased or have high variance. GRPO eliminates the need for a trainable value model during the fine-tuning process. By leveraging relative comparisons among multiple candidate responses within the same group, GRPO directly guides the policy model based on the relative quality of different outputs. This comparative approach eliminates the dependency on value estimation, reducing both the memory space and diminishing the computational overhead associated with training a separate value model. Despite this simplification, GRPO maintains performance levels comparable to PPO because the relative ranking within each group provides a more stable and informative learning signal. This not only accelerates the training process but also enhances efficiency, making GRPO a more scalable and cost-effective solution for large-scale reinforcement fine-tuning of language models like DeepSeek.

To further enhance reinforced training in DeepSeek, the reward model in GRPO supports three primary modes: rule-based rewards, model-based rewards, and tool-based rewards. Each mode serves a distinct purpose in shaping the policy updates and optimizing the generated responses.

1) Rule-Based Rewards: Rule-based rewards are typically employed in the early stages of model development. A notable example is DeepSeek-R1-Zero [71], which incorporates a rule-based reward mechanism to guide initial model alignment. In this approach, predefined heuristics or logical correctness checks determine the reward signals.

2) Model-Based Rewards: Model-based rewards leverage a separate language model to evaluate the quality of generated responses. This method is exemplified by DeepSeek-V3-SFT [12], which employs a fine-tuned model as a reward function. The reward model assesses responses based on linguistic quality, coherence, informativeness, and relevance to the given prompt. By using a supervised fine-tuned variant as the reward function, this approach captures nuanced preferences that go beyond predefined rules, allowing for a more dynamic and adaptive learning signal. Compared to rule-based rewards, model-based rewards provide greater flexibility and scalability, enabling reinforcement fine-tuning to align the policy model with human-like response quality.

3) Model-Based Rewards: Tool-based rewards utilize external tools or verifiers to provide structured feedback. An example of this approach is DeepSeek-Prover-V1.5 [25], which incorporates proof assistant feedback as a reward signal. This is particularly useful in domains such as mathematical reasoning and formal verification, where correctness must be rigorously validated. The proof assistant evaluates the logical soundness of generated proofs and assigns rewards accordingly. By integrating domain-specific validation tools, DeepSeek ensures that its model not only generates fluent and coherent responses but also adheres to strict correctness constraints in specialized tasks.

Training Loss on DeepSeekMoE. In Mixture of Experts, load imbalance among experts can lead to routing collapse [72] and reduced computational efficiency. Traditional methods typically use auxiliary losses [50], [51] to encourage balanced expert usage. These auxiliary losses penalize load imbalance across experts during training, promoting more uniform utilization. Deepseek-V2 [12] also employs similar auxiliary losses for load balance:

where N represents the number of experts, and $f_i$ denotes the fraction of tokens selecting expert i, while $P_i$ is the average activation probability of expert i. For the Device-Level Balance Loss, $|{\cal{E}}|$ is the number of devices, with each device assigned a group of experts. $f_j'$ represents the average fraction of tokens processed by all experts on device j, and $P_j'$ is the sum of their activation probabilities. In the Communication Balance Loss, D denotes the number of devices, where $f_p''$ is the fraction of tokens sent to device p, and $P_p''$ is the total activation probability of all experts assigned to that device.

However, excessively large auxiliary losses may negatively impact model performance by interfering with the primary training objective [73]. This trade-off between load balancing and performance is a key limitation of traditional MoE approaches.

In contrast, DeepSeekMoE from the Deepseek-V3 eliminates the need for auxiliary loss by introducing a bias term $b_i$ for each expert, which adjusts dynamically during training to maintain balanced expert usage. Routing decisions are determined using the modified score $g'_{i,t} = s_{i,t} + b_i$, where $s_{i,t}$ is the original affinity score. The bias term is updated after each training step: it is decreased by $\gamma$ if the expert is overloaded and increased by $\gamma$ if the expert is underutilized. Here, $\gamma$ is a hyperparameter that controls the rate of bias adjustment. Unlike traditional methods, this auxiliary-loss-free strategy ensures load balancing without compromising model performance, as the routing mechanism self-regulates through dynamic bias updates.

To prevent extreme imbalance within a single sequence, DeepSeek-V3 also incorporates a lightweight sequence-wise auxiliary loss defined as ${\cal{L}}_{\rm{Bal}} = \alpha \sum\nolimits_{i=1}^{N_r} f_i P_i$, where $f_i$ represents the expert usage frequency within the sequence, and $P_i$ is the average normalized score of the expert across the sequence. The balance factor $\alpha$ is set to a very small value, ensuring that this auxiliary loss has minimal impact on the model’s optimization. While traditional MoE models rely heavily on auxiliary losses to balance load across batches, DeepSeek-V3 achieves this balance primarily through dynamic bias adjustment, with the sequence-wise auxiliary loss serving only as a complementary measure to prevent localized imbalances. This approach reduces the trade-off between load balancing and performance, resulting in more efficient and effective training.

Multi-token Prediction. Multi-token prediction (MTP) [19] differs from the traditional next-token prediction (NTP) objective by requiring the model to predict multiple future tokens simultaneously, rather than just the immediate next token. Formally, given a sequence of tokens $(x_1, x_2, \dots, x_T)$, the NTP objective aims to maximize the likelihood of the next token $x_{t+1}$ given the context $(x_1, \dots, x_t)$, using a cross-entropy loss defined as:

where CE is the cross entropy loss, T denotes the input sequence length, $x_{t+1}$ is the ground-truth token at position $t+1$, and $p_t^1[x_{t+1}]$ is the probability assigned to the correct token by the next-token prediction module.

In contrast, the MTP objective extends this by predicting a sequence of future tokens $(x_{t+1}, \dots, x_{t+k})$ for a predefined horizon k. The cross-entropy loss for the k-th prediction depth is calculated as:

where $x_i$ denotes the ground-truth token at position i, and $p_i^k[x_i]$ represents the prediction probability of the correct token at depth k. To obtain the overall MTP loss, we compute the average loss across all prediction depths and scale it by a weighting factor $\lambda$:

where D represents the number of prediction depths. The multi-token objective provides two main advantages. First, it densifies the training signal by enabling the model to learn from multiple predictions at each step, thus improving data efficiency. Second, MTP encourages the model to plan its internal representations more strategically, anticipating longer-term dependencies, which can enhance the coherence and accuracy of generated sequences. Consequently, MTP aligns better with real-world generation tasks where maintaining consistency over longer contexts is crucial.

Fill-in-the-Middle. Fill-in-the-Middle (FIM) [74], proposed by OpenAI, is a data transformation technique employed during the pretraining process. In FIM, an input instance is segmented into three parts: prefix, middle, and suffix before tokenization. Each part is then encoded (Enc), with sentinel tokens prepended at the beginning of each section. These sentinel tokens are denoted as $\mathtt{<PRE>}$, $\mathtt{<MID>}$, and $\mathtt{<SUF>}$. The FIM instance can be formulated as follows:

Compared to Fill-in-Blank (FIB) [75], which masks a segment randomly within the input, FIM explicitly conditions the model on both the preceding and succeeding context, thereby enhancing its ability to reason about intermediate content. The FIB approach can be formulated as follows:

where a single sentinel token (e.g., $\mathtt{<MASK>}$) is used to indicate missing content, and the model is tasked with predicting the masked segment based solely on the surrounding context. While FIB enables inpainting, its reliance on a single generic mask results in less structured conditioning compared to FIM.

Starting from DeepSeek-Coder-V1 [21], the DeepSeek family models adopted FIM for pretraining, recognizing its advantages in structured text completion and code generation tasks. By leveraging explicit bidirectional conditioning, FIM improves the model’s ability to reconstruct missing code blocks, generate function bodies from surrounding context, and synthesize coherent structured text. Moreover, its effectiveness in enhancing reasoning and inpainting capabilities makes it a crucial component in modern generative AI applications.

Takeaway 2: DeepSeek advances generative AI capabilities through key innovations in training algorithm. Group Relative Policy Optimization (GRPO) enhances reinforced fine-tuning by leveraging comparative feedback. DeepSeekMoE redesigns a new complementary sequence-wise auxiliary loss, ensuring efficient Mixture of Experts (MoE) balancing. Multi-Token Prediction (MTP) strengthens long-range dependency modeling by enabling simultaneous future token prediction, and Fill-in-the-Middle (FIM) improves structured text completion and code generation by utilizing bidirectional context.

C Infrastructure

Co-designed software and hardware architecture. DeepSeek constructs a Fire-Flyer AI-HPC architecture [76], a synergistic hardware-software system for LLM training. The key technical components are summarized as follows.

1) Network Co-Design: The primary changes in the network topology include the selection of a Two-Layer Fat-Tree topology [77] with integrated storage and computation, as well as a two-zone configuration instead of a three-layer Fat-Tree solution to reduce costs. Each zone consists of an 800-port Fat-Tree connected to approximately 600 GPU compute nodes, with storage servers equipped with dual IB NICs connected to both zones, enabling shared access to storage resources. The two zones are interconnected with limited links, and cross-zone computing tasks are minimized through a scheduling strategy that allows only one pair of nodes to communicate across zones. The Dragonfly topology [78] was not chosen because, despite its cost-effectiveness and performance, it lacked sufficient bisection bandwidth, making it unsuitable for the integrated storage and computation network design required for the cluster of 10 000 A100 GPUs and nearly 200 storage servers.

2) HFReduce: HFReduce, a library specifically designed for efficient allreduce operations, offers several key advantages that enhance communication performance in large-scale deep learning training. Firstly, it reduces PCIe bandwidth consumption by optimizing the data transfer process. Unlike NCCL [79], where each unit of data must go through multiple PCIe bidirectional transmissions, HFReduce requires only one Device-To-Host (D2H) and one Host-To-Device (H2D) data transfer per unit of data, significantly lowering PCIe bandwidth usage and enhancing performance. Secondly, HFReduce minimizes GPU kernel overhead by utilizing the GPU’s Copy Engine (CE) for asynchronous transfers, unlike NCCL, which occupies GPU kernel execution, thereby reducing computational interference. Lastly, HFReduce achieves superior inter-node bandwidth performance, reaching up to 6.3–8.1 GB/s compared to NCCL’s 1.6–4.8 GB/s when performing allreduce on a 186 MiB data size. Additionally, when combined with NVLink, HFReduce further enhances intra-node communication, leveraging NVLink’s 600 GB/s bandwidth to reduce memory-bound issues and achieve inter-node bandwidths exceeding 10 GB/s. These advantages make HFReduce highly efficient and versatile for large-scale allreduce operations, improving both speed and scalability.

3) HaiScale Distributed Data Parallel: The HaiScale framework [76] is a system designed to optimize the training of large-scale models by implementing various parallelism strategies (Similar to DeepSpeed [80] and Megagron [59]). These strategies include data parallelism, model parallelism (encompassing tensor and pipeline parallelism), and hybrid approaches that combine multiple techniques to enhance training efficiency and scalability.

4) 3FS Distributed File System: 3FS is a high-performance distributed file system designed to maximize the IOPS of NVMe SSDs and the throughput of Remote Direct Memory Access (RDMA) networks [81]. Unlike similar tools like WekaFS [82] and DAOS [83], 3FS uses Chain Replication with Apportioned Queries (CRAQ) [84] to enhance SSD utilization and ensure data consistency. Its request-to-send control mechanism prevents client-side congestion, while its Fat-Tree topology provides full-bisection bandwidth, allowing every client to access any storage service without bottlenecks.

5) HAI Platform: The HAI Platform [85] is a time-sharing scheduling system for cluster resource management. It interrupts and resumes tasks based on resource availability and cluster load, ensuring tasks can continue from breakpoints by handling interruption signals, saving checkpoints, notifying the cluster, and resuming from saved states. Unlike systems that pool GPUs, HAI classifies resources by computing nodes, optimizing parallel GPU utilization to achieve up to 99% utilization.

Takeaway 3: DeepSeek enhances performance and scalability through a Two-Layer Fat-Tree network, HFReduce for efficient allreduce, HaiScale for optimized distributed training, 3FS for high-throughput storage with CRAQ, and the HAI Platform for up to 99% GPU utilization. These innovations maximize speed, efficiency, and scalability for large-scale AI training.

IV. DeepSeek Applications

In this section, we delve into the transformative applications of DeepSeek models, showcasing their role in enhancing efficiency, accuracy, and accessibility across diverse fields such as healthcare, software engineering, business, education, scientific research, and personal assistants. A summary of the content is provided in Table III.

Table  III.  Applications of DeepSeek Models

Field	Paper	Functionalities	DeepSeek Models
Medical and Healthcare	[86]	Electronic Health Record analysis	DeepSeek-R1
	[71]	Diagnosing and managing ophthalmology clinical cases	DeepSeek-R1
	[87]	Pediatric Clinical Decision Support	DeepSeek-R1
	[88]	Medical Licensing Examination	DeepSeek-R1
	[89]	Patient education material generation	DeepSeek-R1/V3
	[90]	User acceptance	DeepSeek-R1
	[91]	Drug-drug interaction prediction	DeepSeek-R1
Software Engineering	[92]	Code optimization recommendations	DeepSeek-R1
	[93]	Automatic code generation	DeepSeek-R1/V3/V2.5
	[94]	Automatic code generation	DeepSeek-V3
	[95]	Code generation and repair	DeepSeek-coder-V2
	[96]	Cybersecurity expertise and understanding	DeepSeek-R1
Business and Finance	[37]	Financial reasoning	DeepSeek-R1/V3
	[97]	Stock Market prediction	DeepSeek-R1
	[36]	Accounting	DeepSeek-R1
	[98]	Financial services	DeepSeek-R1/V3
Education	[99]	Science Education	DeepSeek-VL/Prover
	[100]	Improving learning process in higher education	DeepSeek-R1
	[38]	Scoliosis education material generation	DeepSeek-R1/V3
Scientific Research	[24]	Theorem proving	DeepSeek-prover
	[101]	Theorem proving	DeepSeek-prover
	[102]	Theorem proving	DeepSeek-prover
	[103]	Theorem proving	DeepSeek-math
	[104]	Mathematical reasoning	DeepSeek-R1
	[105]	Mathematical reasoning	DeepSeek-R1
	[106]	Social science survey response simulation	DeepSeek-R1
Personal Assistants and Interactive AI	[107]	AI-assisted micro-video creation	DeepSeek-V3
	[108]	Real-time simultaneous human-AI collaboration	DeepSeek-V3

 | Show Table

DownLoad: CSV

A Medical and Healthcare

DeepSeek models have shown significant potential in healthcare [35], offering cost-effective and accessible solutions for clinical decision-making [86], diagnostics [71], [87], patient education [89], [88] and pharmaceutical research [109], [110], [91]. In clinical decision support, DeepSeek-R1 was fine-tuned using the MIMIC-IV dataset to predict prescribed medications from Electronic Health Records (EHR), improving prediction accuracy through NLP techniques and semantic similarity evaluation[86]. For diagnostics, DeepSeek-R1 demonstrated robust performance in ophthalmology [71] and pediatrics [87]. It matched OpenAI o1 with 82% accuracy in ophthalmology cases and achieved 87% accuracy in pediatric diagnostics, slightly below ChatGPT O1’s 92.8%. However, its open-source nature and lower costs make it suitable for resource-limited settings. In patient education, DeepSeek-R1 provided the most readable content among tested LLMs when explaining spinal surgeries, though information quality remained “fair” due to limited citations[89]. User adoption is influenced by ease of use, trust, and perceived usefulness. DeepSeek’s transparency and affordability contribute to increased trust, supporting its integration into healthcare systems [90]. Beyond clinical applications, LLMs, especially DeepSeek-R1 and GPT-4, are also transforming the pharmaceutical industry by streamlining drug discovery and development. Recent studies highlight that LLMs assist in identifying novel drug targets [91], optimizing molecular design [109], and accelerating the preclinical research phase [110], ultimately shortening the pharmaceutical research cycle.

Overall, DeepSeek models offer practical, cost-efficient tools for improving clinical care and patient communication [111].

B Software Engineering

DeepSeek models, particularly DeepSeek-R1 and DeepSeek-Coder, have shown strong capabilities in code optimization [92], generation [93]-[95], and repair [96]. In code optimization, DeepSeek-R1 was integrated into SCALENE, an open-source Python profiler, to analyze performance bottlenecks and suggest optimizations. It delivered competitive recommendations while maintaining SCALENE’s open-source accessibility [92]. For code generation, DeepSeek-Coder, trained with test-case synthesis and reinforcement learning, improved coding benchmarks like HumanEval-plus and MBPP-plus by up to 25% and 6%, respectively, within just 80 optimization steps [93]. This highlights reinforcement learning’s potential in advancing code models. Benchmarking on backend application generation [94], proof-oriented programming [95], and cybersecurity tasks [96] further demonstrated DeepSeek’s competitive performance against models like GPT-4o and OpenAI o1. With its cost efficiency and open-source nature, DeepSeek is driving more accessible and reliable software development tools.

C Business and Finance

DeepSeek models are proving valuable in financial reasoning [36], [37], market prediction [97], and FinTech innovation [98]. In financial analysis, DeepSeek-R1 outperformed models like GPT-4o, achieving the highest score (68.93) on tasks involving financial text, tabular data, and equations[37]. Its success is attributed to reinforcement learning and strong numerical reasoning, particularly in XBRL-Math tasks. For stock market prediction, DeepSeek demonstrated effectiveness in capturing investor sentiment from financial news, although it lagged behind ChatGPT in forecasting future returns and macroeconomic indicators, likely due to differences in language training [97]. In FinTech, DeepSeek’s open-source and cost-efficient approach is lowering barriers for startups, enabling applications in lending, investment management, and insurance [36]. Its accessibility fosters financial inclusion and competition, though challenges remain regarding security, data governance, and regulatory compliance [98], [112]. Overall, DeepSeek’s affordability and performance are reshaping financial services, promoting both innovation and efficiency.

D Education

DeepSeek models have shown significant potential in transforming education by offering personalized learning experiences, supporting advanced research, and enhancing student engagement. In science education, DeepSeek-R1 aids students with complex, visual concepts and simulations, fostering critical thinking and problem-solving skills, while ChatGPT excels at interactive dialogue [99]. In higher education, particularly in computer science, DeepSeek-R1 achieves 88% accuracy in providing feedback, improving grades by 20-25%. Unlike GPT-4, which is better for content generation, DeepSeek enhances comprehension and performance [100]. Its readability, with a Flesch-Kincaid Grade Level of 6.2 and Reading Ease Score of 64.5, surpasses other models, making content more accessible, especially for medical education [38]. Additionally, DeepSeek integrates with learning management systems (LMS) platforms to enable personalized learning, automate assessments, and streamline content creation, boosting both teaching efficiency and student engagement [113]. These capabilities position DeepSeek as a key tool for modern, scalable education.

E Scientific Research

DeepSeek models have significantly contributed to scientific research, particularly in theorem proving and structured data analysis. In formal mathematics, DeepSeek-Prover, trained on 8 million synthetic proof statements, achieved a 52% accuracy on the Lean 4 miniF2F test, surpassing GPT-4 and other baselines [24], [25]. Iterative methods like Goedel-Prover and self-play theorem proving (STP) further improved proof generation, with STP achieving a 61.1% pass rate on miniF2F [101], [102]. Additionally, reinforcement learning from theorem prover feedback (RLTPF) refined proof validation and reasoning accuracy [103]. Dyve, a verification framework integrating fast and slow thinking, further improved reasoning accuracy by guiding LLMs to select better solution paths [105]. However, an analysis of mathematical reasoning capabilities revealed persistent flaws in LLMs’ strategic planning and constraint handling, highlighting areas for further improvement [103]. Beyond mathematics, DeepSeek models have also been used to simulate survey response distributions, providing an efficient alternative to costly human-run surveys and demonstrating the potential of LLMs in social science research [106]. These applications demonstrate DeepSeek’s impact on both formal reasoning and data-driven research methodologies.

F Personal Assistants and Interactive AI

DeepSeek models power diverse personal assistant applications. For media creation, DeepSeek-V3 and DeepSeek-R1 assist in AI-generated micro-videos, achieving human-level popularity through prompt enhancements [107] In human-AI collaboration, DPT-Agent integrates fast intuitive decision-making with reflective reasoning, improving real-time interactions using DeepSeek models [108]. Beyond research, DeepSeek’s official GitHub showcases extensive integrations for real-world applications like DeepChat, a smart assistant for conversations and searches, and Coco AI, a knowledge management tool [114]. These applications highlight DeepSeek’s practicality, affordability, and open-source accessibility as a foundation for AI-driven productivity and interactive experiences.

V. Security, Privacy, and Ethic in Deepseek

This section provides a taxonomy of security and privacy threats, along with existing and potential countermeasures on DeepSeek family models. The security and privacy issues are summarized in Table IV.

Table  IV.  Security, Privacy, and Ethical Issues of DeepSeek Family Models

#	Month Year	Paper	Risk Source	Target Effects	Affected Models	Affected Domain	Threats	Issue Type			Risk Layer
								Security	Privacy		Model	Application
1	Jan. 2025	[115]	Deployment	Unsafe outputs	DeepSeek-R1	NLP	Benchmark	√	√			√
2	Oct. 2024	[116]	Development	Unsafe outputs	DeepSeek-R1	NLP	Benchmark	√			√
3	Feb. 2025	[39]	Development	Unsafe outputs	DeepSeek-V3/R1	NLP	Benchmark	√	√		√
4	Feb. 2025	[117]	Development	Unsafe outputs	DeepSeek-R1	NLP	Benchmark	√	√		√
5	Feb. 2025	[118]	Development	Unsafe outputs	DeepSeek-VL	Multimodality	Benchmark	√	√		√
6	Dec. 2024	[119]	Development	Code Vulnerability	DeepSeek-Coder	Coding	Benchmark	√			√
7	Feb. 2025	[120]	Reasoning	Trustworthiness	DeepSeek-R1	Finance	Supply Chain Attack	√				√
8	Dec. 2024	[121]	Inference	Unsafe outputs	DeepSeek-V2.5	Peer Review	Supply Chain Attack	√				√
9	Jan. 2025	[122]	Deployment	Deceptive tendencies	DeepSeek-R1	AI Agent	Supply Chain Attack	√				√
10	Feb. 2025	[123]	Reasoning	Unsafe outputs	DeepSeek-R1	NLP	Jailbreak	√			√
11	Oct. 2024	[124]	RL on inference	Harmful outputs	DeepSeek-LLM	NLP	Jailbreak	√			√
12	Feb. 2025	[125]	Manipulation	Hallucination	Janus	Multimodality	Jailbreak	√			√
13	Feb. 2025	[126]	Reasoning	Unsafe outputs	DeepSeek-R1	NLP	Jailbreak	√				√
14	Jan. 2025	[127]	Reinforced Training	Harmful outputs	DeepSeek-R1	NLP	Reward Hacking	√			√
15	July 2024	[128]	Training	Unsafe outputs	DeepSeek-Coder	Coding	Backdoor Attack	√			√
16	Sep. 2024	[129]	Training	Code Vulnerability	DeepSeek-Coder	Coding	Backdoor Attack	√				√
17	Apr. 2024	[130]	Training	Code Vulnerability	DeepSeek-Coder	Coding	Backdoor Attack	√			√
18	Feb. 2025	[131]	Training	Unsafe outputs	DeepSeek-R1	NLP	Backdoor Attack	√			√
19	Feb. 2025	[132]	Training	Unsafe outputs	DeepSeek-R1	Peer Review	Backdoor Attack	√	√			√
20	Oct. 2024	[133]	Training	Memorization	DeepSeek-Coder	Coding	Data Extraction Attack		√		√
21	Feb. 2025	[134]	Training	Bias	DeepSeek-R1	NLP	Algorithmic Bias	√				√
22	Feb. 2025	[90]	Inference	Social Behavior	DeepSeek-R1	Healthcare	Targeted Phishing Attack	√				√
√: the item is applicable

 | Show Table

DownLoad: CSV

A Security Threats

Benchmarks. Evaluating the security and safety of DeepSeek models requires well-defined benchmarks that assess their ability to generate accurate, reliable, and ethically appropriate outputs while mitigating harmful, biased, or misleading content. Arrieta et al. [115] applied the ASTRAL safety testing tool [135] to the automated assessment of the outcomes from GPT-o3-mini and DeepSeek-R1, evaluating their performance in handling safety-related concerns. However, content security evaluation varies across regions due to differing regulatory standards.

Several benchmark datasets have been developed to assess these aspects. ChineseSafe [116] and CHiSafetyBench [39] are specifically designed for evaluating content security in Chinese, ensuring alignment with local safety standards. In contrast, Safety-R1 [117] focuses on English-language evaluations, capturing different ethical and safety concerns. For multimodal large language models (MLLMs), MLLMGUARD [118] provides a comprehensive benchmark covering both Chinese and English safety assessments. Additionally, LLMSecCode [119] serves as a specialized benchmark for evaluating the security of code generated by language models, addressing risks in software development.

Performance on these benchmarks also highlights regional biases in training data. For instance, DeepSeek-V2 [12] performs slightly worse on the MMLU Humanity-Moral benchmark [46], as its training data primarily originates from China, whereas the benchmark itself is more aligned with American moral perspectives. These discrepancies underscore the need for diverse and region-specific benchmarks to ensure fair and effective evaluations of content security across different linguistic and cultural contexts.

Security from the Model Inference. The processes of model inference is vulnerable to Supply Chain Attack and Jailbreak.

1) Supply Chain Attack: A Supply Chain Attack occurs when a model, after being integrated into downstream applications, exhibits vulnerabilities due to system interaction or manipulated data. For instance, in financial scenarios [120], DeepSeek-R1 may compromise its trustworthiness to safeguard its own interests. Similarly, in peer review settings [121], an injected system prompt could manipulate the review process, leading to biased or unfair feedback. Furthermore, when DeepSeek-R1 serves as a base model in multi-agent environments [122], its reasoning mechanisms may prioritize self-interest, producing deceptive outcomes that mislead other agents, ultimately causing task failure. The root cause of these vulnerabilities in DeepSeek-R1 is its lack of robustness. Its inability to maintain consistent and reliable behavior across different downstream applications makes it susceptible to manipulation and unintended self-serving actions.

2) Jailbreak: Jailbreaking refers to attacks that circumvent the built-in safety mechanisms of language models, enabling them to produce outputs they were originally designed to avoid, such as harmful or inappropriate content. Recent studies have investigated various jailbreak techniques targeting models like DeepSeek-R1, DeepSeek-LLM, and Janus. One prominent method, H-CoT [123], embeds harmful behavior into the intermediate steps of long chain-of-thought (CoT) reasoning, significantly compromising DeepSeek-R1’s safety mechanisms. This approach achieves an attack success rate close to 100%. Another attack, PathSeeker [124], utilizes a multi-agent reinforcement learning (RL) framework to iteratively uncover input strategies that effectively bypass DeepSeek-R1’s safeguards. Additionally, Islam et al. [125] expose weaknesses in DeepSeek-R1’s multimodal reasoning by systematically optimizing image embeddings, compelling the model to generate hallucinated outputs. Further analysis from SafeChain [126] evaluates jailbreak performance across different benchmarks and reveals that, despite DeepSeek-R1’s strong reasoning capabilities, it struggles to resist sophisticated jailbreak attempts. Collectively, these studies highlight the evolving nature of jailbreak techniques and the ongoing challenges in securing DeepSeek family models against adversarial exploits.

To mitigate these risks from model inference, prompt-based defenses [136]—such as inserting intentional prompts to remind LLMs to resist malicious attempts—and auditing mechanisms [137] in the deployment phase that detect and block unsafe outputs can be employed to prevent model misuse and enhance system robustness.

Security from the Model Training. The processes of model training is vulnerable to reward hacking attack and backdoor attack.

1) Reward Hacking Attack: During RL training, such as GRPO, the process heavily relies on positive or negative signals from reward models to train the policy model. However, in cases of reward hacking [127], the model becomes highly sensitive to these signals, which may lead to outputs exhibiting anti-ethical or harmful tendencies. This vulnerability allows adversarial manipulation of reward signals, potentially causing the model to optimize for unintended behaviors that deviate from human values or ethical constraints.

2) Backdoor Attack: A backdoor attack is a technique where a model is intentionally manipulated during training to produce malicious or unintended behaviors when triggered by specific inputs, while functioning normally otherwise. In DeepSeek family models, backdoor attacks primarily occur in two scenarios: code-related scenarios and long-chain-of-thought (Long CoT) reasoning. In code-related scenarios, Yang et al. [128] found that using cross-entropy is particularly conducive to backdoor attacks due to overfitting. Similarly, Li et al. [129] and Hossen et al. [130] discovered that code vulnerabilities are often unintentionally retained in training datasets, leading to the generation of code vulnerabilities. In the Long CoT scenario, if the trigger is embedded within the chain-of-thought reasoning, it is more likely to pose risks to safety alignment [131], [132].

To address the aforementioned security vulnerabilities from the model training, it is crucial to design robust and adversary-resistant reward models [138] that better reflect nuanced human intent, minimizing exploitable shortcuts during reinforcement learnin. Against backdoor threats, techniques such as dataset sanitization [139], differential privacy [140], and gradient analysis [141] can help detect or prevent the insertion of malicious triggers.

Intellectual Property (IP) Issues. DeepSeek has been banned from use on federal devices in several countries, such as the USA [142] and Australia [143], with the one concern being its IP issues. These issues typically refer to the use of data owned by others without proper authorization. DeepSeek has faced allegations that the data used for its training was utilized without the consent of the rightful owners [144]. This raises significant legal and ethical concerns, particularly regarding the transparency and compliance of AI development processes. Ensuring that training datasets are sourced with proper permissions is essential to maintain trust and mitigate potential legal repercussions, especially as regulations surrounding AI and data usage continue to evolve globally.

To reduce IP risks, DeepSeek should prioritize the use of legally licensed or publicly available datasets and establish clearer data provenance tracking. Incorporating copyright detection tools [145] and obtaining explicit permissions during data collection are critical steps.

B Privacy Threats

Data Privacy on DeepSeek Models. Data privacy on DeepSeek Models is a critical concern, as these models are often trained on extensive datasets that may contain sensitive information. One key issue is the presence of Personally Identifiable Information (PII) within the training data. Despite efforts to anonymize data, achieving complete anonymization is challenging due to the complexity and diversity of data sources. As highlighted in [133], an analysis of the DeepSeek-Coder revealed instances of PII within its training dataset, raising concerns about potential privacy breaches and emphasizing the importance of robust data preprocessing and privacy-preserving techniques.

To reduce the risk of exposing PII in training data, DeepSeek should adopt stricter data filtering protocols [146] and integrate privacy-preserving techniques such as differential privacy [140], and advanced anonymization [147]. Regular audits of training datasets are also essential to ensure that sensitive information is effectively removed before model training.

Data Privacy on DeepSeek Application. Data privacy on DeepSeek applications refers to the concerns surrounding the storage of user prompts and responses on third-party servers during model deployment. This raises significant privacy issues [148], as sensitive or confidential information shared by users may be accessible to external entities. Such concerns have led to regulatory scrutiny, with several governments [143] considering or enforcing bans on the official DeepSeek API to mitigate potential privacy risks and ensure better control over user data. This highlights the importance of implementing secure data handling practices and ensuring compliance with data protection regulations when deploying AI applications.

To address privacy concerns in deployed applications, DeepSeek should implement end-to-end encryption [149], minimize prompt injection possibility [150], and offer on-device or self-hosted deployment options.

C Ethical Threats

The ethical threats associated with DeepSeek primarily stem from biases in human and algorithmic decision-making [134], the unintended influence of DeepSeek family models on societal behaviors [151], and concerns over DeepSeek’s potential role in modern digital slavery [152].

Bias in Human and Algorithmic Decision-Making. Bias in AI is a well-documented issue, and DeepSeek is not exempt from such concerns. It inherits biases from training data [134] and human fine-tuning [127]. Large-scale pretraining reflects societal biases, while reinforcement learning with human feedback (RLHF) [153] can introduce new ones. This interaction between human and algorithmic biases may lead to discrimination, misrepresentation of marginalized groups, and reinforcement of stereotypes.

DeepSeek’s Influence on Human and Societal Behavior. AI does not merely reflect human behavior; it actively shapes it. AI-driven content, from search results to recommendations, influences opinions and decisions [154], [90]. DeepSeek, as a generative AI, can amplify beliefs, manipulate narratives, and alter behavior at scale, raising concerns about misinformation and echo chambers. Unregulated AI deployment may erode critical thinking, underscoring the need for transparency and accountability.

Digital Exploitation and Modern Slavery.

The rapid expansion of large models (e.g., GPT and DeepSeek) presents significant ethical challenges, notably their potential to facilitate digital exploitation and modern slavery [152]. For instance, OpenAI reportedly employed [155] Kenyan workers at wages below $2 per hour for essential data labeling tasks, a practice that exacerbates economic inequality and reinforces exploitative labor conditions [156], [157]. Such practices illustrate how leading AI firms often overlook fundamental human rights, highlighting the urgent need for ethical standards and oversight in the development of advanced technological systems.

To address ethical threats, DeepSeek should implement fairness-aware training techniques, such as bias correction in data sampling [158] and adversarial debiasing during fine-tuning [159], to reduce both algorithmic and human-induced biases. Transparent algorithmic auditing [137] and third-party evaluations [160] can help ensure accountability for the model’s societal influence, while clear labeling of AI-generated content can mitigate risks of misinformation. To combat digital exploitation, DeepSeek and its collaborators must adopt ethical labor standards, including fair wages, safe working conditions, and responsible sourcing of annotation services, ensuring human rights are respected throughout the AI development lifecycle [152].

VI. Future Research Directions

In this section, we will analyze and discuss the potential future development directions of DeepSeek.

A More Extensive Open-Source DeepSeek Family Models

Despite its promise, the current DeepSeek family models still face notable limitations in handling multi-modal tasks. As illustrated in Table I, DeepSeek’s multi-modal capabilities primarily focus on image-text processing (i.e., DeepSeek-VL and Janus). This narrow scope restricts its application in more complex domains such as video-based large generative models [6], [161], interactive 3D visualization [162], [163], and other emerging modalities. In future iterations, more comprehensive DeepSeek family models could extend well beyond image-text modalities. Such a framework would integrate advanced modules for video understanding and generation, as well as tools for 3D environment modeling or reconstruction. By embracing open source practices, the DeepSeek community could collaborate to develop flexible, extensible models that accommodate novel modalities, ultimately driving broader research and industrial applications.

B More DeepSeek’s Downstream Task Applications

To fully realize the capabilities of a powerful model, it must be seamlessly integrated into the entire pipeline of downstream applications. Current AI agent frameworks [164], [165] typically involve planning, decision-making, and action steps based on large-scale language models. However, different models exhibit varying levels of utility and robustness, the way they respond to downstream tasks can differ significantly [166]. While the official DeepSeek project [114] has already incorporated certain applications through the DeepSeek API—primarily for chat or code-related tasks—these represent only a fraction of its potential.

In future iterations, AI agents enhanced by DeepSeek should support more complex interactions, such as embodied AI agents [167] that can perform tasks in the physical world. By broadening the scope of downstream tasks to include real-world, interactive applications, DeepSeek can accelerate advancements in fields ranging from robotics [168] and manufacturing [169] to autonomous systems.

C Advancements in Multi-Lingual and Cross-Cultural Capabilities

While DeepSeek has demonstrated impressive performance in English and Chinese [19], [13], its multi-lingual capabilities remain relatively limited compared to globally oriented models. Future development should emphasize expanding support for additional languages, including low-resource languages, to promote inclusive AI that serves diverse populations worldwide. By broadening its linguistic scope, DeepSeek can bridge communication barriers, empowering individuals and organizations to interact more seamlessly across linguistic boundaries.

Furthermore, enhancing cross-cultural comprehension would enable DeepSeek to better understand cultural nuances and context-specific meanings, improving its performance in tasks involving global communication, media, and content creation. This involves not only expanding language support but also incorporating cultural context into model training, ensuring that the AI responds appropriately across diverse cultural scenarios. Open-sourcing multi-lingual datasets [170] would accelerate research in this area, fostering greater collaboration within the global AI community and promoting ethical, culturally aware AI development.

D Enhanced Security and Trustworthiness

As large-scale models like DeepSeek expand into critical real-world applications, ensuring robust security and privacy measures becomes paramount [171]-[176]. This entails not only developing stringent data protection [177], [178] and access control mechanisms [179]-[182] but also proactively addressing potential vulnerabilities that could be exploited by adversarial actors. Given the increasing sophistication of cyber threats, a multi-layered security approach is essential, incorporating techniques such as differential privacy [183] and homomorphic encryption [184] to minimize risks associated with data breaches and unauthorized access.

Moreover, compliance with evolving regulations—such as the General Data Protection Regulation (GDPR) [185], a comprehensive EU framework that safeguards individual rights over personal data (including access, correction, and deletion), and the California Consumer Privacy Act (CCPA) [186], which ensures consumers’ rights to know, control, and delete their personal data—is essential for the legal and ethical deployment of AI systems, particularly in sensitive domains. DeepSeek has made initial progress by integrating basic data filtering, but it still falls short of full compliance, especially in areas such as fine-grained data traceability and enforceable user data deletion within large-scale models.

Additionally, transparency and explainability further reinforce DeepSeek’s security posture. By integrating interpretable AI techniques, such as SHAP (SHapley Additive exPlanations) [187], developers and stakeholders can better understand the model’s decision-making processes. This is particularly vital in high-stakes environments like finance, healthcare, and legal applications, where opaque decision-making could have serious consequences [188].

Finally, fostering a security-aware development culture is crucial. Implementing best practices such as secure model training [189], red-teaming exercises [190], and routine adversarial testing [191] can help identify vulnerabilities before they are exploited in real-world scenarios. By prioritizing security, privacy, and transparency, DeepSeek can establish itself as a security, trustworthy and resilient AI system in sensitive applications.

E Auditing on Ethics and Human Behaviours.

To ensure responsible AI development and mitigate ethical risks, DeepSeek must implement systematic auditing mechanisms addressing biases [134], societal influences [151], and potential labor exploitation [192. This requires comprehensive bias auditing protocols, including dataset analysis to identify imbalances, diverse and inclusive human feedback processes to reduce subjective biases, and fairness-aware training techniques to mitigate algorithmic discrimination. Additionally, monitoring DeepSeek’s societal impact is essential [154], [90], with user interaction studies assessing behavioral influence, transparency measures improving AI explainability, and algorithmic safeguards preventing the reinforcement of harmful narratives. To prevent AI from enabling digital oppression or modern slavery, DeepSeek should enforce ethical labor practices by ensuring fair compensation for data annotation workers, auditing AI-driven surveillance systems to prevent misuse, and collaborating with policymakers to establish digital rights protections. By embedding these auditing frameworks into its development process, DeepSeek can contribute to a more transparent, equitable, and ethically responsible AI ecosystem.

VII. Conclusion

This survey provides a comprehensive overview of DeepSeek, covering model architecture & system pipeline, training algorithm, and infrastructure. We explored how DeepSeek models leverage cutting-edge techniques, such as Mixture of Experts (MoE), Multi-Head Latent Attention (MLA), and Group Relative Policy Optimization (GRPO), to enhance performance while maintaining computational efficiency. Additionally, we highlight their impact across various domains, including healthcare, finance, education, and software engineering, demonstrating their versatility and cost-effectiveness.

Beyond technical advancements, we analyze security, privacy, and ethical considerations of DeepSeek, underscoring the challenges associated with content safety, data governance, and trustworthiness. Looking ahead, we identify key directions for future development, including expanding DeepSeek’s multimodal capabilities, strengthening multilingual and cross-cultural adaptability, and improving security frameworks for responsible AI deployment. As an open-source initiative, DeepSeek continues to push the boundaries of AI accessibility and innovation, fostering collaboration within the global AI research community.