As enterprise AI applications move from validation to production, SMTX AI Platform helps organizations manage compute, deploy models, govern model access, and operate AI services on a unified infrastructure, accelerating the adoption of AI across business scenarios.
GPU inventory by model
| Model | Total | Free |
|---|---|---|
| NVIDIA L20 | 2 | 1 |
| Tesla T4 | 2 | 1 |
Cluster utilization
SMTX AI Platform is an infrastructure platform for enterprise AI inference and model services. It provides unified management of heterogeneous compute resources, diverse model assets, and model access points in on-premises environments, helping enterprises build a secure, efficient, manageable, governable, and observable foundation for AI services.
With a flexible and compatible architecture, the platform supports physical servers, virtual machines, and Kubernetes, as well as AI accelerators from NVIDIA, Ascend, AMD, and Intel. Through model repositories, inference engines, model gateways, resource scheduling, access governance, and observability, SMTX AI Platform reduces the complexity of model deployment and compute management, helping AI applications move from validation to production.
An open architecture brings heterogeneous compute across physical, virtualized, and containerized environments under unified management, abstracting underlying differences and centrally managing all models. Deploy the platform in one day and make models available on Day 1.
Deploy the platform wherever you need it and support diverse workloads while making full use of existing infrastructure and reducing duplicate investment. Centrally managed models can be reused across applications. GPU virtualization and fine-grained partitioning by memory and compute capacity enable more precise and efficient resource utilization.
The built-in model gateway provides unified usage statistics, token quotas, access logs, and access control, making model usage visible, controllable, auditable, and measurable.
Inference services support high availability with multiple replicas, load balancing, and native KV Cache offloading. Backed by production-grade container and storage services, unified monitoring, and logging, SMTX AI Platform delivers lightweight, stable, and observable inference services for continuous business operations.
Bring AI compute from physical servers, virtual machines, and Kubernetes clusters under unified platform management. Abstract environment differences and enable consistent resource scheduling, monitoring, and allocation.
Manage heterogeneous AI accelerators from NVIDIA, Ascend, AMD, and Intel, including GPUs and NPUs. Centrally manage existing and new compute resources, reduce duplicate infrastructure investment, improve reuse, and support domestic and heterogeneous compute requirements.
Dynamically partition NVIDIA GPUs, Ascend NPUs, and other heterogeneous resources, or pass through entire devices according to model size and performance requirements. Use full-device passthrough for large-model inference to maintain stable performance and throughput; share partitioned resources across lightweight OCR, ASR, embedding, and reranking models to balance performance, utilization, and cost.
Centrally manage large language models, multimodal models, embedding and reranking models, and vision models. Flex Engine extends support to MinerU, PaddleOCR, and selected machine-learning models, enabling deployment, publishing, and access to language, vision, document-processing, OCR, and machine-learning assets on one platform.
Manage public and private model repositories in one place, with visibility into model source, connection status, visibility, model count, storage usage, and update time. Model detail pages show key attributes such as parameter count, size, precision, and context length, reducing information-gathering effort before deployment.
Automatically analyze model architecture and calculate KV Cache and GPU memory requirements based on context length, concurrency, and other parameters. Count model weights and KV Cache together in recommended memory capacity to assess resource requirements in advance, reduce deployment failures caused by insufficient resources, and avoid memory waste from excessive reservation.
Use model catalogs and templates to capture inference engines, task types, and recommended parameters. Rapidly deploy, validate, and publish mainstream models while reducing repetitive adaptation, parameter configuration, and manual validation.
Centrally manage mainstream inference engines and multiple versions. Select the right engine version for production workloads and new-model validation, maintaining production stability while improving model adaptation and validation efficiency.
Connect private models, public models, and external model APIs through a unified model gateway. Applications use a consistent interface across models, minimizing changes to endpoints, authentication methods, and integration settings when models change.
Centrally manage model access keys and identify business ownership by Project. Group multiple API keys by project and view workspace, status, usage, rate limits, supported models, and creation time in one interface.
Track token usage and request activity by API key, model, and other dimensions. Understand resource consumption across businesses and models to support capacity planning, cost accounting, and service optimization. Set token quotas for API keys and allocate model-service resources according to business needs.
Set request rate limits, concurrency limits, and accessible model scopes for each API key, applying least-privilege access and reducing misuse and unauthorized calls. Record request source, access key, model, status, token consumption, throughput, latency, and termination reason at request level. Retain request and response details for troubleshooting, performance analysis, and security audits.
View inference-instance health, real-time logs, TTFT, TPOT, end-to-end latency, throughput, queue depth, and KV Cache usage to quickly identify performance bottlenecks and runtime anomalies.
Monitor GPU, NPU, CPU, and other resources in one view. Inspect utilization by node, accelerator, and model instance, then plan, schedule, and optimize resources based on actual workloads.
Use workspaces and RBAC to isolate and authorize key resources, including clusters, model repositories, model catalogs, image repositories, and inference instances. Enable secure collaboration when multiple teams and business units share the AI platform.
SMTX AI Platform is an AI-inference component of the SmartX Enterprise Cloud Platform. It focuses on compute and model management, model governance, and high-performance inference. Together with the container, storage, networking, security, and observability capabilities of the SmartX Enterprise Cloud Platform, it helps enterprises build production-grade, high-performance, and governable model-inference infrastructure.
Provide a standardized runtime for model inference, GPU driver management, and high-performance networking for stable inference and efficient scheduling.
Provide shared, manageable storage for model files, reducing duplicate storage and distribution. High-performance, high-capacity storage also supports KV Cache caching and persistence.
Cover GPU hardware, container status, inference health, model resource consumption, access logs, token usage, and end-to-end latency to support troubleshooting, security audits, performance assessment, and continuous optimization.
The SmartX Enterprise Cloud Platform supports internal applications such as regulatory guidance, translation, and operations. SMTX AI Platform provides unified management and governance for existing and custom-built models.
Based on SMTX AI Platform, the hospital centrally manages compute and models to support diverse AI applications.
The SmartX Enterprise Cloud Platform provides the compute resources for video analytics and other lightweight models, while SMTX AI Platform enables unified management of large language models.
