Pegatron Scales Factory Operations With Visual AI Agents and Digital Twins

Conventional production-line planning relies on factory personnel's experience to arrange stations and tasks, but lacks foresight into actual operational conditions. The overall equipment effectiveness (OEE) of a production line can only be evaluated once operations have started, which is often too late because the capital investment has already been spent. Using NVIDIA Omniverse, Pegatron built PEGAVERSE, a digital twin platform that creates physically accurate simulated environments of factory operations, digitizing data to identify improvement opportunities such as detecting system bottlenecks early.

By merging simulation and real-world intelligence, PEGAVERSE enables rapid operational improvements before a production line is built. Production line plans can first be simulated to estimate cycle times, predict effectiveness, and identify bottlenecks. Based on the results, the simulation can be optimized and adjusted to maximize utilization. Through simulation and evaluation, PEGAVERSE helps to effectively reduce time and money spent on tedious modifications of the physical line.

Pegatron has deployed six virtual factories in parallel with physical production facilities, ensuring every aspect of their spaces is designed, tested, and optimized before real-world implementation. This is expected to  help reduce construction time for a new factory by 40%.

At Pegatron, there are many opportunities to optimize processes with generative AI and AI agents. However, each department was creating its own agents, struggling with compute requirements, MLOp pipelines, and minimal standardization. To unlock higher levels of efficiency, Pegatron built PEGA AI, an AI Factory that allows users to build, train, and deploy a variety of AI agents. Pegatron uses NVIDIA AI Enterprise plus NVIDIA DGX™ to accelerate AI agent development with a rich portfolio of foundation and reasoning models and tools to streamline data processing, model customization, retrieval-augmented generation (RAG), and guardrails. Models include large language models (LLMs) and customized  vision language models (VLMs) based on  NVIDIA VILA architecture. 

PEGA AI has accelerated the development of AI agents across the company by 400% over the last four years. These PEGA AI agents absorb sensor-based data from robots using NVIDIA Isaac Sim and camera infrastructure using NVIDIA Metropolis for video analytics. The agents can be deployed as intelligent customer service agents, quality inspectors, or warehouse operations agents checking on machinery health monitoring, safety compliance, and process optimization.

As a leader in precision product assembly, Pegatron used PEGA AI and the NVIDIA AI Blueprint for video search and summarization (VSS) to build the PEGA Visual Analytics Agent (VAA). A persistent challenge to scaling AI factories is having complex manual procedures, while also experiencing high workforce turnover. For example, monitoring manual assembly process compliance is labor-intensive, error-prone, and hard to scale when resources are scarce.

To solve this, Pegatron deployed an Assembly Guiding Agent that uses VSS to help monitor and refine assembly processes in real time. The agent analyzes the assembly process to spot potential anomalies and confirm that safety standards are met throughout the assembly process. For example, if there was a misstep in assembling a phone or laptop—such as forgetting a screw—Pegatron workers would receive a real-time alert and fix the error on the fly. Factory floor operators can even review a video snippet of the incident and ask questions to the Assembly Guiding Agent for further clarification.

By augmenting the assembly process with this AI agent, Pegatron is seeing a 7% reduction in labor costs per assembly line and a 67% decrease in defect rates. The collaboration with NVIDIA has allowed Pegatron to ensure consistent quality, deliver more efficient process management, and drive innovation and success across their manufacturing operations.

Another example is Kinsus International Technology, a global IC substrate provider and Pegatron custom who was facing challenges in identifying and resolving manufacturing defects. This required time-consuming manual inspections by manufacturing engineers looking at thousands of units to pinpoint issues by correlating machine settings, material temperatures, and other factors. With PEGA AI, Kingsus built a multimodal AI agent that combines image analysis with manufacturing data to automatically analyze and resolve issues. Now, they can accurately identify not just defects, but also their root causes. This AI-driven approach, enhanced by the NVIDIA VLM, has improved analysis accuracy from 76% to nearly 95%, drastically reducing the time needed for defect analysis from days to near zero. As a result, Kinsus has significantly improved its product quality control and accelerated its move toward autonomous manufacturing.

By combining the power of digital twins and AI-driven factories, Pegatron is training AI agents at scale and becoming a leader in industrial AI. For example, they've built an AI agent that helps their glue-dispensing robots dynamically adapt to changing environmental conditions, such as glue viscosity or room temperature. Using advanced simulation and PEGA AI, the team developed an agent to learn the glue-dispensing policies and practice them in the simulated PEGAVERSE. 

The agent is fine-tuned in the PEGAVERSE data flywheel until it can autonomously self-evaluate and optimize the glue-dispensing machine parameters, and dynamically adjust to environmental changes in real time. This ensures consistent quality and efficiency. By simulating various scenarios in the PEGAVERSE and rapidly iterating improvements, the system accelerates development cycles from days to minutes, delivering scalable, intelligent automation for smart factories. PEGAVERSE accelerates sim-to-real policy transfer, evolving from identifying defects or redundancies to applying contextual reasoning and causation analysis. Watch Pegatron's NVIDIA GTC talk, Harmonizing Digital Twins and AI Factory: Unlocking Industrial Autonomy, to see how AI-driven factories can train AI agents at scale to optimize real-world operations and drive automation.