On February 1, 2025, Dr. Michael Sharp, a researcher at the National Institute of Standards and Technology (NIST), published the second installment of a four-part Manufacturing Extension Partnership (MEP) blog series titled Beginner’s Guide to Industrial Artificial Intelligence Applications. This blog, hosted on NIST’s Manufacturing Innovation Blog, focuses on the critical role of data in ensuring Industrial Artificial Intelligence (IAI) systems deliver measurable value to manufacturing operations. This article provides a comprehensive analysis of Dr. Sharp’s insights, detailing the data considerations, challenges, and best practices for IAI, while contextualizing the discussion within the broader landscape of smart manufacturing and Industry 4.0.
Background
Dr. Michael Sharp and NIST’s Role
Dr. Sharp leads the Industrial Artificial Intelligence Management and Metrology (IAIMM) Project within NIST’s Communications Technology Laboratory. With a Ph.D. in nuclear engineering from the University of Tennessee (2012) and extensive experience in data modeling, sensing capabilities, and AI, Dr. Sharp is a key figure in advancing IAI for manufacturing. NIST, a non-regulatory agency under the U.S. Department of Commerce, is tasked with promoting innovation through measurement science, standards, and technology. Its work in IAI focuses on developing evaluation guides, metrics, and standards to ensure AI tools are reliable, trustworthy, and practical for industrial applications.
Context of the Blog Series
The four-part MEP blog series aims to demystify IAI for manufacturers, particularly small and medium-sized enterprises (SMEs), by providing accessible guidance on adopting AI technologies. The first blog, published in November 2024, introduced IAI fundamentals and outlined ten basic questions for evaluating AI-driven tools. The second blog, the focus of this analysis, delves into data considerations, emphasizing the importance of high-quality, representative data for successful IAI deployment. Subsequent blogs in the series are expected to cover training, evaluation, and practical implementation of IAI systems.
Key Data Considerations for IAI
Dr. Sharp’s blog underscores that the success of IAI systems hinges on the quality, relevance, and representativeness of the data used during training, testing, development, and deployment. Below are the primary data considerations highlighted in the blog, expanded with technical details and implications.
1. Matching Data to Real-World Conditions
- Concept: IAI systems must be trained and tested on data that accurately reflects the complexities of real-world manufacturing environments. This includes capturing the “faults, flaws, and eccentricities” inherent in industrial processes, such as equipment wear, environmental variations, and operator errors.
- Importance: Data that fails to represent real-world conditions can lead to AI models that perform well in controlled settings but fail in operational environments. For example, an AI system for predictive maintenance trained on idealized data may miss subtle signs of equipment degradation.
- Practical Implications:
- Data Collection: Manufacturers must collect data from actual production lines, including sensor readings, machine logs, and operator inputs, to ensure fidelity.
- Simulation Risks: Over-simplified simulations that do not account for real-world variability can mislead AI models, resulting in poor generalization. Dr. Sharp warns against relying on “cleaned-up” data that removes critical variability.
- Example: In a CNC milling operation, data should include variations in tool wear, material properties, and ambient conditions to train an AI system for tool condition monitoring.
2. Representing the Full Scope of Use Cases
- Concept: Data must encompass the entire range of scenarios the IAI system is expected to handle. This includes edge cases, rare events, and diverse operating conditions.
- Importance: Incomplete data coverage can result in AI systems that are brittle or biased, performing well only in specific situations. For instance, an AI system for quality assurance may fail to detect defects in new product variants if trained on a narrow dataset.
- Practical Implications:
- Data Diversity: Datasets should include examples of both normal and abnormal conditions, such as machine failures or defective parts.
- Data Volume: While large datasets are often beneficial for deep learning, Dr. Sharp notes that quality and relevance are more critical than sheer quantity.
- Example: For an AI-enabled part-tracking system, data should cover different part types, production speeds, and environmental factors to ensure robust performance.
3. Avoiding Common Data Pitfalls
Dr. Sharp identifies several data-related pitfalls that can undermine IAI performance, along with strategies to mitigate them:
- Incomplete Data:
- Issue: Missing data points, such as gaps in sensor readings, can distort AI model predictions. For example, incomplete temperature data may lead to inaccurate thermal stress predictions.
- Mitigation: Implement robust data collection systems and use data imputation techniques cautiously to fill gaps without introducing bias.
- Inadequate Data Variation:
- Issue: Datasets that lack diversity (e.g., only capturing optimal conditions) can result in overfitting, where the AI performs well on training data but poorly on new data.
- Mitigation: Use data augmentation techniques to artificially expand dataset diversity, such as simulating equipment failures or environmental changes.
- Large Gaps in Data:
- Issue: Significant temporal or contextual gaps (e.g., missing data during machine downtime) can prevent the AI from learning critical patterns.
- Mitigation: Ensure continuous data collection and integrate data from multiple sources, such as IoT sensors and operator logs, to provide a comprehensive view.
- Cherry-Picked Examples:
- Issue: Training on idealized or curated data (e.g., only successful production runs) can create unrealistic expectations of AI performance.
- Mitigation: Include “messy” data that reflects real-world variability, such as production anomalies or human errors.
- Excessive Data Cleaning:
- Issue: Over-processing data to remove noise or outliers can strip away valuable information about real-world variability.
- Mitigation: Balance data cleaning with the need to preserve natural variations, using domain expertise to guide preprocessing decisions.
4. Understanding Data Inputs and Assumptions
- Concept: Manufacturers must understand the data, assumptions, rules, and shortcuts that feed into IAI systems at every stage of their lifecycle (training, testing, development, deployment).
- Importance: Transparency into these elements helps set realistic expectations and identify potential biases or limitations. For example, an AI system trained with biased data (e.g., favoring certain machine types) may produce skewed results.
- Practical Implications:
- Data Provenance: Document the sources, collection methods, and preprocessing steps for all data used in IAI systems.
- Bias Detection: Regularly audit datasets for biases or imbalances, such as over-representation of certain operating conditions.
- Stakeholder Education: Train non-expert stakeholders (e.g., shop floor managers) to ask critical questions about data quality and AI behavior.
5. Connecting to Relevant Data Sources
- Concept: IAI systems must be integrated with all relevant data sources, such as sensors, enterprise resource planning (ERP) systems, and human operator logs, to provide a holistic view of the manufacturing process.
- Importance: Missing or disconnected data sources can limit the AI’s ability to make informed decisions. For example, an AI system for supply chain optimization needs data from both production and logistics systems.
- Practical Implications:
- Industrial IoT (IIoT): Leverage IIoT technologies to collect real-time data from connected devices and systems.
- Data Interoperability: Adopt standards like the Quality Information Framework (QIF) to ensure seamless data exchange between disparate systems.
- Example: In a multi-stage manufacturing process, an IAI system for process control should integrate data from machine sensors, quality inspections, and operator feedback.
Technical and Practical Challenges
Implementing these data considerations presents several challenges, particularly for SMEs with limited resources:
- Data Collection Costs: Installing sensors and data storage infrastructure can be expensive and labor-intensive.
- Data Quality: Ensuring high-quality, representative data requires ongoing maintenance and domain expertise.
- Scalability: Managing large volumes of data from IIoT devices and integrating it into IAI systems demands robust computational and networking capabilities.
- Expertise Gap: Many manufacturers lack in-house AI expertise, making it difficult to evaluate data quality or AI performance.
- Data Security: Protecting sensitive manufacturing data from cyber threats is critical, especially in connected environments.
To address these challenges, Dr. Sharp advocates for NIST’s Collaborative Robotic Operations Workcell (CROW) and Digital Twin Lab, which provide manufacturers with access to high-fidelity data streams and testing environments. These resources help SMEs test IAI systems without the need for extensive in-house infrastructure.
Broader Context: IAI in Manufacturing
Alignment with Industry 4.0
IAI is a cornerstone of Industry 4.0, which emphasizes digitalization, automation, and connectivity in manufacturing. Dr. Sharp’s work aligns with global initiatives like Made in China 2025 and the U.S. Manufacturing USA network, which aim to advance smart manufacturing through AI and data-driven technologies. Key IAI applications include:
- Predictive Maintenance: Using AI to predict equipment failures and schedule maintenance proactively.
- Quality Assurance: Detecting defects and ensuring product consistency through AI-powered vision systems.
- Process Optimization: Enhancing efficiency by optimizing production schedules, resource allocation, and energy usage.
- Supply Chain Resilience: Mitigating disruptions by predicting demand, inventory needs, and logistics challenges.
NIST’s Contributions
NIST’s IAIMM project, led by Dr. Sharp, focuses on developing Standard Operating Procedures (SOPs) for IAI use and evaluation, particularly in multi-stage manufacturing. The project emphasizes:
- Data Standards: Promoting interoperability through standards like QIF and Manufacturing Data Exchange Standards.
- Metrology for AI: Creating metrics and testing methods to evaluate IAI performance objectively.
- Trustworthy AI: Ensuring IAI systems are reliable, secure, and transparent, in line with NIST’s AI Risk Management Framework.
NIST’s broader AI efforts, including the AI Standards “Zero Drafts” Pilot Project and the upcoming AI for Resilient Manufacturing Institute, aim to accelerate AI adoption while addressing data quality, security, and ethical considerations.
Recommendations for Manufacturers
Based on Dr. Sharp’s insights, manufacturers can take the following steps to ensure effective IAI deployment:
- Prioritize Data Quality:
- Collect high-fidelity, real-world data that reflects the full range of operating conditions.
- Avoid over-cleaning or cherry-picking data to preserve variability.
- Leverage NIST Resources:
- Use CROW and the Digital Twin Lab to access representative data and test IAI systems.
- Refer to NIST’s evaluation guides and SOPs for IAI implementation.
- Ask Critical Questions:
- Are all relevant data sources connected to the IAI system?
- Does the data accurately represent real-world scenarios?
- Are there biases or gaps in the dataset?
- Invest in Interoperability:
- Adopt IIoT and data exchange standards to ensure seamless integration of data sources.
- Collaborate with Experts:
- Partner with NIST, Manufacturing USA institutes, or academic institutions to bridge expertise gaps.
- Address Security:
- Implement NIST’s Risk Management Framework to protect manufacturing data from cyber threats.
Implications for 2025 and Beyond
Dr. Sharp’s blog highlights the growing importance of data in IAI as manufacturing evolves toward greater automation and connectivity. In 2025, key trends include:
- Increased AI Adoption: The global market for AI in manufacturing is projected to reach $2.057 billion in China alone, driven by applications in predictive maintenance, quality assurance, and supply chain management.
- Focus on Sustainability: AI is increasingly used to optimize energy usage, reduce waste, and support circular economy practices, aligning with global sustainability goals.
- Workforce Development: Training programs, such as those supported by Manufacturing USA, will equip workers with skills to manage AI-driven systems.
- Ethical Considerations: Balancing technological innovation with ethical concerns, such as data privacy and job displacement, will be critical.
NIST’s ongoing efforts, including the AI for Resilient Manufacturing Institute and collaborations with industry and academia, will play a pivotal role in addressing these trends and ensuring IAI delivers tangible benefits to U.S. manufacturers.
Conclusion
Dr. Michael Sharp’s February 2025 blog provides a practical and insightful guide to navigating the data challenges of Industrial Artificial Intelligence. By emphasizing the need for high-quality, representative data and warning against common pitfalls, Dr. Sharp empowers manufacturers to make informed decisions about IAI adoption. His work, supported by NIST’s broader mission, underscores the importance of standards, metrology, and collaboration in building a resilient, AI-driven manufacturing ecosystem. As manufacturers embrace IAI in 2025 and beyond, adhering to these data considerations will be essential for maximizing productivity, efficiency, and competitiveness in the global market.
For further information, refer to NIST’s Manufacturing Innovation Blog, the IAIMM project page, or contact Dr. M. Sharp at NIST’s Smart Connected Systems Division.
