The Increasing Use of the National Research Platform by the CSU Campuses
•Download as PPTX, PDF•
0 likes•90 views
CIO Council Cal Poly Humboldt September 22, 2023
Report
Share
More Related Content
Similar to The Increasing Use of the National Research Platform by the CSU Campuses
Similar to The Increasing Use of the National Research Platform by the CSU Campuses (20)
More from Larry Smarr
More from Larry Smarr (20)
Recently uploaded
Recently uploaded (20)
The Increasing Use of the National Research Platform by the CSU Campuses
- 1. “The Increasing Use of the National Research Platform by the CSU Campuses” CIO Council Cal Poly Humboldt September 22, 2023 Dr. Larry Smarr Founding Director Emeritus, California Institute for Telecommunications and Information Technology; Distinguished Professor Emeritus, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net
- 2. NSF CC*DNI Grant $7.3M 10/2015-10/2020 Extended 2 Years – Ended 10/2022 (GDC) 2015-2022: The Pacific Research Platform NSF Grants Were Built on CENIC Source: John Hess, CENIC Supercomputer Centers
- 3. https://nationalresearchplatform.org/ 2023 - The National Research Platform Unifies The Previous 8 Years of NSF Grants
- 4. 2023: NRP’s Nautilus is a Multi-Institution Hypercluster Which Creates a “Cyberinfrastructure Commons” ~200 FIONAs on 30 Partner Campuses Networked Together at 10-100Gbps Sept. 14, 2023 Installed CPU Cores Installed GPUs Rotating Storage
- 5. The CENIC-Connected Cyberinfrastructure Commons: Hosted by and Available to CENIC Members-Including the CSU Campuses The CENIC- Connected Subset of the NRP’s Nautilus Graphics by Hunter Hadaway, CENIC; Data by Tom DeFanti, UCSD 9630 CPU Cores, 836 GPUs, 2700 TB Storage and Growing!
- 6. Non-MSI Institutions Minority Serving Institutions EPSCoR Institutions The Users of the CENIC-Connected CI Commons Can Burst into NRP’s Nautilus Hypercluster Outside of California 238 GPUs over CENIC CSUSB + SDSU 88 GPUs over CENIC UCI + UCR + UCM + UCSC + UCSB 511 GPUs over CENIC UCSD + SDSC 21 GPUs over MREN UIC 184 GPUs over GPN U. Nebraska-L 8 GPUs over FLR FAMU 12 GPUs over NYSERNet NYU 21 GPUs over SCLR Clemson U 9 GPUs over GPN U S. Dakota + SD State 4 GPUs via Albuquerque GigaPoP U New Mexico 12 GPUs over NYSERNet U Delaware 2 GPUs over OARnet CWRU 1 GPU over CENIC/PW U Hawaii 1 GPU over CENIC/PW U Guam 144 GPUs over NEREN MGHPCC 8 GPUs over GPN U Oklahoma 16 GPUs over GPN U Missouri
- 7. The Users of the CENIC-Connected CI Commons Can Burst into The Open Science Grid (OSG), Which Has Been Integrated With the NRP In aggregate ~ 200,000 Intel x86 cores used by ~400 projects Source: Frank Würthwein, OSG Exec Director; PRP co-PI; UCSD/SDSC OSG Federates ~100 Clusters Worldwide All OSG User Communities Use HTCondor for Resource Orchestration SDSC U.Chicago FNAL Caltech Distributed OSG Petabyte Storage Caches
- 8. The Users of the CENIC-Connected CI Commons Can Burst into SDSC’s Supercomputing and Data Storage Resources
- 9. 8 of 23 CSU Campuses Have 1 or More Users Who Have Logged Onto Nautilus Total: 80 CSU Nautilus Users
- 10. 5 of 23 CSU Campuses Have Created 1 or More Nautilus Namespaces Total: 51 CSU Nautilus Namespaces Last 12 Months 9/13/2022 to 9/13/2023
- 11. 5 of 23 CSU Campuses Have Used Nautilus GPU and/or CPUs 90,800 GPU-Hours 160,850 CPU Core-Hours Last 12 Months 9/13/2022 to 9/13/2023
- 12. CSU Campuses Have Used Up to 30 Nautilus GPUs Simultaneously Over The Last 6 Months Colors Represent Different Namespaces CSUSB CSUN SDSU
- 13. CSU Campuses Have Used Up to 230 Nautilus CPU Cores Simultaneously Over The Last 6 Months Colors Represent Different Namespaces CSUSB CSUN SDSU
- 14. California State University San Bernardino is an Excellent Example of How to Help Your CSU Faculty and Students To Use NRP www.csusb.edu/academic-technologies-innovation/xreal-lab-and-high-performance-computing/high-performance-computing Their Campus HPC Program Enabled CSUSB Faculty & Students to Use More NRP GPU-Hours In the Last 12 Months Than 8 of the 10 UC Campuses!
- 15. The CSUSB HPC Team Is the Major Reason For CSUSB Having the Largest NRP Utilization of 23 CSUs • Dr. Sam Sudhakar – Chief Financial Officer and Vice President, Finance, Technology, & Operations • Gerard Au – Chief Information Officer • Dr. Bradford Owen – AVP for Faculty Development, Chief Academic Technologies Officer • Dr. Dung Vu – HPC Consultant, Analyst/Programmer • James MacDonell – HPC Consultant, Information Security Analyst • Prof. Youngsu Kim – HPC Faculty Fellow, Asst. Prof. of Mathematics
- 16. A Key Reason CSUSB Has The Largest CSU Nautilus Usage: They Installed and Publicized the JupyterHub “Easy Button” https://csusb-jupyter.nrp-nautilus.io/hub/login Slide Adapted from Prof. Youngsu Kim Over 150 Total Users!
- 17. CSUSB Provides Human and On-Line Support For Faculty, Students, and Staff to Easily Use JupyterHub to Access NRP www.csusb.edu/faculty-center-for-excellence/idat/high-performance-computing/jupyterhub-nrp
- 18. The Rapid Rise of Machine Learning, Artificial Intelligence, and Data Science Research Applications & Student Training on the NRP “Graphics processing units (GPUs), originally developed for accelerating graphics processing, can dramatically speed up computational processes for deep learning. They are an essential part of a modern artificial intelligence infrastructure, and new GPUs have been developed and optimized specifically for deep learning.” www.run.ai/guides/gpu-deep-learning
- 19. Top 15 GPU-Consuming ML/AI Academic Research Nautilus Namespaces: Last Six Months-Peaking at Over 700 GPUs Topics: Robotics, Vision, Self-Driving Cars, 3D Deep Learning, Particle Physics & Medical Data Analysis, VR/AR/Metaverse, Brain Architecture… For More Details on Nautilus Applications, Including ML/AI Namespaces Like the Ones Above See my 4NRP Talk: www.youtube.com/watch?v=1yUz0BwObGs&list=PLbbCsk7MUIGdHZzgZqNbZkV7KGVZ7gn1g&index=19
- 20. CSUN Prof. Bingbing Li Machine Learning Research Projects Utilizing Nautilus • Energy Disaggregation for Manufacturing Plant • Faculty Lead: Dr. Bingbing Li (CSUN), Dr. Richard Donovan (UCI) • DNNs: Long Short-Term Memory (LSTM) RNN, PyTorch • Graph Representation Learning for Material Prediction and Recommendation in CAD Automation • Faculty Lead: Dr. Bingbing Li (CSUN) • Collaborator: Dr. Daniele Grandi @ Autodesk and Dr. Thomas Lu @JPL • DNNs: UV-Net Graph Neural Networks (GNN), PyTorch • Knowledge Graph Construction Through the Potential of Large Language Models within Manufacturing • Faculty Lead: Dr. Bingbing Li (CSUN) • Collaborator: Dr. Jerry Fuh & Senthil Kumar @ National University of Singapore • DNNs: LLMs (ChatGPT & LLaMa), Tensorflow • Multi-Domain AI for Future Manufacturing • Faculty Lead: Dr. Bingbing Li (CSUN) • Collaborator: Dr. Edward Chow & Dr. Thomas Lu @JPL • DNNs: LLMs (ChatGPT & LLaMa), Tensorflow • Medical Image Restoration through Optical & CT Scanning • Faculty Lead: Dr. Xiyi Hang & Dr. Bingbing Li (CSUN) • Collaborator: Dr. Ye Pu and Prof. Demetri Psaltis @ Swiss Federal Institute of Technology Lausanne • DNNs: Large-Kernel CNN, Tensorflow CSUN Prof. Bingbing Li Source: Bingbing Li
- 21. CSUN Prof. Bingbing Li’s Publications Powered by NRP: Demonstrates the Research Leverage of Nautilus Namespace: cesmii-scw Li C., Bian S.J., Wu T.Z., Donovan R.P., and Li B.B.*, “Affordable Artificial Intelligence-Assisted Machine Supervision System for Small and Medium-Sized Manufacturers”, Sensors, 2022, Vol. 22 (16): 6246. Kim Y., Donovan R.P., Ren Y.T., Bian S.J., Wu T.Z., Purawat S., Manzo A.J., Altintas I., Li B.B. , and Li G.P.*, "Smart Connected Worker Edge Platform for Smart Manufacturing: Part 1: Architecture and Platform Design”, Journal of Advanced Manufacturing and Processing, 2022, Vol. 4 (4): e10129. Donovan R.P., Kim Y., Manzo A.J., Ren Y.T., Bian S.J., Wu T.Z., Purawat S., Helvajian H., Wheaton M., Li B.B., and Li G.P.*, "Smart Connected Worker Edge Platform for Smart Manufacturing: Part 2: Implementation and On-site Deployment Case Study”, Journal of Advanced Manufacturing and Processing, 2022, Vol. 4 (4): e10130. Bian S.J., Li C., Fu Y.W., Ren Y.T., Wu T.Z., Li G.P., Li B.B.*, “Machine Learning-based Real-time Monitoring System for Smart Connected Worker to Improve Energy Efficiency”, Journal of Manufacturing Systems, 2021, Vol. 61, pp. 66-76. Bian S.J., Lin T.C., Li C., Fu Y.W., Jiang M.R., Wu T.Z., Hang X.Y., Li B.B.*, “Real-time Object Detection for Smart Connected Worker in 3D printing”, 2021 International Conference on Computational Science (ICCS 2021), Krakow, Poland, June 16-18, 2021. Namespace: nsf-maica Fan H.L., Fuh J.Y., Lu W.F., Kumar A.S., and Li B.B.*, “Unleashing the Potential of Large Language Models for Knowledge Graph Construction: A Practical Experiment on Incremental Sheet Forming”, Proceedings of International Conference on Industry 4.0 and Smart Manufacturing (ISM 2023), Lisbon, Portugal, November 22-24, 2023. Zhang H.J., Jiao Y.C., Yuan Y.Z., Li Y.C., Wang Y.Q., Lu W.F., Fuh J.Y., and Li B.B.*, “Object Detection and Text Recognition for Immersive Augmented Reality Training in Laser Powder Bed Fusion”, International Conference on Industry 4.0 and Smart Manufacturing (ISM 2023), Lisbon, Portugal, November 22-24, 2023. Rose A., Nisce I., Gonzalez A., Davis M., Uribe B., Carranza J., Flores J., Jia X.D., Li B.B., Jiang X.F.*, “A Cloud-based Real-Time Traffic Monitoring System with Lidar-based Vehicle Detection”, 2023 IEEE Green Energy and Smart Systems Conference (IGESSC 2023), Long Beach, CA, November 13-14, 2023. Bian S.J., Grandi D., Liu T.Y., Jayaraman P.K., Willis K., Sadler E.T., Borijin B., Lu T., Otis R., Ho N., and Li B.B.*, “HG-CAD: Hierarchical Graph Learning for Material Prediction and Recommendation in CAD”, Journal of Computing and Information Science in Engineering, 2023, pp. 1-28. Li B.B., Wu T.Z., Bian S.J., Sutherland J.W.*, “Predictive Model for Real-time Energy Disaggregation Using Long Short-term Memory”, CIRP Annals, 2023, Vol. 72 (1): 25-28. Morales-Badajoz A., Elieh N., Diederich A., Sadler E., Glover J., Nizampatnam M., Israel T., Wang A., Zhang L., Besnilian A., George A., Miller J., Jiang X.F., Li B.B.*, “Astro Cultivators: Autonomous Growth System for Space Farming based on Machine Vision and Multi-Sensor Fusion”, ACM Cyber-Physical Systems and Internet of Things Week, San Antonio, TX, May 9-12, 2023. Bian S.J., Grandi D., Hassani K., Sadler E., Borijin B., Fernandes A., Wang A., Lu T., Otis R., Ho N.T., Li B.B.*, “Material Prediction for Design Automation Using Graph Representation Learning”, ASME 2022 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2022), St. Louis, MO, August 14-17, 2022. CSUN Prof. Bingbing Li’s Nautilus Namespace nsf-maica used over 9,000 GPU-Hours in the Last Year UCI Prof. GP Li’s Nautilus Namespace cesmii-scw (mainly used by Bingbing’s research team) used nearly 12,000 GPU-Hours in the Last Year Source: Bingbing Li
- 22. The Congress Has Asked NSF to Develop a National AI Research Resource A widely accessible AI research cyberinfrastructure that brings together computational resources, data, testbeds, algorithms, software, services, networks, and expertise, as described in this report, would help to democratize the AI R&D landscape in the United States for the benefit of all. In order to achieve its vision and goals, the Task Force estimates the budget for the NAIRR as $2.6 billion over an initial six-year period. The NAIRR must provide a platform that can be used for educational activities in order to lower the barriers to participation in the AI research ecosystem.
- 23. UCSD’s Information Technology Services Has Adapted PRP FIONA8s To Support Students in Data Science (DS) & Machine Learning (ML) Courses • Student-Focused Platform For: – Undergraduate & Graduate Coursework – For-Credit Independent Study – Thesis/Dissertation Research – Capstones & Projects • Research-Driven Architecture • Managed by UCSD IT Services Software Used By Students Source: Adam Tilghman, UCSD ITS SDSC Racked FIONAs: 139 32-bit GPUs (10% NRP) 2048 CPU-cores 10/25/40G Networking Not Federated With NRP
- 24. UCSD’s DS/ML Platform Supported Nearly 60 Courses Spring Quarter 2023 Source: Adam Tilghman, UCSD ITS
- 25. UCSD’s DS/ML Platform Supported Over 6,000 Students Spring Quarter 2023 Source: Adam Tilghman, UCSD ITS
- 26. Selected Courses, Spring 2023 • Advanced Computer Vision • Bioinformatics for Immunologists • Computational Physics: Probabilistic Models/Sim. • Data Analysis/Design for Biologists • Data Science/Spatial Analysis • Deep Learning and Applications • Intro to Causal Inference • Neural Networks/Pattern Recognition • Numerical Analysis for Multiscale Biology • Robot Manipulation and Control
- 27. ● Dell PowerEdge Cluster for Instructional Use, with 15 Nodes with 32 A100 GPUs (managed as 232 independent GPUs), 768 CPU cores, and 960 TB Storage. ● A Learning Resource that Surges for Research and Instruction ● System Administration as a Service from the NRP ● Friendly User mode this Spring with ● ~90 Users in Computational Engineering 361 & Astronomical Techniques ASTR 350 ● Research in Developing Synthetic Medical Data for Machine Learning Training SDSU’s VERNE Expands the UCSD Instructional Model by Federating with NRP: Visionary, Education, Research, Network, Ecosystem Slide courtesy of Jerry Sheehan, SDSU CIO
- 28. But Who Will Train the Trainers? NSF is Providing CyberTraining Funding NSF award #2230127 SDSC at UCSD, SDSU & CSUSB have been awarded $6.7 million for training and mentoring a cohort of CI professionals. Our training program cohort will be drawn from populations including domain scientists, interdisciplinary research professionals, faculty, students and other NSF workforce individuals and underrepresented STEM communities.” --PI Mary Thomas, SDSC
- 29. Two Recently Submitted NSF Proposals With CSUs as NRP Drivers PI Tom DeFanti, UCSD PI Jerry Sheehan, SDSU
- 30. NRP Support and Community: • US National Science Foundation (NSF) awards and subawards to UCSD --CNS-1456638, CNS-1730158, CNS-2100237, CNS-2120019 --ACI-1540112, ACI-1541349, OAC-1826967, OAC-2029306, OAC-2112167 • DOD DURIP awards to UCSD • UC Office of the President, Calit2 and Calit2’s UCSD Qualcomm Institute • San Diego Supercomputer Center and UCSD’s Research IT and Instructional IT • CENIC, Pacific Wave/PNWGP, StarLight/MREN, The Quilt, Great Plains Network, NYSERNet, Open Science Grid, Internet2, DOE ESnet, NCAR/UCAR & Wyoming Supercomputing Center, AWS, Google, Microsoft, Cisco, Juniper, Arista