The Pacific Research Platform:a Science-Driven Big-Data Freeway System
•Download as PPTX, PDF•
1 like•875 views
The Pacific Research Platform will create a regional "Big Data Freeway System" along the West Coast to support science. It will connect major research institutions with high-speed optical networks, allowing them to share vast amounts of data and computational resources. This will enable new forms of collaborative, data-intensive research for fields like particle physics, astronomy, biomedicine, and earth sciences. The first phase aims to establish a basic networked infrastructure, with later phases advancing capabilities to 100Gbps and beyond with security and distributed technologies.
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (7)
Similar to The Pacific Research Platform:a Science-Driven Big-Data Freeway System
Similar to The Pacific Research Platform:a Science-Driven Big-Data Freeway System (20)
More from Larry Smarr
More from Larry Smarr (20)
Recently uploaded
Recently uploaded (20)
The Pacific Research Platform:a Science-Driven Big-Data Freeway System
- 1. “The Pacific Research Platform: a Science-Driven Big-Data Freeway System.” Invited Presentation 2015 Campus Cyberinfrastructure PI Workshop Austin, TX September 30, 2015 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net 1
- 2. Vision: Creating a West Coast “Big Data Freeway” Connected by CENIC/Pacific Wave to Internet2 & GLIF Use Lightpaths to Connect All Data Generators and Consumers, Creating a “Big Data” Freeway Integrated With High Performance Global Networks “The Bisection Bandwidth of a Cluster Interconnect, but Deployed on a 20-Campus Scale.” This Vision Has Been Building for 25 Years
- 3. The Pacific Research Platform Creates a Regional End-to-End Science-Driven “Big Data Freeway System” NSF CC*DNI $5M 10/2015-10/2020 PI: Larry Smarr, UC San Diego Calit2 Co-Pis: • Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2, • Philip Papadopoulos, UC San Diego SDSC, • Frank Wuerthwein, UC San Diego Physics and SDSC NSF-Funded Workshop For PRP Members October 14-16 Calit2@UCSD
- 4. NCSA Telnet--“Hide the Cray” Paradigm That We Still Use Today • NCSA Telnet -- Interactive Access – From Macintosh or PC Computer – To Telnet Hosts on TCP/IP Networks • Allows for Simultaneous Connections – To Numerous Computers on The Net – Standard File Transfer Server (FTP) – Lets You Transfer Files to and from Remote Machines and Other Users John Kogut Simulating Quantum Chromodynamics He Uses a Mac—The Mac Uses the Cray Source: Larry Smarr 1985 Data Generator Data Portal Data Transmission
- 5. Interactive Supercomputing End-to-End Prototype: Using Analog Communications to Prototype the Fiber Optic Future “We’re using satellite technology… to demo what It might be like to have high-speed fiber-optic links between advanced computers in two different geographic locations.” ― Al Gore, Senator Chair, US Senate Subcommittee on Science, Technology and Space Illinois Boston SIGGRAPH 1989 “What we really have to do is eliminate distance between individuals who want to interact with other people and with other computers.” ― Larry Smarr, Director, NCSA
- 6. NSF’s PACI Program was Built on the vBNS to Prototype America’s 21st Century Information Infrastructure The PACI Grid Testbed National Computational Science 1997
- 7. Chesapeake Bay Simulation End-to-End Collaboratory: vBNS Linked CAVE, ImmersaDesk, Power Wall, and Workstation Alliance Project: Collaborative Video Production via Tele-Immersion and Virtual Director UIC Donna Cox, Robert Patterson, Stuart Levy, NCSA Virtual Director Team Glenn Wheless, Old Dominion Univ. Alliance Application Technologies Environmental Hydrology Team 4 MPixel PowerWall Alliance 1997
- 8. Two New Calit2 Buildings Provide New Laboratories for “Living in the Future” • “Convergence” Laboratory Facilities – Nanotech, BioMEMS, Chips, Radio, Photonics – Virtual Reality, Digital Cinema, HDTV, Gaming • Over 1000 Researchers in Two Buildings – Linked via Dedicated Optical Networks UC Irvine www.calit2.net Preparing for a World in Which Distance is Eliminated…
- 9. Linking the Calit2 Auditoriums at UCSD and UCI With HD Streams September 8, 2009 Photo by Erik Jepsen, UC San Diego Sept. 8, 2009
- 10. NSF’s OptIPuter Project: Using Supernetworks to Meet the Needs of Data-Intensive Researchers OptIPortal– Termination Device for the OptIPuter Global Backplane Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent 2003-2009 $13,500,000 In August 2003, Jason Leigh and his students used RBUDP to blast data from NCSA to SDSC over the TeraGrid DTFnet, achieving18Gbps file transfer out of the available 20Gbps
- 11. Integrated “OptIPlatform” Cyberinfrastructure System: A 10Gbps Lightpath Cloud National LambdaRail Campus Optical Switch Data Repositories & Clusters HPC HD/4k Video Images HD/4k Video Cams End User OptIPortal 10G Lightpath HD/4k Telepresence Instruments LS 2009 Slide
- 12. So Why Don’t We Have a National Big Data Cyberinfrastructure? “Research is being stalled by ‘information overload,’ Mr. Bement said, because data from digital instruments are piling up far faster than researchers can study. In particular, he said, campus networks need to be improved. High-speed data lines crossing the nation are the equivalent of six-lane superhighways, he said. But networks at colleges and universities are not so capable. “Those massive conduits are reduced to two-lane roads at most college and university campuses,” he said. Improving cyberinfrastructure, he said, “will transform the capabilities of campus-based scientists.” -- Arden Bement, the director of the National Science Foundation May 2005
- 13. Based on Community Input and on ESnet’s Science DMZ Concept, NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways Red 2012 CC-NIE Awardees Yellow 2013 CC-NIE Awardees Green 2014 CC*IIE Awardees Blue 2015 CC*DNI Awardees Purple Multiple Time Awardees Source: NSF See Esnet’s Eli Dart Talk on Future of Science DMZs
- 14. Creating a “Big Data” Freeway on Campus: NSF Funded Prism@UCSD and CHeruB Prism@UCSD, Phil Papadopoulos, SDSC, Calit2, PI CHERuB, Mike Norman, SDSC PI CHERuB
- 15. FIONA – Flash I/O Network Appliance: Linux PCs Optimized for Big Data Cost $7700 $21,000 Intel Xeon Haswell Multicore E5-1650 v3 6-Core 2x E5-2697 v3 14-Core RAM 1 TB 16 TB SSD 4 TB 16 TB Network Interface 10GbE/40GbE 100GbE GPU NVIDIA Tesla K80 24GB RAID Drives 0 to 112TB (add ~$100/TB) UCOP Rack-Mount Build: FIONAs Are Science DMZ Data Transfer Nodes & Optical Network Termination Devices UCSD CC-NIE Prism Award & UCOP Phil Papadopoulos & Tom DeFanti Joe Keefe & John Graham
- 16. Customizing Prism@UCSD to Specific Big Data Requirements for Rob Knight’s Lab – PRP Does This on a Sub-National Scale FIONA 12 Cores/GPU 128 GB RAM 3.5 TB SSD 48TB Disk 10Gbps NIC Knight Lab 10Gbps Gordon Prism@UCSD Data Oasis 7.5PB, 100GB/s Knight 1024 Cluster In SDSC Co-Lo CHERuB 100Gbps Emperor & Other Vis Tools 64Mpixel Data Analysis Wall 120Gbps 40Gbps
- 17. CC*DNI FIONA DTN Existing DTNs As of October 2015 FIONA DTNs FIONAs as Uniform DTN End Points
- 18. Ten Week Sprint to Demonstrate the West Coast Big Data Freeway System: PRPv0 Presented at CENIC 2015 March 9, 2015 FIONA DTNs Now Deployed to All UC Campuses And Most PRP Sites
- 19. Digital Research Platform: Distributed IPython/Jupyter Notebooks: Cross-Platform, Browser-Based Application Interleaves Code, Text, & Images IJulia IHaskell IFSharp IRuby IGo IScala IMathics Ialdor LuaJIT/Torch Lua Kernel IRKernel (for the R language) IErlang IOCaml IForth IPerl IPerl6 Ioctave Calico Project • kernels implemented in Mono, including Java, IronPython, Boo, Logo, BASIC, and many others IScilab IMatlab ICSharp Bash Clojure Kernel Hy Kernel Redis Kernel jove, a kernel for io.js IJavascript Calysto Scheme Calysto Processing idl_kernel Mochi Kernel Lua (used in Splash) Spark Kernel Skulpt Python Kernel MetaKernel Bash MetaKernel Python Brython Kernel IVisual VPython Kernel Source: John Graham, QI
- 20. PRP Has Deployed Powerful FIONA Servers at UCSD and UC Berkeley to Create a UC-Jupyter Hub Backplane FIONAs Have GPUs and Can Spawn Jobs to SDSC’s Comet Using inCommon CILogon Authenticator Module for Jupyter. Deep Learning Libraries Have Been Installed Source: John Graham, QI
- 21. PRP Timeline • PRPv1 – A Layer 2 and Layer 3 System – Completed In 2 Years – Tested, Measured, Optimized, With Multi-domain Science Data – Bring Many Of Our Science Teams Up – Each Community Thus Will Have Its Own Certificate-Based Access To its Specific Federated Data Infrastructure. • PRPv2 – Advanced Ipv6-Only Version with Robust Security Features – e.g. Trusted Platform Module Hardware and SDN/SDX Software – Support Rates up to 100Gb/s in Bursts And Streams – Develop Means to Operate a Shared Federation of Caches
- 22. Pacific Research Platform Multi-Campus Science Driver Teams • Particle Physics • Astronomy and Astrophysics – Telescope Surveys – Galaxy Evolution – Gravitational Wave Astronomy • Biomedical – Cancer Genomics Hub/Browser – Microbiome and Integrative ‘Omics – Integrative Structural Biology • Earth Sciences – Data Analysis and Simulation for Earthquakes and Natural Disasters – Climate Modeling: NCAR/UCAR – California/Nevada Regional Climate Data Analysis – CO2 Subsurface Modeling • Scalable Visualization, Virtual Reality, and Ultra-Resolution Video 22
- 23. Particle Physics: Creating a 10-100 Gbps LambdaGrid to Support LHC Researchers ATLASCMS U.S. Institutions Participating in LHC LHC Data Generated by CMS & ATLAS Detectors Analyzed on OSG Maps from www.uslhc.us
- 24. LHC Scientists Across Eight CA Universities Benefit From Petascale Data & Compute Resources across PRP SLAC Data & Compute Resource Caltech Data & Compute Resource UCSD & SDSC Data & Compute Resources UCSB UCSC UCD UCR CSU Fresno UCI Harvey Newman and Azher Mughal of Caltech have been lead researchers in 40Gbps and 100Gbps DTNs Source: Frank Wuerthwein, UCSD Physics; SDSC; co-PI PRP
- 25. Goal: Allow LHC Community to Use Five Major Data & Compute Resources in CA: SLAC, NERSC, Caltech, UCSD, SDSC • Aggregate Petabytes of Disk Space & Petaflops of Compute • Transparently Compute on Data at Their Home Institutions & These 5 Major Centers – Uniform Execution Environment – XrootD Data Federations for ATLAS & CMS – Serving Local Disks Outbound to Remotely Running Jobs – Caching Remote Data Inbound for Locally Running Jobs – HTCondor “Overflow” of Jobs from Local Cluster to Major Centers – Satisfy Peak Needs to Accelerate Path from Idea to Publication • Collaboration of PRP, SDSC, and Open Science Grid – PRP Builds on SDSC LHC-UC Project 25 Source: Frank Wuerthwein, UCSD Physics; SDSC; co-PI PRP
- 26. Two Automated Telescope Surveys Creating Huge Datasets Will Drive PRP 300 images per night. 100MB per raw image 30GB per night 120GB per night 250 images per night. 530MB per raw image 150 GB per night 800GB per night When processed at NERSC Increased by 4x Source: Peter Nugent, Division Deputy for Scientific Engagement, LBL Professor of Astronomy, UC Berkeley Precursors to LSST and NCSA PRP Allows Researchers to Bring Datasets from NERSC to Their Local Clusters for In-Depth Science Analysis- see UCSC’s Brad Smith Talk
- 27. Cancer Genomics Hub (UCSC) is Housed in SDSC CoLo: Large Data Flows to End Users at UCSC, UCB, UCSF, … 1G 8G 15G Cumulative TBs of CGH Files Downloaded Data Source: David Haussler, Brad Smith, UCSC 30 PB
- 28. Dan Cayan USGS Water Resources Discipline Scripps Institution of Oceanography, UC San Diego much support from Mary Tyree, Mike Dettinger, Guido Franco and other colleagues Sponsors: California Energy Commission NOAA RISA program California DWR, DOE, NSF Planning for climate change in California substantial shifts on top of already high climate variability SIO Campus Climate Researchers Need to Download Results from NCAR Remote Supercomputer Simulations to Make Regional Climate Change Forecasts
- 29. Collaboration Between EVL’s CAVE2 and Calit2’s VROOM Over 10Gb Wavelength EVL Calit2 Source: NTT Sponsored ON*VECTOR Workshop at Calit2 March 6, 2013
- 30. Optical Fibers Link Australian and US Big Data Researchers
- 31. Next Step: Use AARnet/PRP to Set Up Planetary-Scale Shared Virtual Worlds Digital Arena, UTS Sydney CAVE2, Monash U, Melbourne CAVE2, EVL, Chicago
- 32. The Pacific Research Platform Creates a Regional End-to-End Science-Driven “Big Data Freeway System” Opportunities for Collaboration with Other Regional Systems