Artificial Intelligence Opportunities for State and Local DOTs: A Research Roadmap (2024)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM NCHRP Web-Only Document 403 Artificial Intelligence Opportunities for State and Local DOTs A RESEARCH ROADMAP Abhijit Sarkar Aditi Manke Matthew Camden Tammy Trimble Surendrabikram Thapa Debanjan Datta Laurel Glenn Virginia Polytechnic Institute and State University Blacksburg, VA Alejandra Medina FM Consultants Blacksburg, VA Conduct of Research Report for NCHRP Project 23-12 Submitted March 2024 

  NCHRP Web-Only Document 403 Artificial Intelligence Opportunities for State and Local DOTs A RESEARCH ROADMAP Abhijit Sarkar Alejandra Medina Aditi Manke FM Consultants Matthew Camden Blacksburg, VA Tammy Trimble Surendrabikram Thapa Debanjan Datta Laurel Glenn Virginia Polytechnic Institute and State University Blacksburg, VA Conduct of Research Report for NCHRP Project 23-12 Submitted March 2024 © 2024 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and the graphical logo are trademarks of the National Academy of Sciences. All rights reserved. NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM Systematic, well-designed, and implementable research is the most effective way to solve many problems facing state departments of transportation (DOTs) administrators and engineers. Often, highway problems are of local or regional interest and can best be studied by state DOTs individually or in cooperation with their state universities and others. However, the accelerating growth of highway transportation results in increasingly complex problems of wide interest to highway authorities. These problems are best studied through a coordinated program of cooperative research. Recognizing this need, the leadership of the American Association of State Highway and Transportation Officials (AASHTO) in 1962 initiated an objective national highway research program using modern scientific techniques—the National Cooperative Highway Research Program (NCHRP). NCHRP is supported on a continuing basis by funds from participating member states of AASHTO and receives the full cooperation and support of the Federal Highway Administration (FHWA), United States Department of Transportation, under Agreement No. 693JJ31950003. COPYRIGHT INFORMATION Authors herein are responsible for the authenticity of their materials and for obtaining written permissions from publishers or persons who own the copyright to any previously published or copyrighted material used herein. Cooperative Research Programs (CRP) grants permission to reproduce material in this publication for classroom and not-for-profit purposes. Permission is given with the understanding that none of the material will be used to imply TRB, AASHTO, APTA, FAA, FHWA, FTA, GHSA, or NHTSA endorsement of a particular product, method, or practice. It is expected that those reproducing the material in this document for educational and not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or reproduced material. For other uses of the material, request permission from CRP. DISCLAIMER The opinions and conclusions expressed or implied in this report are those of the researchers who performed the research. They are not necessarily those of the Transportation Research Board; the National Academies of Sciences, Engineering, and Medicine; the FHWA; or the program sponsors. The Transportation Research Board does not develop, issue, or publish standards or specifications. The Transportation Research Board manages applied research projects which provide the scientific foundation that may be used by Transportation Research Board sponsors, industry associations, or other organizations as the basis for revised practices, procedures, or specifications. The Transportation Research Board, the National Academies, and the sponsors of the National Cooperative Highway Research Program do not endorse products or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the object of the report. The information contained in this document was taken directly from the submission of the author(s). This material has not been edited by TRB.   

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, non- governmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president. The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president. The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president. The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine. Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org. The Transportation Research Board is one of seven major program divisions of the National Academies of Sciences, Engineering, and Medicine. The mission of the Transportation Research Board is to mobilize expertise, experience, and knowledge to anticipate and solve complex transportation-related challenges. The Board’s varied activities annually engage about 8,500 engineers, scientists, and other transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the public interest. The program is supported by state transportation departments, federal agencies including the component administrations of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation. Learn more about the Transportation Research Board at www.TRB.org. 

COOPERATI VE RESEAR CH PROGRAMS CRP STAFF FOR NCHRP WEB-ONLY DOCUMENT 403 Waseem Dekelbab, Deputy Director, Cooperative Research Programs, and Manager, National Cooperative Highway Research Program Sid Mohan, Associate Program Manager for Implementation and Technology Transfer Mireya Kuskie, Senior Program Assistant Natalie Barnes, Director of Publications Heather DiAngelis, Associate Director of Publications Jennifer J. Weeks, Publishing Projects Manager NCHRP PROJECT 23-12 PANEL Field of Administration—Area of Agency Administration J. Neil Mastin, Mott MacDonald, LLC, Raleigh, NC (Chair) Pouria Asadi, University of Rhode Island, Manchester, CT Edgardo D. Block, Connecticut Department of Transportation, Newington, CT Robert C. Cooney, eVision Partners, Inc., Raleigh, NC Sayuri Koyamatsu, Washington State Department of Transportation, Shoreline, WA Leni Oman, Spy Pond Partners, LLC, Olympia, WA Thomas Pannett, Kegler Brown Hill & Ritter, Columbus, OH Lubna Shoaib, East-West Gateway Council of Governments, St. Louis, MO Maryam Tagh Bostani, HDR, Vancouver, BC Alejandro Toriello, Georgia Institute of Technology, Atlanta, GA Faisal Saleem, FHWA Liaison AUTHOR ACKNOWLEDGMENTS We would like to thank Prof. Gerardo Flintsch and Dr. Rich Hanowski for their continuous guidance and advice to help with this project. Dr. Hanowski especially helped us in reaching out to a larger audience during the outreach activities. Prof. Flintsch helped us with his technical insight and insight on the effectivity of the project outcomes to TRB. We would also like to thank some of our colleagues at VTTI who occasionally contributed to this project through their technical know-how and expertise in AI. This includes Calvin Winkowski, Steven Gregory, Neal Feieraband from the information technology team at VTTI, Dr. Balachandar Gudury, Dr. Omkar Kaskar from the division of data and analytics, and Dr. Debanjan Datta who was a graduate student and an expert in natural language process. Without the support of them this project would have been incomplete. Finally, I want to thank Rebecca Hammond and our editing team at VTTI including Dr. Michael Buckley, Laura Krisch, and Lydia Lunning for their continuous support. 

Table Of Contents SUMMARY ..................................................................................................................................................................................... 1 CHAPTER 1 .................................................................................................................................................................................... 2 PROJECT BACKGROUND ................................................................................................................................................................ 2 Intersection of Transportation Research and AI at DOTs ...................................................................................................... 2 Expected Benefits of AI Application in State and Local DOTs ............................................................................................... 3 Scope and Challenges Towards AI Application in State and Local DOTs ............................................................................. 4 PROJECT OBJECTIVE ..................................................................................................................................................................... 4 RESEARCH APPROACH .................................................................................................................................................................. 5 SUMMARY OF TASKS AND DELIVERABLES ..................................................................................................................................... 5 SUMMARY OF PROJECT OUTCOMES ............................................................................................................................................... 6 ARRANGEMENT OF THE DOCUMENT .............................................................................................................................................. 7 CHAPTER 2 .................................................................................................................................................................................... 8 INTRODUCTION ............................................................................................................................................................................. 8 PART 1. RESEARCH TREND IDENTIFICATION USING TOPIC MODELING AND CO-OCCURRENCE MATRIX........................................... 8 RESULTS ....................................................................................................................................................................................... 9 PART 2. TREND ANALYSIS OF AI IN TRANSPORTATION RESEARCH AT DOT LEVEL ...................................................................... 12 RESULTS ..................................................................................................................................................................................... 13 PART 3. A COMPREHENSIVE SUMMARY OF AI TOOLS AND INFRASTRUCTURE .............................................................................. 14 DATA-DRIVEN AI ECOSYSTEM .................................................................................................................................................... 15 KEY COMPONENTS OF AI TOOLS ................................................................................................................................................. 16 CONCLUSION .............................................................................................................................................................................. 17 CHAPTER 3 .................................................................................................................................................................................. 18 INTRODUCTION ........................................................................................................................................................................... 18 PART 1: INTERVIEWS ................................................................................................................................................................... 18 Interview Results .................................................................................................................................................................. 18 PART 2: WORKSHOPS .................................................................................................................................................................. 20 Workshop 1 .......................................................................................................................................................................... 20 Workshop 2 .......................................................................................................................................................................... 21 CONCLUSION .............................................................................................................................................................................. 24 CHAPTER 4 .................................................................................................................................................................................. 25 RESEARCH PROBLEM STATEMENTS ............................................................................................................................................. 26 IMPLEMENTATION SCHEDULE ...................................................................................................................................................... 28 RESEARCH ROADMAP PROPOSED TIMELINE................................................................................................................................. 29 CHALLENGES AFFECTING POTENTIAL IMPLEMENTATION ............................................................................................................. 31 Workforce Development ....................................................................................................................................................... 31 CHAPTER 5 .................................................................................................................................................................................. 33 APPENDIX A: LITERATURE REVIEW .................................................................................................................................. 35 INTRODUCTION ........................................................................................................................................................................... 36 What Is AI?........................................................................................................................................................................... 36 Focus of the Task ................................................................................................................................................................. 37 PART 1. RESEARCH TREND IDENTIFICATION USING TOPIC MODELING AND CO-OCCURRENCE MATRIX......................................... 38 Results and Analysis ............................................................................................................................................................. 49 Trend Analysis...................................................................................................................................................................... 59 Key Findings ........................................................................................................................................................................ 59 PART 2. TREND ANALYSIS OF AI IN TRANSPORTATION RESEARCH ............................................................................................... 62 Methodology......................................................................................................................................................................... 62 Results .................................................................................................................................................................................. 62 Key Findings ........................................................................................................................................................................ 66 REFERENCES ............................................................................................................................................................................... 68 APPENDIX A1 ............................................................................................................................................................................. 70 APPENDIX A2 ............................................................................................................................................................................. 81 APPENDIX A3 ............................................................................................................................................................................. 92 iv 

APPENDIX B: LITERATURE SURVEY OF AI TOOLS ...................................................................................................... 108 INTRODUCTION ......................................................................................................................................................................... 109 Data-driven AI Ecosystem .................................................................................................................................................. 110 Workflow of AI Development ............................................................................................................................................. 110 Key Components of AI Tools .............................................................................................................................................. 112 AI SOFTWARE PLATFORMS ....................................................................................................................................................... 114 Traditional ML-based Solutions ......................................................................................................................................... 114 Deep Learning Frameworks ............................................................................................................................................... 116 Open-source Toolboxes ...................................................................................................................................................... 118 Statistical Analysis Platforms ............................................................................................................................................. 120 Development and Deployment............................................................................................................................................ 120 Visualization and Analytics Software ................................................................................................................................. 121 Data Annotation Tools ....................................................................................................................................................... 124 Solution-based Systems ...................................................................................................................................................... 128 AI DATA MANAGEMENT ........................................................................................................................................................... 132 Data Provider .................................................................................................................................................................... 132 Data Modality .................................................................................................................................................................... 133 Data Sharing ...................................................................................................................................................................... 134 Large-scale Data Collection, Storage, and Management................................................................................................... 135 LARGE-SCALE DATA PROCESSING ON GPU/EDGE/CLOUD ......................................................................................................... 137 Computing .......................................................................................................................................................................... 137 Edge Computing ................................................................................................................................................................. 137 Cloud Computing ............................................................................................................................................................... 137 COST AND BENEFIT ................................................................................................................................................................... 138 AI Software Tools ............................................................................................................................................................... 138 Big Data Management ....................................................................................................................................................... 140 Computing .......................................................................................................................................................................... 141 Outsourcing, Crowdsourcing, Knowledge Building ........................................................................................................... 141 CONCLUSION ............................................................................................................................................................................ 143 REFERENCES ............................................................................................................................................................................. 144 APPENDIX B1 ........................................................................................................................................................................... 145 CNNs .................................................................................................................................................................................. 145 Recurrent Neural Network ................................................................................................................................................. 145 Auto-encoder ...................................................................................................................................................................... 145 APPENDIX C: SYNERGY ANALYSIS AND INTERVIEW WITH DOTS ......................................................................... 146 INTRODUCTION ......................................................................................................................................................................... 147 METHODS ................................................................................................................................................................................. 147 Participant Recruitment ..................................................................................................................................................... 148 Data Collection .................................................................................................................................................................. 148 Data Analysis ..................................................................................................................................................................... 148 RESULTS AND ANALYSIS ........................................................................................................................................................... 149 Current AI Practices .......................................................................................................................................................... 149 Challenges with AI Work.................................................................................................................................................... 152 Ways to Overcome Challenges ........................................................................................................................................... 153 Workforce and Infrastructure ............................................................................................................................................. 153 Future Scope of AI Integration ........................................................................................................................................... 155 SUMMARY ................................................................................................................................................................................ 156 CONCLUSIONS........................................................................................................................................................................... 158 REFERENCES ............................................................................................................................................................................. 159 APPENDIX C1 ........................................................................................................................................................................... 160 APPENDIX C2 ........................................................................................................................................................................... 161 APPENDIX C3 ........................................................................................................................................................................... 164 APPENDIX D: WORKSHOP REPORT .................................................................................................................................. 165 INTRODUCTION ......................................................................................................................................................................... 166 Project Progress and Scope of This Document .................................................................................................................. 166 METHODS ................................................................................................................................................................................. 167 Step 1: Participant Recruitment and Outreach .................................................................................................................. 168 Step 2: Finalize Topics ....................................................................................................................................................... 168 Step 3: Conduct Workshops ............................................................................................................................................... 171 v 

Step 4: Analysis of Outcomes ............................................................................................................................................. 172 RESULTS ................................................................................................................................................................................... 172 Part I: Workshop 1 ............................................................................................................................................................. 172 Part II: Workshop 2............................................................................................................................................................ 179 CONCLUSION ............................................................................................................................................................................ 188 APPENDIX E: RESEARCH PROBLEM STATEMENTS ..................................................................................................... 189 INTRODUCTION ......................................................................................................................................................................... 190 Research Problem Statement 1: Case Studies of Implementation of Artificial Intelligence Programs in State and Local Departments of Transportation .......................................................................................................................................... 193 Research Problem Statement 2: Toolbox to Guide the Selection and Deployment of Artificial Intelligence Technologies in State and Local Transportation Agencies........................................................................................................................... 195 Research Problem Statement 3: Workforce Needs and Development to Prepare Transportation Agencies for the Application of Existing and Emerging Artificial Intelligence Approaches ......................................................................... 197 Research Problem Statement 4: Implementable Funding Strategies for Artificial Intelligence Opportunity Applications for State and Local DOTs ........................................................................................................................................................ 199 Research Problem Statement 5: Develop a Guidebook to Understand the Vulnerability and Security Concerns for AI-based Transportation Solutions .................................................................................................................................................... 201 Research Problem Statement 6: Exploring the Integration of AI-based Methods in Multimodal Transportation Planning ........................................................................................................................................................................................... 203 Research Problem Statement 7: Validation of Artificial Intelligence Applications for Automated Pavement Condition Evaluation .......................................................................................................................................................................... 205 Research Problem Statement 8: Explore Natural Language Processing-based Methods for Document Management and Public Interaction at DOTs ................................................................................................................................................ 207 Research Problem Statement 9: Develop a Guidebook for Successful Collaboration with Industry Partners that Provides AI-based Solutions. ............................................................................................................................................................ 209 Research Problem Statement 10: Guidebook to Create Sharable, Reliable Sources of Data Sets ..................................... 211 Research Problem Statement 11: Creating a framework to process and manage data collected by DOTs. ...................... 213 APPENDIX F: IMPLEMENTATION OF RESEARCH FINDINGS AND DISSEMINATION PLAN .............................. 215 INTRODUCTION ......................................................................................................................................................................... 216 PROBLEM STATEMENTS ............................................................................................................................................................ 216 Project Workshop ............................................................................................................................................................... 216 IMPLEMENTATION PLAN ............................................................................................................................................................ 220 CHALLENGES AFFECTING POTENTIAL IMPLEMENTATION ........................................................................................................... 225 Workforce Development ..................................................................................................................................................... 229 Developing Workforce........................................................................................................................................................ 231 CONCLUSION ............................................................................................................................................................................ 232 vi 

List of Figures Figure 1. Project task outline. ....................................................................................................................... 5  Figure 2. Chord diagram illustrating the interdependencies among transportation topics. The width of the arcs represents the strength of interrelation between transportation topics, providing insights into their relative importance and interdependencies. ................................................................................................ 10  Figure 3. A detailed view of different AI topics used in Transportation topics "Traffic Management" and "Highway Management/Design." ............................................................................................................... 11  Figure 4. Trends of different AI topics over the years. ............................................................................... 12  Figure 5. Transportation areas across the selected projects ........................................................................ 13  Figure 6. Distribution of Projects over the years ........................................................................................ 14  Figure 7. A typical AI ecosystem with multiple components, including data processing pipeline, human- in-the-loop interaction, and development for AI-based systems. ............................................................... 15  Figure 8. High level overview of topic modeling. It takes a several keywords and identifiers and automatically cluster them to a number of independent topics ................................................................... 39  Figure 9. Literature review to assess the scope of AI in transportation research and development. .......... 40  Figure 10. Schematic diagram of the process to find associations and interdependencies between transportation topics and AI. ....................................................................................................................... 41  Figure 11. Schematic overview of the process to analyze trends. .............................................................. 42  Figure 12. Schematic of the overall idea of the topic modeling ................................................................. 43  Figure 13. (a) Word cloud for transportation topic "traffic management" (b) Word cloud for transportation topic "accessibility.” ................................................................................................................................... 43  Figure 14. (a) World cloud for AI topic "numerical methods and optimization" (b) Word cloud for AI topic "Advanced Machine Learning." ......................................................................................................... 46  15. Overlap within transportation topics. .................................................................................................... 49  Figure 16. Overlap within AI topics ........................................................................................................... 50  Figure 17. Interdependencies within transportation topics. ........................................................................ 51  Figure 18. Interdependency of traffic management with other transportation topics. ................................ 52  Figure 19. Interdependency of winter road management with other transportation topics. ........................ 53  Figure 20. Sankey plot of transportation problems solved by AI topics. .................................................... 54  Figure 21. AI topics used in solving work-zone analysis. .......................................................................... 54  Figure 22. Interdependency of commercial vehicle and freight operations with other transportation topics. .................................................................................................................................................................... 57  Figure 23. Trends of different AI topics over the years. We can see some topics like big data analysis, advanced machine learning has positive trend while statistical machine learning has slight negative trend. Topics like Evaluation stays similar across years. ...................................................................................... 61  Figure 24. Transportation areas across the selected projects. ..................................................................... 63  Figure 25. Distribution of Projects over the years ...................................................................................... 65  Figure 26. A typical AI ecosystem with multiple components, including data processing pipeline, human- in-the-loop interaction, and development for AI-based systems. ............................................................. 110  Figure 27. Typical workflow of an AI deployment process with six components (Adapted from Volet, 2018). ........................................................................................................................................................ 111  Figure 28. An example dashboard from Power BI that can summarize data from tabular data and link them to maps and other modalities. Image from Antdata (2021). ............................................................ 122  Figure 29. Key elements related to data annotation tools (CloudFactory, n.d.)........................................ 124  Figure 30. Various commercial data annotation tools and their supported data types (CloudFactory, n.d.). .................................................................................................................................................................. 125  Figure 31. Various commercial data annotation tools and their supported data types and capabilities (CloudFactory, n.d.). ................................................................................................................................. 126  vii 

Figure 32. A developer needs to guarantee the performance of an algorithm in the targeted image. The same image may produce different results for different algorithms. ........................................................ 132  Figure 33. Number of responses on AI integration by the DOTs. ............................................................ 157  Figure 34. Number of responses on challenges faced by DOTs. .............................................................. 157  Figure 35. Project task outline. ................................................................................................................. 166  Figure 36. Top transportation research areas identified by workshop participants. ................................. 172  Figure 37. Challenges faced at DOT level. ............................................................................................... 174  Figure 38. Type of Organizations needed by DOTs. ................................................................................ 177  Figure 39. Difficult in determining if AI program is ready for implementation. ...................................... 178  viii 

List of Tables Table 1. List of milestones (M) and deliverables (D). .................................................................................. 6  Table 2. Prioritize the ideas from one to six for workforce and infrastructure needs, readiness, and evaluation of AI programs. ......................................................................................................................... 22  Table 3. Prioritize ideas from one to eight for current practices of AI in transportation and challenges faced by DOTs. ........................................................................................................................................... 22  Table 4. Rank the Roadmap ideas based on the likeliness of receiving funding. ....................................... 23  Table 5. Final research problem statements, objectives and areas. ............................................................. 26  Table 6. Problem statement potential partnerships and expected budget and duration of each research problem statement proposed. ...................................................................................................................... 28  Table 7. Research roadmap timeline to show the temporal relations between the projects. For more details on dependencies, please refer to the text..................................................................................................... 30  Table 8. Journal articles in the report. ......................................................................................................... 40  Table 9. Definition of transportation AI topics. .......................................................................................... 44  Table 10. Definition of AI topics used in this project. ................................................................................ 46  Table 11. Top five AI topics used for transportation topics. ...................................................................... 55  Table 12. Associations Within Transportation Topics. We presented the top five transportation topics that are seen to have strong co-occurrence in research. ..................................................................................... 57  Table 13. Number of AI topic papers having co-occurrence with transportation topics (Part A). ............. 92  Table 14. AI Topics having co-occurrence with transportation topics (Part B). ........................................ 93  Table 15. Correlation of normalized number of papers and years (Part A). ............................................... 95  Table 16. Correlation of normalized number of papers and years (Part B). ............................................... 97  Table 17. A cost-benefit analysis of different AI-based tool sets available for use. ................................. 138  Table 18. Cost-benefit analysis for big data management. ....................................................................... 141  Table 19. Cost-benefit analysis of computing resources for AI applications. .......................................... 141  Table 20. Grouping the roadmap ideas by research focus areas. .............................................................. 170  Table 21. Prioritize the ideas from one to six for workforce and infrastructure needs, readiness, and evaluation of AI programs. ....................................................................................................................... 185  Table 22. Prioritize ideas from one to eight for current practices of AI in transportation and challenges faced by DOTs. ......................................................................................................................................... 185  Table 23. Rank the Roadmap ideas based on the likeliness of receiving funding. ................................... 186  Table 24. Grouping the roadmap ideas by research focus areas. .............................................................. 190  Table 25. Project ideas with corresponding research areas....................................................................... 216  Table 26. Final Research Problem Statements, Objectives, and Areas. (1) workforce development and infrastructure development, (2) readiness and evaluation of AI, (3) challenges in adopting AI, (4) current practices and prioritization, (5) external collaboration, and (6) equity, policy & planning. ..................... 218  Table 27. Problem statement potential partnerships and expected budget and duration of each research problem statement proposed. .................................................................................................................... 223  Table 28. Risk rating and probability definitions. .................................................................................... 226  Table 29. Challenge matrix. ...................................................................................................................... 226  Table 30. Typical roles for AI personnel (Source: GSA, 2022). .............................................................. 230  ix