#10 CTO Alert: Generative AI's transformative potential for the IT landscape.

McKinsey & Co on July 11 released a Generative AI CIO and CTO Guide in which they highlight nine actions CIOs and CTOs can take to reimagine business and technology with generative AI:

1. Move quickly to determine the company’s posture for the adoption of generative AI, and develop practical communications to, and appropriate access for, employees.
2. Reimagine the business and identify use cases that build value through improved productivity, growth, and new business models. Develop a “financial AI” (FinAI) capability that can estimate the true costs and returns of generative AI.
3. Reimagine the technology function, and focus on quickly building generative AI capabilities in software development, accelerating technical debt reduction, and dramatically reducing manual effort in IT operations.
4. Take advantage of existing services or adapt open-source generative AI models to develop proprietary capabilities (building and operating your own generative AI models can cost tens to hundreds of millions of dollars, at least in the near term).
5. Upgrade your enterprise technology architecture to integrate and manage generative AI models and orchestrate how they operate with each other and existing AI and machine learning (ML) models, applications, and data sources.
6. Develop a data architecture to enable access to quality data by processing both structured and unstructured data sources.
7. Create a centralized, cross-functional generative AI platform team to provide approved models to product and application teams on demand.
8. Invest in upskilling key roles—software developers, data engineers, MLOps engineers, and security experts—as well as the broader nontech workforce. But you need to tailor the training programs by roles and proficiency levels due to the varying impact of generative AI.
9. Evaluate the new risk landscape and establish ongoing mitigation practices to address models, data, and policies.

Reimagining the Technology Function

In this edition of the newsletter I would like to dive deeper into #3: Reimagine the Technology Function. The article emphasizes the transformative power of Generative AI for technology operations. It urges CIOs and CTOs to understand and utilize its benefits quickly. Three key areas are highlighted:

1. Software development: speed up coding, refactoring, and documentation processes, automate testing, and assist with onboarding new developers.
2. Technical debt: help reduce technical debt by accelerating code refactoring, translation, and automated test-case generation.
3. IT operations (ITOps): streamline ITOps by automating basic tasks, improving triage and resolution, and assisting in log analysis and documentation development.

Digging deeper, let's see what some specific use cases related to each one of these can be. The one's that come top of mind for me are:

1. Software development: AI coding assistant like OpenAI's Codex or GitHub Copilot. These tools can help developers to quickly write, refactor, and document their code. They can suggest solutions to problems or blocks of code, significantly speeding up development time. Moreover, these AI tools can assist new developers in understanding an unfamiliar codebase, providing them with insights about specific functions or code logic, essentially acting as an automated tutor. (I highlighted Code Interpreter—a new OpenAI tool for ChatGPT in a previous edition of the newsletter)
2. Technical debt: Refactoring legacy codebase.

Generative AI can help refactor legacy code written in an older language like COBOL into a more modern language like Python, reducing the technical debt associated with maintaining outdated, poorly documented code. Automated test-case generation can also assist by ensuring that the new code maintains the same functionality as the original, further minimizing the risk and cost of the transition.

3. IT operations (ITOps): IT help desk Automation. Generative AI can automate responses to common requests like password resets, thereby reducing the workload on IT staff. Moreover, AI algorithms can sift through log data, identifying and prioritizing issues based on their severity and impact, thus speeding up the resolution process. The use of AI can extend to generating post-incident reports, which traditionally is a time-consuming task, thereby freeing up resources for other critical tasks

Perils of Slow Adoption of Generative AI

Ignoring or slow adoption of Generative AI could lead a CTO and their organization to face several potential pitfalls:

1. Falling behind competition: If competitors are leveraging Generative AI, they can develop, test, and deploy software more quickly, and thereby respond faster to market changes. This can result in lost market share and reduced competitive edge.
2. Inefficiency and higher costs: Without the efficiencies brought about by Generative AI, teams might spend unnecessary time on routine or repetitive tasks that could have been automated. This inefficiency can lead to higher operational costs and lower productivity.
3. Increased technical debt: Overlooking the capabilities of Generative AI could lead to an accumulation of technical debt. AI can significantly help refactor and modernize the code, minimizing maintenance costs and freeing resources for more innovative projects.
4. Poor IT operations performance: Without Generative AI, routine IT operations tasks can take longer to resolve, leading to increased downtime, lower productivity, and a poor user experience. Additionally, insights derived from the AI-based analysis of log files or data streams could be missed, leading to potential performance issues or security threats.
5. Talent attrition: Developers and IT professionals want to work with cutting-edge technology. If a company lags in adopting new technologies like Generative AI, it may find it harder to attract and retain top talent, who might prefer companies that provide opportunities to work with the latest tools and technologies.

In sum, not harnessing the potential of Generative AI could lead to competitive disadvantages, inefficiencies, higher costs, and missed opportunities for innovation and talent acquisition.

Avoiding the Pitfalls and Driving for Success: A Step-by-Step Approach

A succesful path to building out Generative AI capabilities to enhance IT operations involves a step-by-step approach. Here are some recommended steps for a CTO to consider. I do not want to sound too prescriptive here, as many of these are best practices for adoption of any nascent technology by the organization; and for Digital Transformation; and would be second nature for most CTOs and technology professionals (I build upon some points highlighted in previous editions of the newsletter: #8:The Challenge: Building a Generative AI Roadmap for the Organization- Some Pointers and #5 The CXO Dilemma- How to respond to the advent of Generative AI? The 10 Mantras (Pointers) ):

1. Educate and Strategize: Start by familiarizing yourself and your team with the capabilities of Generative AI. It's important to understand its potential and limitations, and how it could impact the IT operations. Develop a strategy that outlines how AI will be used, which tasks it will perform, and how it will integrate with existing systems and workflows.
2. Identify Key Areas: Not all areas of IT operations may benefit equally from the application of Generative AI. Identify those tasks and processes that are repetitive, time-consuming, and can be automated. These might include handling routine user queries, analyzing logs, and generating reports.
3. Prioritize: Once the key areas have been identified, prioritize them based on potential impact, feasibility, and alignment with business goals. Prioritization can help ensure that the most impactful changes are made first.
4. Collaborate and Partner: Building Generative AI capabilities might require external expertise. Consider partnering with AI vendors, consulting firms, or academic institutions to access the required skills and knowledge. Collaboration can also help in learning best practices from others who have implemented similar projects.
5. Prototype and Pilot: Before going all in, start with a prototype or a pilot project in a controlled environment. This will help you understand the challenges, test the system's effectiveness, and evaluate the benefits.
6. Train and Upskill: Make sure your team has the necessary skills to work with Generative AI. This might involve training on AI systems management, data science, or coding in AI languages. Upskilling the existing workforce is crucial to ensure the smooth functioning of AI operations.
7. Implement and Iterate: After successful pilot projects, gradually implement Generative AI in the prioritized areas. Monitor the performance and continuously iterate to improve. Feedback from these initial implementations can help refine further roll-outs.
8. Ethical and Legal Considerations: Always consider the ethical implications of AI use, ensuring that decisions made by AI systems are fair, transparent, and accountable. Also, be aware of any legal or compliance requirements related to AI use in your industry.

Remember that successful adoption of Generative AI is not just about technology; it's also about managing change effectively within the organization. A thoughtful, measured approach that considers the people, processes, and culture will increase the likelihood of success.

POSTSCRIPT

Last week Gamiel Gran (COO, Mayfield Fund ) recorded an interview with me for Mayfield CXO of the Future Podcast Series. The series focuses on the goal of offering insights for CIOs & CTOs and IT Leaders on where they think the future role of IT is headed, plus to share in a way that supports the growth of the up-and-coming next generation of IT leaders. The intent is to share a range of key and strategic perspectives on leadership and operations, emerging technology, and driving innovation.

Stay Tuned. Will share the podcast when it is available.

Some other AI related developments which caught my attention:

• The Economist continues to promote the doom and gloom scenario about Generative AI. Yuval Noah Hariri has got it wrong this time and The Economist is doing a disservice to itself and it’s readers by this hyped up scaremongering and clickbait headlines. (My viewpoint is contrary to this.)
• AI enabled Drones instead of humans picking apples

• AI enabled weed control operations on organic farms

• The world's most powerful AI model suddenly got 'lazier' and 'dumber.'  A radical redesign of OpenAI's GPT-4 could be behind the decline in performance. OpenAI might be creating several smaller GPT-4 models that would act similarly to the large model but would be less expensive to run. This approach is called a Mixture of Experts, or MOE. The smaller expert models are trained on their own tasks and subject areas, meaning there could be a GPT-4 specializing in biology and one for physics, chemistry, and so on. When a GPT-4 user asks a question, the new system would know which expert model to send that query to. The new system might decide to send a query to two or more of these expert models just in case and then mash up the results.
• In June, mobile and desktop traffic to ChatGPT’s website fell by 9.7 percent globally. As for what could be causing the decline, the end of the school year may have something to do with it. With most college students on summer break, Washington Post speculates not as many young adults are using ChatGPT to write their papers. Another reason could be that companies like Samsung are prohibiting employees from using AI chatbots over the very real fear of a potential data leak.

Take Care. Stay Safe. See you again next week!