Julian Elliott

Two key soft-skill areas for many Data Scientists to improve are : 1) clear communication 2) actively seeking feedback With communication, the areas I would advise to focus on are simplicity and visuals. The key stakeholders are unlikely to be technical, so its essential to simplify and prioritise what you need to communicate : focus on the 80/20 of the message ie the key variables that explain 80% of the variation in the model. Visuals are essential. Try to develop clear visuals that the stakeholders can use themselves to explain to others. Regarding feedback, don't wait until formal meetings to ask for feedback. Instead be proactive and seek informal feedback ; in this way you will receive plenty of help and domain knowledge.

I advise any Data Scientist, and all those in Data Science teams that I lead, to always have a current and regularly used "individual development plan" (IDP). Sometimes called other things such as a "personal development plan" (PDP). “Invest in as much of yourself as you can. You are your own biggest asset by far.” – Warren Buffett My advice on an IDP, especially after a promotion is to revisit it entirely, every year, and to update every quarter. The IDP should be balanced across two key dimensions: 1) technical skills 2) soft skills and also have a balance between the activities you need to do your current role better, and the capabilities you need to develop to be ready for your next role. This simple framework is well proven.

ML Ops, or Machine Learning Operations, is considered Best Practice in this area. ML Ops is important for deploying and monitoring data science models in production for several reasons: Efficient deployment: automates and standardizes processes like packaging, versioning, and deploying models, reducing the chance of errors and saving time. Continuous Monitoring: ML Ops incorporates monitoring and alerting systems that track metrics. This allows you to detect and address issues promptly, ensuring the models deliver accurate and reliable results. Overall, in my experience, ML Ops plays a crucial role in bridging the gap between data science and operations, enabling the successful deployment of data science models in production at scale.

The people designing the A/B test will typically have a strong statistics background. In many businesses, the stakeholders eg. commercial teams, will not have the same statistical knowledge and background. It is very important to agree how much risk / uncertainty in the test results the stakeholders are comfortable with, because this will impact test design and implementation. So I would suggest having a proper discussion about confidence intervals up front, ideally establishing standards. 90% or 95% are often used depending on the impact of a wrong read. Confidence intervals will then help stakeholders make informed decisions. If the results are within the confidence interval then it provides an agreed basis for implementing changes.

The choice of which metrics to use will depend on the nature of your forecasting problem, data, and specific goals. In most cases, you will need to use a combination of metrics to get a comprehensive view of your predictive model's performance. For example in forecasting, a few simple metrics like Forecast Bias, Root Mean Squared Error, Tracking Signal (used in time series forecasting to monitor deviations over time. It's a measure of whether the forecast errors are trending in a particular direction) are useful. To help you select metrics, pick those which represent your use case, for example if shape is important, RMSE is a good metric. Having metrics which are easy to explain to stakeholders will build confidence in the forecasts.

Patience is probably the most important quality to overcome challenges that will arise in any agile implementation. Specifically, I would say three areas are important : 1) Continuous Improvement: Agile is all about continuous improvement. After each sprint, always hold a retrospective meeting to discuss what went well, what could be improved, and try to implement 2 or 3 concrete improvements in the next sprint. 2) Training and Coaching: Provide ongoing training and coaching to help your team understand and embrace Agile practices. Consider bringing in Agile experts or coaches to support. 3) Tools and Technology: Use Agile project management tools, such as Jira, Trello, or Asana, to manage your Agile processes and tasks efficiently.