Don’t drown in work. Time management and priority management tips for Data Scientists

A Reddit post reminded me that Data Scientists often struggle with time management and priority management. There is a customer expectation of fast turnaround – especially in Analytics. Data is complex. Supporting tools are only starting to catch up with their engineering equivalents. This post shares time management and priority management tips for data scientists that I have learned in building teams. The temptation is to lean on automation and ‘better tools’ but the reality is that discipline and assertiveness will have the biggest effect.

Three challenges to effective time management and priority management for data scientists

The main challenges to effective time management and priority management for data scientists are:

  • Allowing stakeholders to invade team time.
  • Allowing stakeholders to invade priorities.
  • Lack of internal processes and discipline cause high communication and coordinate overhead, reducing the team’s effectiveness.

Time management and priority management tips

Preventing invasion of team time

These tips are straightforward and aim to move disruptions to controlled times on the team’s schedule.

  • blocking team technical time in calendars allows data scientists to engage in several hours of focused work.
  • no meeting blocks or even no meeting days again allow productive data science to be done.
  • office hours help stakeholders who have questions meet with a team member on the team’s schedule.
  • an equivalent of ‘first line support’ allows dedicated team members to respond to quick fire requests without the whole team being disrupted. Done on a rotation, this is an efficient way for mature teams to defend their core working time.

Preventing invasion of team priorities

Firstly, it is impossible to prioritise without a searchable list of the active work and incoming work the team faces. Many teams are pulled in different directions because they cannot communicate their current work and their priorities to stakeholders.

  • Workflow tracking can be as simple as a spreadsheet or as complex as modern workflow tracking software depending on the size of the team, the nature of the work and the number of stakeholders.
  • Maintain backlogs of work that the team are aware of but hasn’t been started yet. This allows you to measure and communicate the work on the team’s plate as well as constructively discuss what should be done next.
  • Educate on how to say ‘no’. This is difficult, involves and cultural change and is often something junior team members struggle with. It is much easier to say No when you can be clear on the day’s current priorities, when the work waiting on the backlog can be discussed and when there are other avenues for a solution such as the office hours and first line mentioned above.

These tips involve discipline from the team to always write up requests as well as training in how to write requests clearly in a way that the deliverable is understood.

Improving team internal processes

Even with the tools of prioritisation and time management in place, data science teams, by the nature of their work, are often far less effective than they could be. Changing data, changing understanding of business processes, a cultural lack of awareness of version control and release management, a habit of ‘solo’ development in notebooks and personal spaces all contribute to burdensome internal communication. The incorrect reaction is often bespoke configuration documentation to try and keep the team in sync.

Convention is a far more effective approach that configuration for coordinating the team’s activities and outputs. Fortunately, Guerrilla Analytics can help with principles and practices to help data scientists adopt conventions that allow them to operate more effectively with minimal overhead.

Building a Data Science Capability: Inspirational Keynote at Data Leaders Summit Europe

I’ve just delivered the inspirational keynote at Data Leaders Summit Europe, 2018. Lots of great engagement and feedback. In particular, it seems people liked a clear definition of what data science actually is and the practical steps (miss-steps) I took in building a capability at Sainsbury’s.

Continue reading “Building a Data Science Capability: Inspirational Keynote at Data Leaders Summit Europe”

13 Steps to Better Data Science: A Joel Test of Data Science Maturity

Data Science teams have different levels of maturity in terms of their ways of working. In the worst case, every team member works as an individual. Results are poorly explained and impossible to reproduce. In the best case, teams reach full scientific reproducibility with simple conventions and little overhead. This leads to efficiency and confidence in results and minimal friction in productionising models. It is important to be able to measure a team’s maturity so that you can improve your ways of working and so you can attract and retain great talent. This series of questions is a Joel Test of Data Science Maturity. As with Joel’s original test for software development, all questions are a simple Yes/No and a score below 10 is cause for concern. Depressingly, many teams seem to struggle around a 3.

Continue reading “13 Steps to Better Data Science: A Joel Test of Data Science Maturity”

Data Science jargon buster – for Data Scientists

Data Scientists need to communicate without jargon so customers understand, believe and care about their recommendations. Here is a Data Science jargon buster to help with communicating data science project results.

Bamboozled. That’s your customers’ reaction to the Data Scientists in your organisation. Data Scientists need to communicate without jargon so customers understand, believe and care about their recommendations. Here is a Data Science jargon buster to help with communicating data science project results.

Continue reading “Data Science jargon buster – for Data Scientists”

Reproducible Data Science: faster iterations, reviews and production

Data Science involves applying the scientific method to the discovery of opportunities and efficiencies in business data. An essential part of the scientific method is reproducibility. Reproducible Data Science is essential for scientific credibility but also improves your Data Science efficiency in 3 keys ways – faster iterations, reviews and pushes to production.

Data Science involves applying the scientific method to the discovery of opportunities and efficiencies in business data. An essential part of the scientific method is reproducibility. Reproducible Data Science is essential for scientific credibility but also improves your Data Science efficiency in 3 keys ways – faster iterations, reviews and pushes to production.
If you start to apply the 7 Principles of Guerrilla Analytics your teams will quickly achieve reproducibility and benefit from these efficiencies.

Continue reading “Reproducible Data Science: faster iterations, reviews and production”

To Become A Data Scientist, Focus On Competencies before Skills

Too often, the path to becoming a Data Scientist focuses on technology skills in vogue rather than more permanent competencies. Competencies are a more general combination of skills, behaviours and knowledge. You can have great Powerpoint skills creating beautiful slides but still be a terrible communicator. It is competencies that are most important when you build a Data Science career that is robust to changing trends in skills like languages and technology platforms. This post describes the most important competencies for being successful in data science.

Too often, the path to becoming a Data Scientist focuses on technology skills in vogue rather than more permanent competencies. Competencies are a more general combination of skills, behaviours and knowledge. You can have great Powerpoint skills creating beautiful slides but still be a terrible communicator. It is competencies that are most important when you build a Data Science career that is robust to changing trends in skills like languages and technology platforms. This post describes the most important competencies for being successful in data science.

Continue reading “To Become A Data Scientist, Focus On Competencies before Skills”

The Rigour of Science is Essential for Successful Data Science in Business

The rigour of Science is essential for successful Data Science in business. The scientific method helps drive successful data science projects in business. This post will show you how.

The rigour of Science is essential for successful Data Science in business. The scientific method helps drive successful data science projects in business. This post will show you how.

Continue reading “The Rigour of Science is Essential for Successful Data Science in Business”

Data Science – A Definition And How To Get Started

Confusion, hype, failure to start. Data Science has huge potential to change an organisation. But many organisations become mired in the associated cultural, technological and people change. Data Science is delivered as an interesting report rather than a driver of change. Data Science identifies algorithms that are run in the safety of a lab but never make it into production.

This is my keynote talk from the Polish Data Science with Business Conference.

Confusion, hype, failure to start. Data Science has huge potential to change an organisation. But many organisations become mired in the associated cultural, technological and people change. Data Science is delivered as an interesting report rather than a driver of change. Data Science identifies algorithms that are run in the safety of a lab but never make it into production.

This is my keynote talk from the Polish Data Science with Business Conference.

Continue reading “Data Science – A Definition And How To Get Started”

Irish Language Data Science lecture at Engineers Ireland

I gave the following lecture to Engineers Ireland which is the professional body for Engineers. It’s about “Data Science and the benefits for engineering” and is entirely in Irish.

It was an interesting exercise to brush up on my Gaelic and also to see the wealth or resources that now exist for using Irish with modern technical vocabulary. If you are curious or are trying to get your Gaelic up to scratch then please get in touch!

I may be the first person to coin Data Science in Gaeilge!

I gave the following lecture to Engineers Ireland which is the Irish professional body for Engineers. The lecture is about “Data Science and the benefits for engineering” and is entirely in Irish.

It was an interesting exercise to brush up on my Gaelic and also to see the wealth of resources that now exist for using Irish with modern technical vocabulary. If you are curious or are trying to get your Gaelic up to scratch then please get in touch!

In terms of content, it covers what data and data science look like and how traditional engineering problems might benefit from the application of data science.

The full video is linked below.

And here are the slides 2016-04 Engineering Ireland_04_Gaeilge.

Data Science Patterns: Preparing Data for Agile Data Science

Are you a data scientist working on a project with constantly changing requirements, flawed changing data and other disruptions? Guerrilla Analytics can help.

The key to a high performing Guerrilla Analytics team is its ability to recognise common data preparation patterns and quickly implement them in flexible, defensive data sets.

After this webinar, you’ll be able to get your team off the ground fast and begin demonstrating value to your stakeholders.

You will learn about:
* Guerrilla Analytics: a brief introduction to what it is and why you need it for your agile data science ambitions
* Data Science Patterns: what they are and how they enable agile data science
* Case study: a walk through of some common patterns in use inreal projects

I recently gave a webinar on Data Science Patterns. The slides are here.

Data Science Patterns, as with Software Engineering Patterns, are ‘common solutions to recurring problems’. I was inspired to put this webinar together based on a few things.

  • I build Data Science teams. Repeatedly, you find teams working inconsistently in terms of the data preparation approaches, structures and conventions they use. Patterns help resolve this problem. Without patterns, you end up with code maintenance challenges, difficulty in supporting junior team members and all round team inefficiency due to having a completely ad-hoc approach to data preparation.
  • I read a recent paper ‘Tidy Data’ by Hadley Wickham in the Journal of Statistical Software http://vita.had.co.nz/papers/tidy-data.pdf. This paper gives an excellent clear description of what ‘tidy data’ is – the data format used by most Data Science algorithms and visualizations. While there isn’t anything new here if you have a computer science background, Wickham’s paper is an easy read and has some really clear worked examples.
  • My book, Guerrilla Analytics (here for USA or here for UK), has an entire appendix on data manipulation patterns and I wanted to share some of that thinking with the Data Science community.

I hope you enjoy the webinar and find it useful. You can hear the recording here. Do get in touch with your thoughts and comments as I think Data Science patterns is a huge area of potential improvement in the maturity of Data Science as a field.