Enterprise AI Practice — Responsibilities and Deliverables
Using TOGAF and CRISP-DM to build Enterprise AI Practice
As the number of data science projects increases, it's important to make sure these initiatives align to overall business objectives and adhere to the enterprise architecture principles and practices. Most of them today use a process model called Cross Industry Standard for Data Mining (CRISP-DM) is the process for developing data science projects.
CRISP-DM Model:
It was originally devised as a standard process model for data mining. However, this model is currently is increasingly being used by data science projects. The CRISP-DM methodology is described in terms of a hierarchical process model, comprising four levels of abstraction (from general to specific): phases, generic tasks, specialized tasks, and process instances.
As per the paper published by Rüdiger Wirth and Jochen Hipp on CRISP-DM
The description of phases and tasks as discrete steps performed in a specific order represents an idealized sequence of events. In practice, many of the tasks can be performed in a different order and it will often be necessary to backtrack to previous tasks and repeat certain actions.
The agility of this process model makes it easy to adapt the model for any data science project. Here is a representation of the model and flow that generally represents the Data Sciences project workflow.
Now when a loose implementation of a process like CRISP-DM with agile project management will provide better results. More specifically we have seen that the Kanban model followed along with agile works best rather than a Scrum model.
When Implementing data science at scale in an enterprise we need to make sure that the overall governance and architecture framework is still in place. I would like to focus on how these process steps are related to the general TOGAF ADM and what should be the responsibilities of individual project teams and Enterprise Architecture (or Governance) teams and their deliverables.
Business Requirements:
Data Collection & Pre-Processing
Data Analysis & Preparation
Modeling
Evaluation
Deployment & Serving
The overall governance and validation, selection of the tools, techniques, and process can reside with the enterprise architecture teams but the actual implementations should be left to the individual teams with better business domain context.
Reference
http://www.cs.unibo.it/~montesi/CBD/Beatriz/10.1.1.198.5133.pdf
https://www.linkedin.com/pulse/data-science-architecture-building-bridges-crisp-dm-togaf-schiller/
