In this episode, I talk with Will Uppington, CEO of TruEra about how trust is a critical output of machine learning and AI systems and AI quality management must be baked into development in order to build trust-worthy systems. We delved into the five core dimensions of model quality, the importance of iterating data and models in parallel, the state of the model development platforms and tools market, the regulatory environment around trusted AI, and ended with some career advice for people entering the field.
-- Timing –
03:48 Trust is critical to machine learning success
05:21 We’re missing AI Quality Management
06:58 How do you define model quality?
11:22 Model risk management seems to happen “too late”
11:42 AI has some large failure cases
15:00 How do we improve quality?
17:00 AI needs to be explainable
18:05 The five dimensions of model quality
25:18 What’s more important - the data or the model?
28:43 Data and models need to be developed in tandem
31:00 State of the model development tool-chain
35:59 Open source doesn’t mean somebody read the code
37:10 The regulatory environment
48:04 AI is already hard. Why add more friction?
53:19 Trusted systems come from quality development processes
59:15 Will’s career advice to people entering the field
-- Links --
Andrew Ng - From Model-centric to Data-centric AI: https://www.youtube.com/watch?v=06-AZXmwHjo