content.dataiku.com Open in urlscan Pro
35.174.108.193  Public Scan

Form analysis
0 forms found in the DOM

Text Content

Thumbnails Document Outline Attachments Layers

Current Outline Item
Cover
Dataiku
Copyright
Table of Contents

Preface
Who Should Read This Book
What Readers Will Learn

Preliminary Book Outline
Part 1
Part 2
Stop Going Fast and Breaking Things
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments

Chapter 1. Contemporary Model Governance
Basic Legal Obligations
AI Incidents

Organizational and Cultural Competencies for Responsible AI
Accountability
Drinking Your Own Champagne
Diverse and Experienced Teams
“Going Fast and Breaking Things”

Organizational Processes for Responsible AI
Forecasting Failure Modes
Model Risk Management
Beyond Model Risk Management

Case Study: The Rise and Fall of Zillow’s iBuying
Fallout
Lessons Learned

Chapter 2. Interpretable and Explainable Machine Learning
Important Ideas for Interpretability and Explainability

Interpretable Models
Additive Models
Decision Trees
An Ecosystem of Interpretable Machine Learning Models

Post-hoc Explanation
Feature Importance
Surrogate Models
Plots of Model Performance
Cluster profiling
Stubborn Difficulties of Post-hoc Explanation in Practice
Case: Graded by Algorithm
Resources

Chapter 3. Debugging Machine Learning Systems for Safety and Performance

Training
Reproducibility
Data Quality and Feature Engineering
Model Specification

Model Debugging
Software Testing
Traditional Model Assessment
Residual Analysis for Machine Learning
Sensitivity Analysis
Benchmark Models
Machine Learning Bugs
Remediation: Fixing Bugs

Deployment
Domain Safety
Model Monitoring

Case Study: Death by Autonomous Vehicle
Fallout
An Unprepared Legal System
Lessons Learned
Resources

Chapter 4. Managing Bias in Machine Learning

ISO and NIST Definitions for Bias
Systemic Bias
Statistical Bias
Human Biases and Data Science Culture
United States Legal Notions of ML Bias
Who Tends to Experience Bias from ML Systems
Harms That People Experience

Testing for Bias
Testing Data
Traditional Approaches: Testing for Equivalent Outcomes
A New Mindest: Testing for Equivalent Performance Quality
On the Horizon: Tests for the Broader ML Ecosystem
Summary Test Plan

Mitigating Bias
Technical Factors in Mitigating Bias
The Scientific Method and Experimental Design
Bias Mitigation Approaches
Human Factors in Mitigating Bias
Case: The Bias Bug Bounty
Resources

Chapter 5. Security for Machine Learning

Security Basics
The Adversarial Mindset
CIA Triad
Best Practices for Data Scientists

Machine Learning Attacks
Integrity Attacks: Manipulated Machine Learning Outputs
Confidentiality Attacks: Extracted Information
General AI Security Concerns

Counter-measures
Model Debugging for Security
Model Monitoring For Security
Privacy-enhancing Technologies
Robust Machine Learning
General Countermeasures

Case Study: Real-world Evasion Attacks
Lessons Learned
Resources

Chapter 6. Explainable Boosting Machines and Explaining XGBoost

Concept Refresher
Additivity vs. Interactions
Steps Toward Causality with Constraints
Partial Dependence and Individual Conditional Explanation
Shapley Values
Model Documentation

The GAM Family of Interpretable Models
Elastic Net Penalized GLM w/ Alpha and Lambda Search
Generalized Additive Models
GA2M and Explainable Boosting Machines

XGBoost with Constraints and Explainable Artificial Intelligence
Constrained and Unconstrained XGBoost
Explaining Model Behavior with Partial Dependence and ICE
Decision Tree Surrogate Models as an Explanation Technique
Shapley Value Explanations
Resources

Chapter 7. Red-teaming XGBoost

Concept Refresher
CIA Triad
Attacks
Countermeasures

Model Training
Attacks for Red-teaming
Conclusion
Resources:
About the Authors



Previous

Next
Highlight All Match Case
Match Diacritics Whole Words

Color
Size
Color
Thickness
Opacity
Presentation Mode Open Print Download Current View

Go to First Page Go to Last Page

Rotate Clockwise Rotate Counterclockwise

Text Selection Tool Hand Tool

Page Scrolling Vertical Scrolling Horizontal Scrolling Wrapped Scrolling

No Spreads Odd Spreads Even Spreads

Document Properties…
Toggle Sidebar

Find
Previous

Next
(1 of 228)
Presentation Mode Open Print Download Current View

FreeText Annotation Ink Annotation

Tools
Zoom Out

Zoom In
Automatic Zoom Actual Size Page Fit Page Width 50% 75% 100% 125% 150% 200% 300%
400%

AI Governance With Dataiku
The Platform for Everyday AI
Leveraging one central solution for AI means more transparent, repeatable, and
scalable AI. Dataiku gives
people (whether technical and working in code or on the business side and low-
or no-code) the ability to
make better day-to-day decisions with data.
Keep critical AI initiatives up and running with Dataiku, which offers
model monitoring, drift detection, automatic model retraining, easy
production project model updates, automated CI/CD, and more.
Dataiku provides critical capabilities for explainable and responsible
AI, including reports on feature importance, partial dependence plots,
subpopulation analysis, model fairness, and individual prediction
explanations.
Advanced AI Governance including customizable governance plans,
production sign-off, risk and value analysis together with permissions
management, user directory integration, audit trails, and secure API
access make Dataiku the perfect choice for streamlined AI Governance
across the organization.
Deploy, Monitor, & Manage Machine Learning Projects in Production
Understand Outputs, Increase Trust, & Identify Potential Bias
Manage Risk & Ensure Compliance at Scale
©2022 dataiku | dataiku.com


Patrick Hall, James Curtis, and Parul Pandey
Machine Learning for High-Risk
Applications
Techniques for Responsible AI
FIRST EDITION
Boston Farnham Sebastopol TokyoBeijing Boston Farnham Sebastopol TokyoBeijing


































































































































































































































More Information Less Information
Close

Enter the password to open this PDF file.

Cancel OK
File name:

-

File size:

-


Title:

-

Author:

-

Subject:

-

Keywords:

-

Creation Date:

-

Modification Date:

-

Creator:

-


PDF Producer:

-

PDF Version:

-

Page Count:

-

Page Size:

-


Fast Web View:

-

Close
Preparing document for printing…
0%
Cancel

Next 
Next 

Deploy and Maintain More Models in Production With Dataiku 10
Dataiku 10 builds on core strengths related to model deployment and brings
additional tools to ML operators maintaining live models in production.
LinkedIn LinkTwitter LinkFacebook LinkEmail LinkLike Button
GET DATAIKU NEWS



GET MORE CONTENT FROM DATAIKU!

You seem new here, sign up to our newsletter for updates on new content and
videos
O'Reilly Machine Learning for High-Risk Applications: Techniques for Responsible
AIpdf
Deploy and Maintain More Models in Production With Dataiku 10webpage
[VIDEO] Who's Responsible for Responsible AI?video
Navigating Targeted Ad Bias With Responsible AIwebpage
[VIDEO] Responsible AI Is About More Than Modelsvideo
So You've Built a Fair Model, Now What?webpage
[DEMO] Dataiku for Coders (Data Scientists, Engineers, & More)video




GET MORE CONTENT FROM DATAIKU!

You seem new here, sign up to our newsletter for updates on new content and
videos