SlideShare a Scribd company logo
1 of 67
Download to read offline
Skeptics in the Church of Data Getting Evangelical
AaronAldrich–CageData/VictorOps
@CrayZeigh#Galaxz17
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 2
Aaron Aldrich
@CrayZeigh
@cagedata
@victorops
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 3
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 4
incidents happen
and they suck
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 5
we monitor
systems religiously
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 6
The Trinity of System Monitoring
• Thou shalt
monitor CPU
Usage
• Thou shalt
track processes
•Thou shalt
monitor disk
space
•Thou shalt
monitor I/Os
• Thou shalt
monitor
memory
availability
• Thou shalt
track activity
Process Memory Disk
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 7
sacraments
baptism
confession
repentance
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 8
100%	uptime
Ein Minuten
Bitte
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 10
monitoring
resources makes
sense when
tracking physical
system health
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 11
The New Data Center
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 12
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 13
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 14
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 15
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 16
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 17
if you’re deciding
which alarms are
important, you have
too many
Empathetic Monitoring
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 19
start monitoring
business metrics
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 20
revenue
courtesey Leon Fayer of OmniTI (@papa_fire)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 21
user latency
courtesey Leon Fayer of OmniTI (@papa_fire)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 22
courtesey Leon Fayer of OmniTI (@papa_fire)
database performance
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 23
courtesey Leon Fayer of OmniTI (@papa_fire)
decline rate (%)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 24
how can I be a superhero?
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 25
“She doesn’t care whether or
not [the data center is literally
on fire] … just as long as his
ship’s coming in…”
-Cake, Italian Leather Sofa
(lightly interpreted)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 26
have a chat with
your business
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 27
Spoiler Alert
90% of the
time you’ll
be listening
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 28
What	happens	on	a	normal	day?	
What	happens	on	a	good	day?	
What	happens	on	a	bad	day?
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 29
If	you	could	have	insight	
into	one	piece	of	the	
organization,	what	would	
it	be?
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 30
listen	to	your	business
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 31
context is crucial
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 32
new allies become evangelists
New Partners
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 34
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 35
what about marketing?
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 36
(https://blogs.adobe.com/digitalmarketing/wp-content/uploads/2015/10/laxalt-fig1.jpg)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 37
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 38
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 39
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 40
The Chasm
(Moore – Crossing the Chasm 1991)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 41
Reformation
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 43
change conducted poorly will
break your organization
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 44
strict change enforcement
creates unrest
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 45
…harder	than	you	think	
…takes	longer	than	you	expect	
…self-initiated
change is…
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 46
1.Flow
2.Feedback
3.Continuous
Experimentation &
Learning
The Three Ways
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 47
1.Flow
2.Feedback
3.Continuous
Experimentation &
Learning
The Three Ways
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 48
-enables continuous improvement
Amplify Feedback Loops
You Them
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 49
retrospectives
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 50
invite the whole
business
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 51
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 52
Reframe the goal:
we don’t want to find out what
should have happened, we want to
learn how to improve.
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 53
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 54
blame creates subject
matter experts in
information
suppression
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 55
ritual is important
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 56
“Before we begin, I want to remind
everyone that postmortems are
blameless. We’re not here to discuss
what should have or could have
happened because we’re here to
learn and gain new information for
the future.”
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 57
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 58
• VictorOps Blameless Postmortem Guide 

http://bit.ly/2nBAvq2

(https://victorops.com/wp-content/uploads/2014/09/Blameless-Post-Mortems.pdf)
• Chef Postmortem Template - http://bit.ly/2noY4S1

(https://gist.github.com/btm/7cb421f5fe7d1003083a)
• Etsy’s Debriefing Facilitation Guide 

http://etsy.me/2iVP7Bj

(https://extfiles.etsy.com/DebriefingFacilitationGuide.pdf)
Retrospective Resources
The charge
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 60
stop thinking about your
systems’ health and start
thinking about your
business’s health
Be Skeptical of your Traditions
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 61
partner with data-driven
departments in your org
to proselytize the benefits
of business monitoring
Make new evangelists
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 62
build safe environments,
amplify feedback and
foster learning
Safely Reform
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 63
Change the
world
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 64
Change the
world(Or at least your monitoring)
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 65
Thanks!
@CrayZeigh
Aaron Aldrich
http://cageit.us/GalaxZ17
SlideShare:
©		2016	All	Rights	Reserved	CONFIDENTIAL@CrayZeigh 66
67
Boots

More Related Content

More from Aaron Aldrich

DevOpsDays 2018 - Operating Human Systems: MTBF v. MTTR
DevOpsDays 2018 - Operating Human Systems: MTBF v. MTTRDevOpsDays 2018 - Operating Human Systems: MTBF v. MTTR
DevOpsDays 2018 - Operating Human Systems: MTBF v. MTTRAaron Aldrich
 
2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR
2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR
2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTRAaron Aldrich
 
DevOpsDays DC 2017 - Better Living Through Storytelling
DevOpsDays DC 2017 -  Better Living Through StorytellingDevOpsDays DC 2017 -  Better Living Through Storytelling
DevOpsDays DC 2017 - Better Living Through StorytellingAaron Aldrich
 
DevOpsDays TO - Managing Fires
DevOpsDays TO - Managing FiresDevOpsDays TO - Managing Fires
DevOpsDays TO - Managing FiresAaron Aldrich
 
DevOpsDays SEA - Managing Fires: Leadership through Crisis
DevOpsDays SEA - Managing Fires: Leadership through CrisisDevOpsDays SEA - Managing Fires: Leadership through Crisis
DevOpsDays SEA - Managing Fires: Leadership through CrisisAaron Aldrich
 
Better Living Through Storytelling
Better Living Through StorytellingBetter Living Through Storytelling
Better Living Through StorytellingAaron Aldrich
 
DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)
DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)
DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)Aaron Aldrich
 
DOD-OH - The Humane Treatment of Engineers
DOD-OH - The Humane Treatment of EngineersDOD-OH - The Humane Treatment of Engineers
DOD-OH - The Humane Treatment of EngineersAaron Aldrich
 

More from Aaron Aldrich (9)

DevOpsDays 2018 - Operating Human Systems: MTBF v. MTTR
DevOpsDays 2018 - Operating Human Systems: MTBF v. MTTRDevOpsDays 2018 - Operating Human Systems: MTBF v. MTTR
DevOpsDays 2018 - Operating Human Systems: MTBF v. MTTR
 
2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR
2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR
2018-01 DevOpsDays NYC: Operating Human Systems: MTBF v. MTTR
 
DevOpsDays DC 2017 - Better Living Through Storytelling
DevOpsDays DC 2017 -  Better Living Through StorytellingDevOpsDays DC 2017 -  Better Living Through Storytelling
DevOpsDays DC 2017 - Better Living Through Storytelling
 
DevOpsDays TO - Managing Fires
DevOpsDays TO - Managing FiresDevOpsDays TO - Managing Fires
DevOpsDays TO - Managing Fires
 
DevOpsDays SEA - Managing Fires: Leadership through Crisis
DevOpsDays SEA - Managing Fires: Leadership through CrisisDevOpsDays SEA - Managing Fires: Leadership through Crisis
DevOpsDays SEA - Managing Fires: Leadership through Crisis
 
Better Living Through Storytelling
Better Living Through StorytellingBetter Living Through Storytelling
Better Living Through Storytelling
 
DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)
DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)
DevOpsDays Madison 2016 - Humane Treatment of On-Call Engineers (Ignite)
 
DOD-OH - The Humane Treatment of Engineers
DOD-OH - The Humane Treatment of EngineersDOD-OH - The Humane Treatment of Engineers
DOD-OH - The Humane Treatment of Engineers
 
Managing fires
Managing firesManaging fires
Managing fires
 

Recently uploaded

Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 

Recently uploaded (20)

Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 

Skeptics in the Church of Data Getting Evangelical