Log Analysis and Security Visualization Workshops
- Information Visualization History
- Visualization Theory
- Data Visualization Tools and Libraries
- Visualization Resources
- Worms in the Mobile World
- Traffic Flow Analyses
- Traffic Characteristic Monitoring
- Firewall Log Analysis
- IDS Signature Analysis
- System Configurations
- User Activity
- Host Activity
- Vulnerability Data
- Web Service Ops Monitoring
- Networking Theory
- Network Traffic Capture
- Traffic Analysis Tools
- Network Traffic Visualization
- Bro IDS
Log Management and SIEM
- Basic Principles
- Logging Standards
- Log Management Tools
- ELK Stack
- Application Logging Guidelines
- Logging as a Service
- Overview of Database Technologies (NoSQL, NewSQL)
- Search Engines (ElasticSearch)
- Hadoop Ecosystem and Distributions (Cloudera, MapR, Hortonworks)
- Spark Ecosystem
- Apache storm
- Big Data Lake
- Data Sources – What you need to know for logging
- Data Analysis and Visualization Linux (DAVIX)
- Log Data Processing
- Data Mining
- Introduction to R
- Splunk introduction
- Data Analysis with splunk
- Packet Capture Analysis in splunk
- Advanced splunk uses (lookups, etc.)
- Introduction to Maltego
- Hands-on lab
As networks become ever more complex, securing them becomes more and more difficult. The solution is visualization. Using today’s state-of- the-art data visualization techniques, you can gain a far deeper understanding of what’s happening on your network right now. You can uncover hidden patterns of data, identify emerging vulnerabilities and attacks, and respond decisively with countermeasures that are far more likely to succeed than conventional methods. The attendees will learn about log analysis, get an overview of visualization, data sources for IT security, and learn how to generate visual representations of IT data. The training is filled with hands-on exercises utilizing the DAVIX live CD.
The audience for this workshop is really broad: Anyone with an interest in data analytics, big data, and visualization. For example, Security Analysts, Security Engineers, Incident Responders, Security Managers, and System Administrators, but also product managers and UI/UX designers.
Content Details for Some Modules
Intro to packet captures, network flows, IDS and firewall data, etc. Each module talks about the data sources first and then discusses a number of tools to look at the data, from wireshark to argus and nfdump, snort, bro, etc.
An overview of big data technologies: key-value stores, search engines, map reduce. We discuss the differences between Hadoop 1 and 2; the different distributions (e.g., CDH), and talk about Spark, which is part of the BDAS stack. We also discuss ElasticSearch which is a search engine used by a number of security related log management solution: logstash, moloch, …
R and Data Science
This modules starts with an intro to data science and the problems we have in security with data science. Then as an example of a data science tool, we’ll introduce R and a few use-cases on how to do security analytics with R. Using ggplot2 and a couple of other things.
Anything about capturing network packets, what information they contain, how they can be captured, and what tools there are to process the captures. We are not only looking at textual tools that are common in analyzing network captures, but will also look at examples of how to visualize network traffic to gain insights quicker. BroIDS is a network-based intrusion detection system that we will be using to look at some traffic to extract intelligence from it.
This section starts with a look at different data sources. We will have already covered packet captures in the network forensics module. Here we talk about network flows, IDS and firewall data, threat feeds, etc. Each module talks about the data sources first and then discusses a number of tools to look at the data, from argus and nfdump to snort and suricata. We will then have a look at how to process these data sources to leverage them for visualization. The section ends with an introduction to data science and the problems we have in security with data science. Finally, as an example of a data science tool, we’ll introduce R and a few use- cases on how to do security analytics with R; using ggplot2 and a couple of other libraries.
Log Management and SIEM
We look at a number of log management and security information and event management (SIEM) principles. From correlation to aggregation and taxonomies. We have a look at a couple of the tools in the log management space, such as the ELK stack and Moloch, which both use ElasticSearch as a backend to provide log management capabilities.
Anything from principles by Edward Tufte and Steven Few to classical visualization theory to talking about all the different charts, what to do and not to do, up to parallel coordinates, tree maps, heat maps, etc. We are using a number of exercises to emphasize and deepen the knowledge around these principles.
“Materials are generated from real-world experiences hence all things learned are really practical and useful.”
“The class was pretty intensive (with loads of stuff – theory and practical).”
“Certainly, it changed my perception on doing log analysis traditionally and paved new ways to work on log analysis.”
“Very informative in understanding core concepts with Security Visualization.”
“Probably the most useful speaker of the day. He provided very good information on how to visualize data. I would like to see him come back in a workshop type format where we could bring the logs that our applications actually create and he could help us filter them and put it in a useful format.
- Visual Analytics – Delivering Actionable Security Intelligence, BlackHat, USA, August 2016
- Visual Analytics – Delivering Actionable Security Intelligence, Bell, Toronto, Canada, November 2015.
- Visual Analytics – Delivering Actionable Security Intelligence, BlackHat, USA, August 2015
- Visual Analytics – Delivering Actionable Security Intelligence, New York Stock Exchange, Atlanta, USA, March 2015
- Visual Analytics – Delivering Actionable Security Intelligence, Kudelski, Switzerland, March 2015
- Visual Analytics – Delivering Actionable Security Intelligence, BlackHat, Europe, October 2014
- Visual Security Intelligence, Underground Economy Conference, Romania, September 2014
- Visual Analytics – Delivering Actionable Security Intelligence, BlackHat, USA, August 2014
- Visual Analytics Workshop, Nationwide, Columbus, OH, July 2014
- Visual Analytics Workshop, WorldBank, Washington, D.C., July 2014
- Visual Analytics – Delivering Actionable Security Intelligence, BlackHat, Seattle, December 2013
- Network Forensics and Security Visualization, Dubai, UAE, November 2013
- Visual Analytics – Delivering Actionable Security Intelligence, BlackHat, Las Vegas, July 2013
- Visualization Workshop for Norman Security, San Diego, California, March 2013
- Log Analysis and Visualization Workshop, Doha, Qatar, February 2013
- Log Analysis and Visualization Workshop, Dubai, UAE, November 2012
- Visual Analytics Workshop, Nordic Security Conference, Reykjavik, Iceland, August 2012
- Information Visualization – Bridging the Gap Between Tufte and Firewalls, Annual Honeynet Workshop, Palo Alto, USA, March 2012
- Log Analysis and Visualization Workshop for Trend Micro, Taipei, Taiwan, September 2011
- Log Analysis and Visualization Workshop, Taipei, Taiwan, March 2011
- Log Analysis Martial Arts, HoneyNet Annual Workshop, Mexico City, April 2010
- Advanced Splunk and Visualization Training, Singapore, January 2010
- Visualization and Logging Workshop, NorCERT, Oslo, Norway, October 2009.
- “Data Analysis And Visualization” Workshop, HoneyNet Alliance, Kuala Lumpur, Malaysia, February 2009.
- “Security Visualization Research” Workshop, Colloquium for Information Systems Security Education (CISSE), Seattle, June 2009.
- “Applied Security Visualization” Workshop, IS Summit 2008, Hong Kong, November 2008.
- “DAVIX Workshop on Visualization”, DefCon, Las Vegas, August, 2008.
- Applied Security Visualization” Workshop, First Conference 2008, Vancouver, June 2008.