Intel IT is building a modern, scalable cyber intelligence platform (CIP), using an Apache Kafka data pipeline based on Confluent Platform.
Advanced cyber threats continue to increase in frequency and sophistication, threatening computing environments and impacting businesses’ ability to grow. More than ever, large enterprises must invest in eﬀective information security, using technologies that improve detection and response times. Apache Kafka is the foundation of our CIP architecture. We achieve economies of scale as we acquire data once and consume it many times. Simplified connection of data sources helps reduce our technical debt, while filtering data helps reduce costs to downstream systems. We selected Conﬂuent as our Kafka partner to help with the typical pain points and risks that accompany pure open source software deployments. We immediately started to realize benefits of Kafka by achieving economies of scale. We can now acquire data once and consume it many times. Other benefits like reduction of technical debt by eliminating legacy point-to-point and custom connectors soon followed. And through data filtering, we have reduced the cost of ingest and storage in a variety of downstream systems. Conﬂuent provides enterprise capabilities beyond open source Apache Kafka that make it more powerful, manageable, maintainable, and easy to use. Some of these key capabilities are:
Conﬂuent Control Center for monitoring our Kafka clusters
Replicator and Multi-Region Clusters for multiple data center design and fault tolerance
Auto Data Balancer for cluster rebalancing and easy scale out, when needed
Enterprise-grade security including Role-based Access Control and audit logs
100+ pre-built connectors to common sources and sinks
Technical Account Manager for feature prioritization and guidance
Architecture and design partnership via a Conﬂuent Resident
Architect and “health check” engagements
Ongoing training available in real time (for example, webinars) and on demand
Availability of 24x7x365 support
Partnership for collaborating on new capabilities, such as security features
Our CIP is based on Intel® Xeon® Scalable processors and Intel® solid state drives (SSDs). Intel vice president and Chief Information Security Officer, Brent Conran, explains, “Kafka helps us produce contextually rich data for both IT and our business units. Kafka also enables us to deploy more advanced techniques in-stream, such as machine-learning models that analyze data and produce new insights. This helps us reduce mean time to detect and respond; it also helps decrease the need for human touch. The most transformational capability of Kafka for cybersecurity is in-stream processing. The ability to operate on data as it is produced helps security responders improve detection techniques and response times. It also enables us to develop and deploy more advanced techniques, such as machine-learning models that perform in-stream processing, which can identify threats in near real time. These advances are helping us achieve our mission to “make it safe for Intel to go fast.” Using industry-leading technologies that provide a modern scalable architecture enables us to continue to transform well into the future.