Day 15 : Real time machine learning architecture for securing the network

Hi fellow network enthusiasts!

Today's blog post is dedicated for building a real time machine learning architecture. Why real time?
Well, real time analysis makes the system more robust to attacks, since the analysis incorporates inspecting packets as it flows through the network.

What we need is a scalable architecture that can handle large amount of data! We cant afford to let the machine learning model to crash because the ingested volume was too much. So, in the spirit of building a scalable, robust, secure SDN network architecture, I propose the following model



To put the picture into words, primarily, collect network traffic at the switches and stream it into Spark Streaming Engine using Kafka as the data ingestion tool. Create RDDs (Resilient Distributed Data-set) and process streaming data into appropriate RDDs. After preprocessing, feed the RDDs into Spark Machine Learning Model (based on best accuracy this model will be finalized, in the future blog posts) / a Real time machine learning model that can handle streaming data. The results obtained from the model will be stored in an in-memory database to facilitate faster data storage and retrieval. Retrieving results from Redis will be developed as an API for easier integration with the network architecture.

Things in focus
1. Speed
2. Scalable
3. Robust
4. Secure
5. Real time
6. Easier to integrate into any network.

That's it for today folks! We will be back with more content on how to achieve the proposed model. This is going to be a fun, challenging and exciting journey!

Comments

  1. Ram P RustagiFebruary 2, 2019 at 1:38 AM

    Alternatively, send a periodic summary of traffic stats e.g. every minute. Otherwise, you will be generating too much data and wasting precious network resources. OpenFlow table does keep statistics and same can be periodically sent.

    ReplyDelete

Post a Comment

Popular posts from this blog

Day 12: Master Slave SDN Controller Architecture

In the previous post, I was struggling to figure out why all controllers can install flows in all switches irrespective of the code. Well, today the mystery is getting resolved. I found this book that explains beautifully the various roles that a controller can play. The three roles are: Equal : this happens by default all the equal controllers in a network that are connected to the switch can add and delete flows from it Slave: it has a read-only limitation to the switch openflow tables Master: very similar to the switch, but only 1 master can be connected to the switch at that point of time, and no other flows can intercept the communication between switch and master. I highly recommend you go through the above link to get a better picture. Since the default role of any controller is "Equal", we couldn't really differentiate between the three controller flows yesterday. But one good news from this is that we don't really need extensive coding to assi
Read more

Day 50: Tcpreplay and tcpliveplay approach

Today we shall be exploring how to copy and redirect the OpenFlow packets to more than one controller from the load balancer. The load balancer is anyway sending the OpenFlow packets to one of the slave controllers - c1 or c2. The packets sent to these two controllers needs to be sent to master controller so the master controller can update the new flows added by c1 or c2 on the switches. This ensures synchronization between controllers. There are two implementation methods to achieve the same: Copy all packets generated from the load balancer to the controller side. Only the packets having destination IP address of c1 and c2 can be redirected to the master controller. Thus we can prevent the packets which already are reaching the master controller to be sent as duplicates Copy and redirect packets from the slave controllers - c1 and c2. We need not take care of duplicate packets reaching master controller. This will reduce the time taken to redirect packets. Since the second a
Read more

Day 10: Mininet Simulation of a basic distributed SDN controller architeture

Hi and welcome back! We have not been able to update the blog since we were busy debugging and figuring out!!! We are back to give you an in-detail explanation of how we have simulated a basic distributed SDN controller architecture. Requirement: 2 Hosts Mininet up and running Ryu controller up and running on each 1 D-link switch We have used two linux machines with Ubuntu 16.04 OS running on them.  How to install Mininet? Run the following command on a terminal to install mininet: > sudo apt-get install mininet How to install Ryu Controller? Make sure to install Ryu controller on each of the Linux machines so that they can act as a controller. Run the following commands for a successful installation of Ryu controller: > git clone git://github.com/osrg/ryu.git > cd ryu > sudo apt-get install python-dev python-pip python-setuptools   > sudo pip install . > sudo pip install webob eventlet paramiko routes Set Up the network
Read more