Day 62:Building the storm topology

Hey guys!

Finally finished building the basic storm topology for extracting the following features. It took me 2.5 weeks to finish integrating kafka with and to build the topology.
The features extracted so far,
1. Total Inflow Packets
2. Average Number of inflow packets (Total/Number of flows)
3. Total Number of flows
4. Total Packet length
5. Average packet length (Total/Number of packets)

What next?
I want to set up an ML model for analyzing the features in real time. 

What are the hiccups along the way?
1. Are the input features enough? What more can be extracted?
2. Outflow features cant be utilized directly since there is a cyclic dependency with the outflow features. 
3. Which ML model to build such that it doesn't slowdown the decision process? Should it be supervised or unsupervised?
4. How to improve accuracy once the model is built?

I am reading up on existing real time ML models for networks. I shall be updating the solution soon on the blog with useful links in the next post.

Comments

Post a Comment

Popular posts from this blog

Day 12: Master Slave SDN Controller Architecture

In the previous post, I was struggling to figure out why all controllers can install flows in all switches irrespective of the code. Well, today the mystery is getting resolved. I found this book that explains beautifully the various roles that a controller can play. The three roles are: Equal : this happens by default all the equal controllers in a network that are connected to the switch can add and delete flows from it Slave: it has a read-only limitation to the switch openflow tables Master: very similar to the switch, but only 1 master can be connected to the switch at that point of time, and no other flows can intercept the communication between switch and master. I highly recommend you go through the above link to get a better picture. Since the default role of any controller is "Equal", we couldn't really differentiate between the three controller flows yesterday. But one good news from this is that we don't really need extensive coding to assi
Read more

Day 50: Tcpreplay and tcpliveplay approach

Today we shall be exploring how to copy and redirect the OpenFlow packets to more than one controller from the load balancer. The load balancer is anyway sending the OpenFlow packets to one of the slave controllers - c1 or c2. The packets sent to these two controllers needs to be sent to master controller so the master controller can update the new flows added by c1 or c2 on the switches. This ensures synchronization between controllers. There are two implementation methods to achieve the same: Copy all packets generated from the load balancer to the controller side. Only the packets having destination IP address of c1 and c2 can be redirected to the master controller. Thus we can prevent the packets which already are reaching the master controller to be sent as duplicates Copy and redirect packets from the slave controllers - c1 and c2. We need not take care of duplicate packets reaching master controller. This will reduce the time taken to redirect packets. Since the second a
Read more

Day 10: Mininet Simulation of a basic distributed SDN controller architeture

Hi and welcome back! We have not been able to update the blog since we were busy debugging and figuring out!!! We are back to give you an in-detail explanation of how we have simulated a basic distributed SDN controller architecture. Requirement: 2 Hosts Mininet up and running Ryu controller up and running on each 1 D-link switch We have used two linux machines with Ubuntu 16.04 OS running on them.  How to install Mininet? Run the following command on a terminal to install mininet: > sudo apt-get install mininet How to install Ryu Controller? Make sure to install Ryu controller on each of the Linux machines so that they can act as a controller. Run the following commands for a successful installation of Ryu controller: > git clone git://github.com/osrg/ryu.git > cd ryu > sudo apt-get install python-dev python-pip python-setuptools   > sudo pip install . > sudo pip install webob eventlet paramiko routes Set Up the network
Read more