Machine Learning component part1

The design choices to be made while building the machine learning server were:
1. Train the model in real time
2. Train the model in batches beforehand and predict in real time
3. Keep samples of data for training. Train and predict in real time

The factors that help decide which approach to use are:
1. Volume of training data and How much time does it take to train the model?
           If the training takes too long, there is no point in real time model building. We would need to build the model apriori. Load the model in real time and use real time streams as test flows.
The machine learning algorithm chosen in this project takes less time to train(~2-5 seconds) , hence we can afford to learn, train and predict in real time.

2. Criticality of accuracy, how much error is reasonable?
           If the accuracy obtained is greater than 85%, its reasonable to go ahead and predict using the model. If not we will need to consider methods to improve the accuracy.

3. How do we improve accuracy of the model? Will feedback be considered?
           If  the training is done offline, we can train the model with the flows it predicted wrong. rebuild the model and load it.
           If the training is done in real time and the accuracy is good enough, we do not have to retrain the model. The real time flows can be used as test flows for prediction.
           If the training is done in real time and the accuracy is bad, then maybe use a different algorithm that fits the data better. The algorithm is time critical. If algorithm cant be changed then use different set of parameters to change the nature of the model.

Some suggestions for the model -
1. Rule based approach - Decision trees, K Nearest Neighbours
2. Neural network with real time feedback - One Shot Learning






Comments

Post a Comment

Popular posts from this blog

Day 12: Master Slave SDN Controller Architecture

In the previous post, I was struggling to figure out why all controllers can install flows in all switches irrespective of the code. Well, today the mystery is getting resolved. I found this book that explains beautifully the various roles that a controller can play. The three roles are: Equal : this happens by default all the equal controllers in a network that are connected to the switch can add and delete flows from it Slave: it has a read-only limitation to the switch openflow tables Master: very similar to the switch, but only 1 master can be connected to the switch at that point of time, and no other flows can intercept the communication between switch and master. I highly recommend you go through the above link to get a better picture. Since the default role of any controller is "Equal", we couldn't really differentiate between the three controller flows yesterday. But one good news from this is that we don't really need extensive coding to assi
Read more

Day 50: Tcpreplay and tcpliveplay approach

Today we shall be exploring how to copy and redirect the OpenFlow packets to more than one controller from the load balancer. The load balancer is anyway sending the OpenFlow packets to one of the slave controllers - c1 or c2. The packets sent to these two controllers needs to be sent to master controller so the master controller can update the new flows added by c1 or c2 on the switches. This ensures synchronization between controllers. There are two implementation methods to achieve the same: Copy all packets generated from the load balancer to the controller side. Only the packets having destination IP address of c1 and c2 can be redirected to the master controller. Thus we can prevent the packets which already are reaching the master controller to be sent as duplicates Copy and redirect packets from the slave controllers - c1 and c2. We need not take care of duplicate packets reaching master controller. This will reduce the time taken to redirect packets. Since the second a
Read more

Day 10: Mininet Simulation of a basic distributed SDN controller architeture

Hi and welcome back! We have not been able to update the blog since we were busy debugging and figuring out!!! We are back to give you an in-detail explanation of how we have simulated a basic distributed SDN controller architecture. Requirement: 2 Hosts Mininet up and running Ryu controller up and running on each 1 D-link switch We have used two linux machines with Ubuntu 16.04 OS running on them.  How to install Mininet? Run the following command on a terminal to install mininet: > sudo apt-get install mininet How to install Ryu Controller? Make sure to install Ryu controller on each of the Linux machines so that they can act as a controller. Run the following commands for a successful installation of Ryu controller: > git clone git://github.com/osrg/ryu.git > cd ryu > sudo apt-get install python-dev python-pip python-setuptools   > sudo pip install . > sudo pip install webob eventlet paramiko routes Set Up the network
Read more