Thursday 13 September 2012

Hacking Gaia

Mozilla's Firefox OS also known as b2g is an awesome feat from the community towards its goal to provide platform for innovation and creativity. It was a necessary step considering the growth of mobile market. Smart phone is a front which lacks an open and powerful OS, which let's you hack into every aspect of the user experience(There is Android but it's not that easy to customize it.). In b2g, just by changing a CSS file or by modifying a JavaScript file, you can change the look and functionality of your phone dramatically.

B2g uses Gaia for it's UI. Gaia is all Html5,CSS and JavaScript. It can even run on a desktop browser(except the ). The whole OS is far from first release but you can start creating apps for it right now and try them on emulator, your desktop browser(for development purpose) or on your phone.

There are two steps for building  b2g and start working on an app. I have tried these steps on Linux and works for an emulator.

Building B2G:

If you don't want the pain of sitting around your laptop for hours, waiting for the compilation to finish, you can directly download latest version of b2g from here.
If you still want to compile it by yourself then go ahead else skip to next part.

Getting the source code from mozilla-central:

 hg clone http://hg.mozilla.org/mozilla-central src  
 If you encounter an 'Command not found hg' you need to install Mercurial: https://developer.mozilla.org/en/Installing_Mercurial  

Update the code:
 cd src  
 hg pull && hg update  

Create a '.mozconfig' file in mozilla-central source directory and paste following in the file. Also read comments and act accordingly.

  mk_add_options MOZ_OBJDIR=../build   
  #Here put NUMBER_OF_CORES_IN_YOUR_PROCESSOR+1 in place of '9'  
  mk_add_options MOZ_MAKE_FLAGS="-j9 -s"  
  ac_add_options --enable-application=b2g   
  ac_add_options --disable-libjpeg-turbo   
  # This option is required if you want to be able to run Gaia's tests   
  ac_add_options --enable-tests   
  # turn on mozTelephony/mozSms interfaces   
  # Only turn this line on if you actually have a dev phone   
  # you want to forward to. If you get crashes at startup,   
  # make sure this line is commented.   
  #ac_add_options --enable-b2g-ril

Now, build:

 make -f client.mk build  
If you face any errors or difficulty in this step, refer this for required library installation and other difficulties, but DO NOT change the '.mozconfig' file.

Now you can go and have some coffee or even a nap because this is going to take long.

Building Gaia Profile:

First, clone repository from Github and update it:

 git clone https://github.com/mozilla-b2g/gaia gaia  
 cd gaia  
 git fetch  
 git merge origin/master  

Compile it:
 make  

Now, if you have downloaded nightly build of b2g then run:

 /path/to/b2g -profile profile  
else:
 ../build/dist/bin/b2g -profile profile  

 


All the apps are in /gaia/apps directory.

Steps to add a new app:

To create and test a new app on the emulator, create a new directory under apps directory.
Add your Html,CSS and JavaScript code in the directory and create a 'manifest.webapp' file with necessary information about the app. You can checkout  pre-installed apps manifest files for reference.
After doing that, append your app details in the file gaia/webapps.js. Like if your app name is 'HelloWorld', then you will make a directory named 'helloworld' in gaia/apps directory and put all you app code there.
Now create a 'manifest.webapp' file in 'helloworld' directory and fill in necessary details like app name, developer name, orientation, language localisation etc.
Now append following in webapps.js file where all the apps are added to a js object:

 "helloworld": {  
   "origin": rootPath.replace('%%%', 'helloworld'),  
   "installOrigin": "http://helloworld.localhost.org:8080",  
   "receipt": null,  
   "installTime": 132333986000,  
   "manifestURL": "http://helloworld.localhost.org:8080/manifest.webapp"  
  }  

Now run following in gaia directory:

 make install-gaia
/path/to/b2g -profile profile

Now you can run your app in emulator and check it out.
You can find more details about running Gaia on phone or on a browser here.

Monday 20 August 2012

Networking Dashboard v01

Networking Dashboard is a tool that shows internals of networking of Firefox. It's an add-on that I have made as part of GSoC project and can be used by typing 'about:networking-dashboard' in URL bar after installing this add-on. It shows current open connections with host names, how many of them are active/idle, if they are using SPDY or not, if they are using SSL or not, DNS cache entries, total data sent and received, data related to WebSocket connections etc. I have blogged about it in detail in my previous post.
Following are some screen shots which might give you some idea about the tool.
Timeline shows graphical view of total data sent and received after starting the add-on.
One feature that I couldn't implement as part of add-on is header window which shows sent and received headers of the packets Http connections.

I have used jQuery and flot - a pure Javascript plotting library for jQuery -  for building the add-on. I can make the add-on available but it won't work because back end part of the code hasn't been migrated to mozilla-central yet.

Http Connections

Socket Connections

DNS cache entries

WebSocket connections


Timeline showing graph of total data sent and received



Monday 13 August 2012

GSoC update

So far I have finally finished implementing back-end interfaces- nsIHttpConnectionLog.idl, nsiWebSocketConnectionLog.idl, nsISocketLog.idl and nsIDNSEntries.idl. for Networking Dashboard.
Now, only thing remains is a UI for the add-on using these interfaces. It's very important part of the project since it will make sense of the json strings returned by these interfaces. Without a UI they are merely strings of data. I will probably be using jquery UI for this. It will be a tabular UI, a tab per interface/feature. Implementation of UI will conclude my GSoC project.

Thursday 12 July 2012

Update of GSoC Till Midterm Evaluation

I have been working on Networking Dashboard for quite a while now and I think the back end part of the project is nearly done. It involves porting out information related to connections of different protocols(HTTP,WebSocket,etc.) through different interfaces.
This blog is an update on how much has been done so far and what am I going to get done in future.

What has been done so far?
So far I have implemented four interfaces. Every interface returns a JSON string containing different information in different fields.

1) nsIHttpConnectionLog.idl :- This interface returns following information related to current HTTP connections in form of a JSON string,

host : Host name of the HTTP connection
- port : port no. of the connection
- spdy : If connection using spdy or not
- active : rtt(Round Trip Time) and ttl(Time To Live) of active connections in respective fields
- idle : rtt and ttl of idle connections in respective fields


The connection data has been collected from nsHttpConnectionMgr.h where data related to http connection is maintained in form of a hash table.
Following is the example of a JSON string which is returned from this interface after opening www.google.com followed by www.twitter.com:
 {  
   "host" : ["mail.google.com","www.google-analytics.com","accounts.google.com","api.twitter.com","ocsp.thawte.com","localhost","accounts.youtube.com","a0.twimg.com","www.mozilla.org","ssl.gstatic.com","ssl.google-analytics.com","gmail.com","scribe.twitter.com","twitter.com","mail.google.com"],  
   "port" : [443,80,443,80,80,80,443,80,80,443,443,80,80,80,80],  
   "active" : [{  
     "rtt" : [0],  
     "ttl" : [145]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [143]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [144]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [144]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [145]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   }],  
   "idle" : [{  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [95]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [86]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [79]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0,0,0,0],  
     "ttl" : [89,86,85,85]  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [],  
     "ttl" : []  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [75]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [86]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [85]  
   },  
   {  
     "rtt" : [0],  
     "ttl" : [75]  
   }],  
   "spdy" : [true,false,true,false,false,false,true,false,false,true,true,false,false,false,false]  
 }  
Empty ttl and rtt array means that that there is no connection with the host.
You can have a better look at it using any online JSON editor. Sorry for posting such long JSON. It looks ugly if I don't format it this way.


2) nsIWebSocketConnectionLog.idl :- This interface returns following information related to current WebSocket connections in form of a JSON string,

- hostport : Host name and port of the WebSocket connection
- msgsent : No. of messages sent
- msgreceived : No. of messages received
- encrypted : If the connection secure or not

There is no maintained data available for WebSocket. So all the data related to the connections is maintained at the implementation of the interface(in file nsWebSocketConnectionLog.h) Every time a new connection is made with new host or an old connection is closed, an entry is made or removed from the database if an instance of the interface has been created.

I found websites using WebSocket from www.websocket.org.
Here is one example of JSON string returned from the interface after opening http://rumpetroll.com/ and wandering around a little bit and after that http://www.kaazing.me :

 {  
   "hostport" : ["rumpetroll.motherfrog.com:8180","demo.kaazing.com"],  
   "msgsent" : [76,8],  
   "msgreceived" : [99,223],  
   "encrypted" : [false,true]  
 }  
3) nsISocketLog.idlThis interface returns following information related to current Socket connections in form of a JSON string,

- host : peer host ip
- port : peer host port
- tcp : if the connection is TCP or not

Data related to socket is collected from nsSocketTransportService2.h.
The data returned after opening www.google.com and http://twitter.com/
in above example is:
 {  
   "host" : ["74.125.236.102","199.59.149.243","124.124.201.161","199.59.150.41","124.124.201.161","124.124.201.161","124.124.201.161","199.59.150.7","74.125.236.98","74.125.236.117","173.194.36.30","74.125.236.111","209.85.175.84","74.125.236.118","199.7.51.72","74.125.236.117"],  
   "port" : [20480,20480,20480,20480,20480,20480,20480,20480,47873,47873,47873,47873,47873,20480,20480,20480],  
   "tcp" : [true,true,true,true,true,true,true,true,false,false,false,false,false,true,true,true]  
 }  
This data is not much I am hoping to add some more fields(like total data sent and received) to this interface.
4) nsDNSEntries.idl - This interface returns cached entries in DNS cache with  
    following fields,

- hostname : name of the host
- hostaddr : ip address of the host
- family : ip address family(ipv4 or ipv6)
- expiration : time when the cache entry expires(in minutes)

Well, this interface is near completion(believe me,it is). One little difficulty is stopping me from posting it's output here. May be I post it in the comment after it's done. It's going to contain above fields.

This concludes what I have completed so far.
I have been maintaining a mozilla-central repository at bitbucket.org and committing to my updates to branch 'Networking Dashboard'. You can have a look at it if you want. To test these interfaces, i have made a temporary addon using jetpack without any GUI.

What am I going to do next?
First of all I am going to finish the DNS interface.
Since basic interfaces have been implemented, I can now start working on GUI part of the addon. On the way of doing that, I might need to make differences to the interfaces or add some new data.
I also want to include time stamps, size of data sent/received to above interfaces of different protocols. These two feature are very important to the addon. I will work on them after I am done giving a face to the 'Networking Dashboard' addon.

This was an overview of my GSoC work so far. I have learnt a lot of new things about networking and Mozilla internals throughout this period. Thanks for reading. I hope you download my addon in future :)

Saturday 16 June 2012

Update on GSoC (May 21 - May 13)

I have ported out connection data related to protocols like HTTP,WebSocket and also related to sockets to javascript.

Now I data related to these protocols can be accessed through new interfaces.
Data related to different protocol in particular are -

HTTP -            Host name,  Port no., No. of active connections, No. of idle
                        connections, if connection is using SPDY or not (In future - socket
                        identifiers for particular connections, RTT for connections, TTL etc.)
WebSocket -   Host name, Port no. (In future - No of messages sent so far)

Socket -          Socket identifier and packets sent or received via that socket can be 
                        seen. (Not perfect right now.)

We can search for different HTTP connections for their packets in Socket data via identifiers.
Next, I might add a new interface over all of these interface which combines all of these data and returns a single JSON string. We can also do some calculations using these data.


Sunday 20 May 2012

Update on GSoC (May 7 - May 20)

Though the coding period starts from 21st May, I have started working on new IDL as part of my project.
I started working from 7th May. My first try is to get Http connection data like- hostname, port, No. of active connections, No. of idle connections. In first week I was getting my head around how to create a new idl, xpcom module, etc. Later on I came to know that I don't have to create new module, I can add new idl to Network module. So, now I have created my new idl and implemented it in c++. Only thing lest is to test it and see if I am getting Http connection data in a javascript or not. But it's not that easy. It seams that firefox doesn't allow use of UniversalXPConnect in a normal javascript. So now I have two options, either use jsctypes(which will take me time to understand) or create an add on to access the idl. I am going with the latter. Right now I am facing some difficulties in developing new add on with a nightly build but I will get over it.

Tuesday 24 April 2012

GSoC 2012 - Networking Dashboard

My proposal for 'Networking Dashboard' for 'Mozilla' got selected for GSoC 2012 today :).
I have been a contributor to Mozilla for a long time now. Till now mostly it was bug fixes.
But now I will be working on something real for Mozilla. My project is to make a tool for developers which allows them to have an insight of Mozilla networking. Google chrome already has something similar - "chrome://net-internals". This tool will be implemented as an add-on. The link to my proposal is here. It has a description about what functionality will be implemented. I wish I have a link to the add-on soon. I might blog about my experience with necko and about my progress in future. Stay tuned.

Fun with Flotype

I came across this new start up Flotype through a blog at hacker news.
It provides cross-language services to expose data and realtime updates to client devices and browsers.
They have simple APIs which can be used right away.  So I created a new account and started playing with the beta version. My first idea is to setup the server on my laptop and client on my android phone. So whenever server receives the message, it makes a system call for the message. That way I could use my laptops terminal remotely. Since it's available only in Javascript,Java,Ruby and Python, I chose Java.
First of all I created a server on my laptop:
 import com.flotype.bridge.*;  
 class ChatObj implements BridgeObject {  
   public void message(String sender, String message) {  
     System.out.println(sender + ":" + message);  
     Process execAcrobatPrs=null;  
     try{  
       execAcrobatPrs = Runtime.getRuntime().exec(new String[]{"sh", "-c",message});  
    }  
    catch(Exception ex)  
    {  
              System.out.println(ex);  
       if(execAcrobatPrs!=null)  
       {  
          execAcrobatPrs.destroy();  
       }  
       ex.printStackTrace();  
       System.exit(-1);  
    }  
   }  
 }  
 public class AuthServer {  
   public static void main(String[] args) throws Exception {  
     Bridge bridge = new Bridge()  
     .setApiKey("myprivatekey");  
     bridge.joinChannel("channel-name", new ChatObj());  
     bridge.connect();  
   }  
 }  
and the client for android:
 package com.jiten;  
 import java.io.IOException;  
 import com.flotype.bridge.*;  
 import android.app.Activity;  
 import android.os.Bundle;  
 import android.util.Log;  
 import android.view.View;  
 import android.view.View.OnClickListener;  
 import android.widget.Button;  
 import android.widget.EditText;  
 interface RemoteChat extends BridgeRemoteObject {  
   public void message(String sender, String message);  
 }  
 public class MessageSenderActivity extends Activity implements OnClickListener{  
   /** Called when the activity is first created. */  
   @Override  
   public void onCreate(Bundle savedInstanceState) {  
     super.onCreate(savedInstanceState);  
     setContentView(R.layout.main);  
     Button b = (Button) findViewById(R.id.button1);  
     b.setOnClickListener((android.view.View.OnClickListener) this);  
   }  
      @Override  
      public void onClick(View arg0) {  
           if(arg0.getId() == R.id.button1) {  
                EditText t = (EditText) findViewById(R.id.editText1);  
                Bridge bridge = new Bridge().setApiKey("myprivatekey");  
          RemoteChat channel = bridge.getChannel("channel-name", RemoteChat.class);  
          channel.message("Jiten", t.getText().toString());  
          try {  
                     bridge.connect();  
                } catch (IOException e) {  
                     // TODO Auto-generated catch block  
                     e.printStackTrace();  
                }  
           }  
      }  
 }  


That's it. Now I can run any terminal command from my laptop through my android phone.
This still is not interactive so it's of no use for commands like 'ls' but you can run applications in which you don't need interaction.
Well, I can't think of any important use for this but it was fun starting 'Catalyst' in my laptop through a command in my mobile phone. In future I might improve it for interactivity. 



Wednesday 4 January 2012

Electives!!!

"Which electives are you going to take?" - asked one of my friends. This was the 5th time today someone asked me about my electives this semester.I hadn't even thought this much over my college selection.
              Sem 6 - probably the most feared(or hyped) semester here at DA-IICT. It is said that if you survived 3rd and 6th semester here, then you can survive DA-IICT. The main source of this fear is SEN(Software Engineering) - the course in which you have to develop a software throughout the semester from scratch in a team of 9-10 students not of your choice. It doesn't stop there. There are one and half hour lectures and 3 hour of lab and exams too and evaluation process in which you have to prioritise your team mates at the end of the  semester in order of the efforts he/she has put in the process. This is just too much. You have to screw all other subjects in order to excel in this only one subject which is forced on us. God knows what is to come.
First lecture - Every one had heard rumours about the course and wondering what bombs of surprise the instructor had in his mind. He began spelling his rules, going through the slides one-by-one and with every change of the slide, there were expressions like  'oh shit!' or 'oh fuck!' and people just staring each other's faces in despair. Well, I just wanted to get out of there.

Back to elective. You have to follow some rules or prerequisites in order to graduate. Here are rough rules:
1) You must finish 143 credits  except the credits of pass fail courses -> mine - 109.5 so far
2) You must have taken 2 or more Group Electives -> done.
    If you take more than 2 GE, it will be converted to technical elective.
3) You must have taken 2 or more Science Electives - > 0
4) You must have taken at least 1 and at most 3(or 4 not sure) Open Elective->done(thank god :) )

Here are the slots -



Since  I hadn't taken any SE in last semester and had taken 2 GE instead, I shall take one SE in this semester. So, my choices are
Slot 1 - Introduction to Information Retrieval
Slot 2 - Artificial neural networks and Fuzzy systems
Slot 3 - Operating System(Though I wanted to take ESP, I have to take this course              because it is necessity for an IT student to have on his transcript )
Slot 4 - Coding Theory
Slot 5 - Digital Image Processing

             And last but definitely not the least
Slot 6 - SEN (Software Engineering)


Since there are many electives to choose from, the whole batch will be divided and the lectures will be in small CEP rooms which I think is a good thing because I have always hated the huge Lecture Theatre in which I can't concentrate on what the instructor is teaching(It's a weird thing). I hope I will be able to pay more attention in the lecture in small class rooms and also have improvement on my grades.

(This is my first blog ever. I hope there are not much grammatical mistakes and I am lazy to recheck the post for mistakes)