A simple file monitoring console with camel, cometd and jquery

US DOD guidance on open source surprisingly correct >>

October 27, 2009

A simple file monitoring console with camel, cometd and jquery

Here’s a simple file monitoring tool which uses cometd to push file changes to a webpage.

The server part consists of a groovy script which uses Apache Camel to monitor some files. It then uses Apache Camel to push the lines added to the files to the browser with Cometd. In the webpage a bit of jquery and a jquery comet library is used to add the received lines to the web page.

First the server part. I’m using the Groovy Ivy support to automatically download all the required libraries. This means that you can just start the script from the command line, no need to deploy it into any application server. Using stream:file you can monitor files. The lines appended to the files are put into a message object, LogMessage, and a source name is added so we can color the lines in the browser based on their source. Finally, the message are made available using the comet protocol on port 8082.

#!/home/akoelewijn/programs/groovy-1.6.3/bin/groovy

import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.language.groovy.GroovyRouteBuilder;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.mortbay.util.ajax.JSONPojoConvertor
import org.mortbay.util.ajax.JSON

@Grab(group='org.apache.camel', module='camel-groovy', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-jetty', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-cometd', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-stream', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-xstream', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-core', version='2.0.0')
class SampleRoute extends GroovyRouteBuilder {
  void configure(){
        from("stream:in").to("seda:logmsgs?size=10000&concurrentConsumers=5")
    from("stream:file?fileName=/var/log/messages&scanStream=true&scanStreamDelay=50").
          process(new AddSourceProcessor('file1')).
      to("seda:logmsgs?size=10000&concurrentConsumers=5")
    from("stream:file?fileName=/var/log/syslog&scanStream=true&scanStreamDelay=50").
          process(new AddSourceProcessor('file2')).
      to("seda:logmsgs?size=10000&concurrentConsumers=5")
        from("jetty:http://localhost:8081/msgs").
          process(new AddSourceProcessor('file3')).
          to("seda:logmsgs?size=10000&concurrentConsumers=5")
    from("seda:logmsgs?size=10000&concurrentConsumers=5").
      to("cometd://0.0.0.0:8082/service/logs?resourceBase=.")
  }
}

public class LogMessage implements JSON.Convertible {
  String source
  String message
  public void fromJSON(Map object){}
  public void toJSON(JSON.Output out){
    (new JSONPojoConvertor(LogMessage)).toJSON(this,out)
  }
}

public class AddSourceProcessor implements Processor {
    def sourceName
    public AddSourceProcessor(name){
      sourceName = name
    }
    public void process(Exchange exchange) throws Exception {
        exchange.in.setHeader('srcName',sourceName)
        def msg = new LogMessage(source:sourceName, message: exchange.in.body)
        exchange.in.setBody(msg)
    }
}

def camelCtx = new DefaultCamelContext()
camelCtx.addRoutes(new SampleRoute());
camelCtx.start();

The html page is pretty basic, just loading all the required javascript files:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
  "http://www.w3.org/TR/html4/strict.dtd">
<html>
  <head>
    <script src="http://www.google.com/jsapi"></script>
    <script src="log-console.js"></script> 
    <link type="text/css" href="style.css" 
          rel="stylesheet" media="all"/>
  </head>
  <body>
    <pre>Log output</pre>
  </body>
</html>

The javascript file uses the jquery library hosted by google. When it is loaded two jquery plugins are loaded from googlecode, jquerycomet, and jquery scrollTo. When everything is loaded, a connect is made to the comet endpoint at 8082. The receive function will be called when a message is received on /service/logs. Receive adds a pre element to the body element with the received message.

google.load("jquery", "1");
google.setOnLoadCallback(function() {
  $.getScript('http://flesler-plugins.googlecode.com/files/jquery.scrollTo-1.4.2-min.js');
  $.getScript("http://jquerycomet.googlecode.com/svn/trunk/jquery.comet.js", function(){
    console.log("done loading js");
    $.comet.init("http://localhost:8082/cometd");
    $.comet.subscribe("/service/logs", receive);
  });
});
function receive(message) {
  console.log("message: " + message + ", " + message.data + ", header: " + message.header );
  var msg = eval(message.data);
  console.log("msg: " + msg);
  $("body").append("<pre class='" + msg.source + "'>" + msg.message.replace(/</g,'&lt;').replace(/>/,'&gt;') + "</pre>");
  $("body:last-child").scrollTo('100%',10,{axis:'y'});
}

Finally a bit a css to give the lines different colors based on their source:

body { margin: 0; padding: 0; }
pre {
  font-family: monospace;
  margin: 0; padding: 3px; border-bottom: 1px solid #ddd;
  background: #fff;
  white-space: pre-wrap;
}
pre.file1 { background-color: #f0e0e0; }
pre.file2 { background-color: #f0f0f0; }
pre.file3 { background-color: #ffeeee; }
pre:hover { background-color: #ffeeee; }

Adding remote file monitors should be pretty simple: just add another script which monitors files and pushes the changes to a shared remotely available queue, something like ActiveMq for example. The central groovy script above just needs to have one more source to read message from.

Andrej Koelewijn

A simple file monitoring console with camel, cometd and jquery

Related