A simple file monitoring console with camel, cometd and jquery
October 27th, 2009 | Published in oss, soa, web | 5 Comments
Here’s a simple file monitoring tool which uses cometd to push file changes to a webpage.
The server part consists of a groovy script which uses Apache Camel to monitor some files. It then uses Apache Camel to push the lines added to the files to the browser with Cometd. In the webpage a bit of jquery and a jquery comet library is used to add the received lines to the web page.
First the server part. I’m using the Groovy Ivy support to automatically download all the required libraries. This means that you can just start the script from the command line, no need to deploy it into any application server. Using stream:file you can monitor files. The lines appended to the files are put into a message object, LogMessage, and a source name is added so we can color the lines in the browser based on their source. Finally, the message are made available using the comet protocol on port 8082.
#!/home/akoelewijn/programs/groovy-1.6.3/bin/groovy
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.language.groovy.GroovyRouteBuilder;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.mortbay.util.ajax.JSONPojoConvertor
import org.mortbay.util.ajax.JSON
@Grab(group='org.apache.camel', module='camel-groovy', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-jetty', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-cometd', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-stream', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-xstream', version='2.0.0')
@Grab(group='org.apache.camel', module='camel-core', version='2.0.0')
class SampleRoute extends GroovyRouteBuilder {
void configure(){
from("stream:in").to("seda:logmsgs?size=10000&concurrentConsumers=5")
from("stream:file?fileName=/var/log/messages&scanStream=true&scanStreamDelay=50").
process(new AddSourceProcessor('file1')).
to("seda:logmsgs?size=10000&concurrentConsumers=5")
from("stream:file?fileName=/var/log/syslog&scanStream=true&scanStreamDelay=50").
process(new AddSourceProcessor('file2')).
to("seda:logmsgs?size=10000&concurrentConsumers=5")
from("jetty:http://localhost:8081/msgs").
process(new AddSourceProcessor('file3')).
to("seda:logmsgs?size=10000&concurrentConsumers=5")
from("seda:logmsgs?size=10000&concurrentConsumers=5").
to("cometd://0.0.0.0:8082/service/logs?resourceBase=.")
}
}
public class LogMessage implements JSON.Convertible {
String source
String message
public void fromJSON(Map object){}
public void toJSON(JSON.Output out){
(new JSONPojoConvertor(LogMessage)).toJSON(this,out)
}
}
public class AddSourceProcessor implements Processor {
def sourceName
public AddSourceProcessor(name){
sourceName = name
}
public void process(Exchange exchange) throws Exception {
exchange.in.setHeader('srcName',sourceName)
def msg = new LogMessage(source:sourceName, message: exchange.in.body)
exchange.in.setBody(msg)
}
}
def camelCtx = new DefaultCamelContext()
camelCtx.addRoutes(new SampleRoute());
camelCtx.start();
The html page is pretty basic, just loading all the required javascript files:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<script src="http://www.google.com/jsapi"></script>
<script src="log-console.js"></script>
<link type="text/css" href="style.css"
rel="stylesheet" media="all"/>
</head>
<body>
<pre>Log output</pre>
</body>
</html>
The javascript file uses the jquery library hosted by google. When it is loaded two jquery plugins are loaded from googlecode, jquerycomet, and jquery scrollTo. When everything is loaded, a connect is made to the comet endpoint at 8082. The receive function will be called when a message is received on /service/logs. Receive adds a pre element to the body element with the received message.
google.load("jquery", "1");
google.setOnLoadCallback(function() {
$.getScript('http://flesler-plugins.googlecode.com/files/jquery.scrollTo-1.4.2-min.js');
$.getScript("http://jquerycomet.googlecode.com/svn/trunk/jquery.comet.js", function(){
console.log("done loading js");
$.comet.init("http://localhost:8082/cometd");
$.comet.subscribe("/service/logs", receive);
});
});
function receive(message) {
console.log("message: " + message + ", " + message.data + ", header: " + message.header );
var msg = eval(message.data);
console.log("msg: " + msg);
$("body").append("<pre class='" + msg.source + "'>" + msg.message.replace(/</g,'\<').replace(/>/,'>') + "</pre>");
$("body:last-child").scrollTo('100%',10,{axis:'y'});
}
Finally a bit a css to give the lines different colors based on their source:
body { margin: 0; padding: 0; }
pre {
font-family: monospace;
margin: 0; padding: 3px; border-bottom: 1px solid #ddd;
background: #fff;
white-space: pre-wrap;
}
pre.file1 { background-color: #f0e0e0; }
pre.file2 { background-color: #f0f0f0; }
pre.file3 { background-color: #ffeeee; }
pre:hover { background-color: #ffeeee; }
Adding remote file monitors should be pretty simple: just add another script which monitors files and pushes the changes to a shared remotely available queue, something like ActiveMq for example. The central groovy script above just needs to have one more source to read message from.
October 28th, 2009 at 5:10 pm (#)
This is really cool. And I’ve tried it, but it doesn’t seem to work.
I ended up adding a Jetty dependency and a Cometd dep to get it to at least start:
@Grab(group=’org.mortbay.jetty’, module=’jetty’, version=’6.1.14′)
@Grab(group=’org.mortbay.jetty’, module=’cometd-jetty’, version=’6.1.14′)
Also, the Javascript REQUIRES Firebug open with the console on.
Even then, no messages were ever posted. Nothing ever appears from cometd no matter what I tried.
Help?
October 28th, 2009 at 5:23 pm (#)
Nevermind. It seems that actually adding those dependencies broke the whole thing with a class cast that I couldn’t see until I turned on Camel tracing.
This is really sweet.
Thanks!
November 17th, 2009 at 11:54 am (#)
Fantastic nice example.
I have added a link to it from the Camel articles webpage.
November 18th, 2009 at 5:46 pm (#)
Just tried this out, it’s very cool.
Couple of questions though..
1) It only reads line by line, so it only updates a line every “camel scan interval” msecs. Do you know how to make it slurp up all the lines since the last scan?
2) It always starts from the top of the file. If I go to the “log viewer” after the app has started running, it takes a LONG time to chew through the file, line by line. Did you work around this at all?
June 13th, 2010 at 7:00 pm (#)
Excellent nice little demonstration!
But I took the liberty to modify the code so that it would actually work on my system (JR Boyens seems to have the same problem). I've uploaded the code to http://my-gnuradio-toolset.googlecode.com/svn/t...
(this link is actually clickable)
I have upgraded the libraries, used json-lib as a bridge between java and cometd, made it actually run in Google Chrome and stripped out all firebug console.log messages so that the code will run even without firebug.
Marcel Maatkamp