Drip..
Thursday, May 01, 2008
Water droplets - dipping a toe in macro photography
he he - so I have a canon S3 IS - got it last year since it allows enough manual control while also having family friendly thingies like video :). Also, with the chdk hack, the S3 IS is good enough for me to experiment.
So, one of these long time itches has been to take a water droplet splash - you know, the immensely close up snaps where you see a single drop splashing...
Here's the snaps after two evenings of trial and error (mostly errors though) - feeling quite smug myself :)
So, one of these long time itches has been to take a water droplet splash - you know, the immensely close up snaps where you see a single drop splashing...
Here's the snaps after two evenings of trial and error (mostly errors though) - feeling quite smug myself :)
Wednesday, April 02, 2008
Free subversion hosting - What's the best?
All - I've just signed up for an Assembla account - these folks provide free subversion hosting with a 500 meg space and unlimited spaces.
Will see how it goes.
Will see how it goes.
Firefox 3 beta 5 released. Yahoo Mail is still broken.
Firefox 3 Beta 5 release today. Release notes and downloads here.
Installed it as soon as I got to know today morning and the first thing to check was whether Yahoo Mail still crashed. Initially, Yahoo Mail seemed to work alright for all of 50 seconds - quickly moving over items in inbox caused Firefox to crash :-(
Guess will wait for some more time. I'm sure there's a bug report somewhere on this - Yahoo mail was broken on Beta 2, got fixed in Beta 3, then was broken in Beta 4 and is still broken on Beta 5.
Will wait for it to be fixed - Any idea if this is a firefox issue or a Yahoo! issue? Seems odd that script can cause the browser to crash so badly.
Installed it as soon as I got to know today morning and the first thing to check was whether Yahoo Mail still crashed. Initially, Yahoo Mail seemed to work alright for all of 50 seconds - quickly moving over items in inbox caused Firefox to crash :-(
Guess will wait for some more time. I'm sure there's a bug report somewhere on this - Yahoo mail was broken on Beta 2, got fixed in Beta 3, then was broken in Beta 4 and is still broken on Beta 5.
Will wait for it to be fixed - Any idea if this is a firefox issue or a Yahoo! issue? Seems odd that script can cause the browser to crash so badly.
Monday, March 31, 2008
Hardy heron - first impressions
He he :-) - finally got Ubuntu Hardy heron beta on my home and work laptop. first impressions below:
1. Wubi install from within windows is easy and works great. If after setting up so many boxes, I can go on and on about it, I'm sure that its great help for anyone who's on Windoze. I mean, the barrier to entry has never gone down so much.
2. I guess once you've installed via Wubi and configured your system to your liking, you can uninstall and take an image that you finall install to a dedicated partition - isn't that just awesome.
3. Comes installed with Firefox 3b4 -which is awesome. Given that FF crashes badly on yahoo, this might be a bummer for many people. Should probably have some first time customization that will let you install Opera.
4. Installation is super fast - took about 10 mins for wubi to install, reboot once, finish installation and reboot again. Grub default to Last selected would probably be a better idea.
The not so good
1. Wifi doesnt work out of the box - didn't on my Dell Inspiron 1501 or the Dell Latitude D620. Its the ye olde broadcom problem. This is really the BIGGEST turn off. Hope it will get fixed by the time the final release is out. Meanwhile, had to jump through hoops getting ndiswrapper in. I didn't go the broadcom fwcutter way since that only allows a 802.11b connection from what I read. I'm still not sure what fixed the issue - irrespective, I had to update the system and then things started working like a charm.
2. Compiz configuration isnt installed by default. If this is your first time on Ubuntu and you've come this way to see the awesome 3D desktop, then this is a bummer. Finding out what you need to do is a pain too.
I think that's all there is to it. Its great once wifi starts working normally.
1. Wubi install from within windows is easy and works great. If after setting up so many boxes, I can go on and on about it, I'm sure that its great help for anyone who's on Windoze. I mean, the barrier to entry has never gone down so much.
2. I guess once you've installed via Wubi and configured your system to your liking, you can uninstall and take an image that you finall install to a dedicated partition - isn't that just awesome.
3. Comes installed with Firefox 3b4 -which is awesome. Given that FF crashes badly on yahoo, this might be a bummer for many people. Should probably have some first time customization that will let you install Opera.
4. Installation is super fast - took about 10 mins for wubi to install, reboot once, finish installation and reboot again. Grub default to Last selected would probably be a better idea.
The not so good
1. Wifi doesnt work out of the box - didn't on my Dell Inspiron 1501 or the Dell Latitude D620. Its the ye olde broadcom problem. This is really the BIGGEST turn off. Hope it will get fixed by the time the final release is out. Meanwhile, had to jump through hoops getting ndiswrapper in. I didn't go the broadcom fwcutter way since that only allows a 802.11b connection from what I read. I'm still not sure what fixed the issue - irrespective, I had to update the system and then things started working like a charm.
2. Compiz configuration isnt installed by default. If this is your first time on Ubuntu and you've come this way to see the awesome 3D desktop, then this is a bummer. Finding out what you need to do is a pain too.
I think that's all there is to it. Its great once wifi starts working normally.
Friday, March 28, 2008
Gnuplot, dstat - easy graphing on Linux
Recently, started fiddling around with how to monitor and graph performance data on linux boxes. Other than the usual tools like top and vmstat, which are either interactive (top) or too textual to do anything much.
First off, vmstat, doesnt lend itself well to graphing without additional scripts to lay out the data so tools like gnuplot can be used. Secondly, and more seriously, it doesn't include a timestamp in the output.
Looking around a bit found that dstat seems to be a good replacement to vmstat (and iostat) - and the generated data is consumable with gnuplot.
Here's a quick example of generating graphs for CPU user, system and idle times
now fire up gnuplot and go ahead and plot it
To make gnuplot generat an output file, you need
gnuplot> set term png
gnuplot> set output "dstat.png"
gnuplot> replot
And you're done. here's the graph generated on my machine. There's loads more that you can do - and admittedly, you can do everything by dumping your file to excel. However, that doesn't lend itself well to a completely automated process. When you're doing performance testing and such like, you will likely repeat this enough number of times. Not having to do it manually helps big time!
First off, vmstat, doesnt lend itself well to graphing without additional scripts to lay out the data so tools like gnuplot can be used. Secondly, and more seriously, it doesn't include a timestamp in the output.
Looking around a bit found that dstat seems to be a good replacement to vmstat (and iostat) - and the generated data is consumable with gnuplot.
Here's a quick example of generating graphs for CPU user, system and idle times
dstat -tc 5 500 > dstat.raw
now fire up gnuplot and go ahead and plot it
gnuplot> set xdata time gnuplot> set timefmt "%s" gnuplot> set format x "%M:%S" gnuplot> plot "dstat.raw" using 1:2 title "User" with lines, "dstat.raw" using 1:3 title "Sys" with lines, "dstat.raw" using 1:4 title "Idle" using lines
To make gnuplot generat an output file, you need
gnuplot> set term png
gnuplot> set output "dstat.png"
gnuplot> replot
And you're done. here's the graph generated on my machine. There's loads more that you can do - and admittedly, you can do everything by dumping your file to excel. However, that doesn't lend itself well to a completely automated process. When you're doing performance testing and such like, you will likely repeat this enough number of times. Not having to do it manually helps big time!
Thursday, March 27, 2008
Working with huge XML files - tools of the trade.
XMLStarlet is great for slicing and dicing huge XML files. Had a run in recently - had a 80 Mb XML file in a single line :D. Guess what, most editors that I tried balked and fell over. This was on a 2Gig Core2 Duo machine.
XMLSpy, vi, emacs, notepad++ all died - and trying to do something with a 80 Gig XML where the 80 gigs are on a single line isnt much fun. So the first order of business was to pretty print the XML. XMLstarlet worked great -
and you're done.
The next order of business was that we needed to validate the XML document against a schema. Our first attempt was with Sun's multi schema validator (MSV). MSV does not validate the whole document but instead stops after a certain number of failures. So, MSV - out, XMLStarlet in. XMLStarlet can validate documents again W3C schema, DTD or a RELAXNG schema.
And presto! - you get an error report that you can slice and dice with sed/awk or anything else at all.
XMLStarlet also allows you to write Xpaths to query the xml - however, I found the syntax too weird and round about. A better alternative is a perl based solutions - XSH2 - a command line xml editing shell. You can install it under cygwin and it supports basic command pipelining and redirection.
So go ahead and launch XSH. At your cygwin prompt
Now, lets load up our document, type
Your prompt changes to
So go ahead and try a few xpaths
and XSH prints out the matching nodes. Now what if you need to create a document fragment of nodes matching a certain xpath? Piece of cake - do ahead
XSH2 has many, many more features - but this should be good enough to get you off the ground.
XMLSpy, vi, emacs, notepad++ all died - and trying to do something with a 80 Gig XML where the 80 gigs are on a single line isnt much fun. So the first order of business was to pretty print the XML. XMLstarlet worked great -
xmlstarlet fo file.xml > output.xml
and you're done.
The next order of business was that we needed to validate the XML document against a schema. Our first attempt was with Sun's multi schema validator (MSV). MSV does not validate the whole document but instead stops after a certain number of failures. So, MSV - out, XMLStarlet in. XMLStarlet can validate documents again W3C schema, DTD or a RELAXNG schema.
xmlstarlet val --err --xsd schema.xsd input.xml > errors.txt
And presto! - you get an error report that you can slice and dice with sed/awk or anything else at all.
XMLStarlet also allows you to write Xpaths to query the xml - however, I found the syntax too weird and round about. A better alternative is a perl based solutions - XSH2 - a command line xml editing shell. You can install it under cygwin and it supports basic command pipelining and redirection.
So go ahead and launch XSH. At your cygwin prompt
[~]xsh
---------------------------------------
xsh - XML Editing Shell version 2.1.1
---------------------------------------
Copyright (c) 2002 Petr Pajas.
This is free software, you may use it and distribute it under
either the GNU GPL Version 2, or under the Perl Artistic License.
Using terminal type: Term::ReadLine::Gnu
Hint: Type `help' or `help | less' to get more help.
$scratch/>
Now, lets load up our document, type
$scratch/>$x:=open formatted.xml
Your prompt changes to
$x/>
So go ahead and try a few xpaths
$x/> ls /path/to/node
and XSH prints out the matching nodes. Now what if you need to create a document fragment of nodes matching a certain xpath? Piece of cake - do ahead
$x/> ls /path/to/node | tee fragment.xml
XSH2 has many, many more features - but this should be good enough to get you off the ground.
Saturday, February 09, 2008
Yahoo! mail fixed for Firefox 3 beta 2
Used to get an error on opening Yahoo mail beta in Firefox 3 beta 2 - and had to switch to the plain 'ole yahoo mail. Here's the bug report
Was pleasantly surprised today morning to see that Yahoo! mail beta now works properly in FF3b2. Thanks!
Was pleasantly surprised today morning to see that Yahoo! mail beta now works properly in FF3b2. Thanks!
Wednesday, January 23, 2008
Pesky little bash quoting problem
Have to admit it - this happens every time I sit down to write a some shell script that manipulates paths on windows (where path names often end up with spaces). Soon I find my nifty little script running into problems when it doesn't handle spaces properly and I find myself reading up on bash quoting rules once again...
Anyway, so this post is mostly for self reference :) and to put down some simple rules in the hope that writing it down will help committing it to memory.
The latest (mis)adventure was to make irfanview run under wine and a little script to allow irfanview to open a file provided on the command line. Irfanview being a windoze executable, its necessary to cd to the folder and then pass the file as argument to irfanview. Trivial isn't it....until I found that the script fell over when it got a path liek /path/to/a folder with spaces/image.jpeg.
#! /bin/bash
DIRNAME=$(dirname "$1") # double quotes necessary - since $1 could have embedded spaces
FILENAME=$(basename "$1")
echo $DIRNAME
echo $FILENAME
cd "$DIRNAME" # once more, double quotes necessart
irfanview $FILENAME
Golden Rule
Anyway, so this post is mostly for self reference :) and to put down some simple rules in the hope that writing it down will help committing it to memory.
The latest (mis)adventure was to make irfanview run under wine and a little script to allow irfanview to open a file provided on the command line. Irfanview being a windoze executable, its necessary to cd to the folder and then pass the file as argument to irfanview. Trivial isn't it....until I found that the script fell over when it got a path liek /path/to/a folder with spaces/image.jpeg.
#! /bin/bash
DIRNAME=$(dirname "$1") # double quotes necessary - since $1 could have embedded spaces
FILENAME=$(basename "$1")
echo $DIRNAME
echo $FILENAME
cd "$DIRNAME" # once more, double quotes necessart
irfanview $FILENAME
Golden Rule
When passing a path as argument, always enclose in double quotes.
Thursday, January 10, 2008
Firefox 3 Beta 2 on Ubuntu Gutsy
I'm having weird problems with firefox 3b2 on ubuntu gutsy - and as far as I can tell, I seem to be the only one. Did not find anything similar on ubuntu forums too.
Installed firefox 3 beta 2 from Mozilla to /usr/lib/firefox3b2 folder and created
When I launch firefox3b2, I get firefox alright, however, in the location bar if I type in a url and press Enter, nothing happens - absolutely nothing at all. I have to go and click the green arrow for the browser to open the URL. The search box is even weirder - neither the Enter key works nor does the mouse!
I'm at a loss - and nor can I find any similar experiences on forums etc - ideas welcome :D
SOLVED 01/20:Backed up my .mozilla folder and started firefox 3 b2 - no problems now :D
Installed firefox 3 beta 2 from Mozilla to /usr/lib/firefox3b2 folder and created
lrwxrwxrwx 1 root root 27 2007-12-30 23:44 /usr/bin/firefox-3b2 -> /usr/lib/firefox3b2/firefox
When I launch firefox3b2, I get firefox alright, however, in the location bar if I type in a url and press Enter, nothing happens - absolutely nothing at all. I have to go and click the green arrow for the browser to open the URL. The search box is even weirder - neither the Enter key works nor does the mouse!
I'm at a loss - and nor can I find any similar experiences on forums etc - ideas welcome :D
SOLVED 01/20:Backed up my .mozilla folder and started firefox 3 b2 - no problems now :D
HOWTO: Access your machine from the internet without a static IP
For machines to be accessible on the internet, usually you need a static IP that's leased from your ISP so that when someone types in your IP address, so that packets can be routed over to your machine. However, getting a static ip is costly and for the most part, internet users have dynamic IP address that the ISP allocates each time an end user connects to the internet. Since the ip address keeps changing on each connection, there's no straightforward way to connect to the machine without knowing the IP address that's been allocated - or so it was at least till Dynamic DNS came along (it isnt new - has been around for ages, but for some reason isn't that well known)
Typically, when you type in www.google.com in your browser, your machine performs a DNS (Domain name service) lookup with the DNS servers from your ISP to find out the IP address corresponding to www.google.com. With DDNS (dynamic DNS) this is made to work with your dynamically allocated IP address also. Here's how it works
If you're running Linux/Ubuntu, make sure your're running SSH service and try ssh <your host name>. If you have a router setup, then you will need an additional step - basically the DDNS name refers to your router IP - and not the machine behind the router that you wish to reach. You will also need to make sure that your machine has a static IP from your router. To set up your router, go to your router administration page.
From a different machine (or from the same one -doesnt matter), try out ssh <your host> and you should be able to login to your machine - via the internet.
Typically, when you type in www.google.com in your browser, your machine performs a DNS (Domain name service) lookup with the DNS servers from your ISP to find out the IP address corresponding to www.google.com. With DDNS (dynamic DNS) this is made to work with your dynamically allocated IP address also. Here's how it works
- Register with a DDNS service provider. Service provider provide free accounts for personal use - go to www.dyndns.org
- Once you've created your account, go ahead and set up your hostname. DDNS service providers will have some domains that you can choose from and you get to choose the host part. For a fee, you can also use a domain name of your choice.
- If your set up has a router at your end, check your router administration page if it supports dynamic DNS. If it does, you need to enter the hostname, account and password. Everytime your router connects to the internet, it sends an update notification to the DDNS service notifying the new IP obtained from your ISP. The DDNS service takes care of sending update notifications to routers on the internet.
- If you dont have a router, then download the DDNS client software from the service provider. Most DDNS providers have windows, mac and linux clients. These run on your machine and do the same thing - notify the DDNS service provider of your new IP whenever you establish a connection with your ISP.
- If you've got all this set up, then you can reach your machine from the net - try ping <your host name>
If you're running Linux/Ubuntu, make sure your're running SSH service and try ssh <your host name>. If you have a router setup, then you will need an additional step - basically the DDNS name refers to your router IP - and not the machine behind the router that you wish to reach. You will also need to make sure that your machine has a static IP from your router. To set up your router, go to your router administration page.
- Go to the LAN section and give a range of IPs outside of the static IP. Most routers have lan addresses like 192.168.x.y - 192.168.x.z. If you want your host to have an IP address of 192.168.1.100, then give a LAN range that does not include this IP - say 192.168.1.110 - 192.168.1.200.
- Save and reboot your router.
- Now go to your network settings and enter your static IP (192.168.1.100), netmask 255.255.255.255, gateway (usually 192.168.1.1).
- Go to your router administration page and look for a section like virtual server - your router will allow you to forward packets received on a particular port to a host and port within your LAN. You will have to enter the external port (we'll use 22), the internal machine to forward (192.168.1.100) and the port to forward to (22). With this in place, any packets received on port 22 (ssh) on your router will be forwarded to the 192.168.1.100 machine on the ssh port.
- Save and reboot your router.
- Give it a spin.
From a different machine (or from the same one -doesnt matter), try out ssh <your host> and you should be able to login to your machine - via the internet.
Thursday, January 03, 2008
Back in circulation
I'm on vacation in Bangalore, and guess what - fixing my home computer. Mostly things like lost drivers, screen resolution, cruft in the drives - its an old machine - a P4/512 Meg, but good enough for surfing the net.
Did a few fun things in the midst, and its been ages since I've added anything to this blog. Will summarize for now and put in longer posts with more details in cases someone's interested.
More later.
Did a few fun things in the midst, and its been ages since I've added anything to this blog. Will summarize for now and put in longer posts with more details in cases someone's interested.
- Fixed my windows C drive which was running out of space - used trusty old windirstat for that.
- Set up wifi at home with ADSL modem from BSNL - MT800. Again, wasn't as straightforward as I'd thought.
- Replaced old pcq linux 2006 with ubuntu gutsy - without losing stuff :D. Need to have /home in a separate partition, but otherwise this is a breeze.
- Having fun with compiz-fusion. Its great - however, the documentation isnt easily locatable/consumable enough for first timers (me).
- Set up DNS caching proxy on my linux box - has improved my net/web experience a 100 fold. Was a piece of cake too.
- Set up Dynamic DNS and remote SSH access to my box - this has been the single most important utility/maintainance action.
More later.
Tuesday, July 03, 2007
Sluggish Firefox - and what a hog!
Okay - for the past few days I've been irritated with the browsing experience at home. Pages (google reader, Yahoo mail etc seemed inexplicably slower than before - but they'd load alright - just seemed that teeny weeny bit slower that's enough to leave you suspicious).
I first suspected my ISP (verizon) for frequent dropped connections (saw the DSL modem lights reset a couple of times a day), then my wifi modem (not a high end one), then spyware/malware. So after the usual barrage of tests - wifi interference sources/anti virus/anti spyware/cable tests etc, I still hadn't nailed it.
Finally, used procexp ( you can use plain 'ole Task manager too - this is just flashier) and saw that FF had 330 MB of RAM with 3 or 4 tabs. Also, just plain clicking on a text box was slow, typing into a text field would echo characters after a noticeable delay - so this was definitely a browser problem.
What a relief!
BTW, addons are also the latest attack vector. So be wary of who you let into your browser!
I first suspected my ISP (verizon) for frequent dropped connections (saw the DSL modem lights reset a couple of times a day), then my wifi modem (not a high end one), then spyware/malware. So after the usual barrage of tests - wifi interference sources/anti virus/anti spyware/cable tests etc, I still hadn't nailed it.
Finally, used procexp ( you can use plain 'ole Task manager too - this is just flashier) and saw that FF had 330 MB of RAM with 3 or 4 tabs. Also, just plain clicking on a text box was slow, typing into a text field would echo characters after a noticeable delay - so this was definitely a browser problem.
When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.
-- Sherlock Holmes
Ain't that an apt quote??? I love the addons I have and probably I have one too many. This page on problematic addons was a life-saver - after disabling a bunch of infrequently used addons (stumble upon toolbar, google toolbar, browser sync, adblock filterset g, foxy tunes and some more), I'm back in browsing heaven. The only addons I have now enabled are
- diigo toolbar
- all in one gestures
- Adblock
- Flashgot
- Piclens
What a relief!
BTW, addons are also the latest attack vector. So be wary of who you let into your browser!
Thursday, June 28, 2007
Piclens - full screen slideshows with flickr (and others)
discovered Piclens
Its a great addin for firefox - and integrates with Flickr to give you full screen slideshows a'la Picasa slideshows on your machine!! Its a bit tricky to figure out how to get it to work - Just hover any picture on any page and click on the blue bubbly overlay button that appears
Its a great addin for firefox - and integrates with Flickr to give you full screen slideshows a'la Picasa slideshows on your machine!! Its a bit tricky to figure out how to get it to work - Just hover any picture on any page and click on the blue bubbly overlay button that appears
Python, cygwin, TurboGears, mysql hell
Ok - here goes - I've always liked python, though definitely a noob. I was interested in python on the web and after a bit of googling, seems like TurboGears is the way to go.
First things first - decided to use mysql as the database (already have it on my machine and didn't want to install one more database (postgresql/sqlite). Now it turns out that MySQL doesnt have a cygwin package. More googling - mysql server can't run on cygwin due to something to do with pthreads. You can compile the mysql client on cygwin though.
That's what I decided to do - grabbed the Linux tar.gz source from mysql.com, got it into a directory and ran ./configure --without-server, followed by make && make install. All went through fine -other than the fact that it was time consuming and pretty boring (more so since I had to download and install gcc, bintools first in cygwin)
I thought I'd got through the hard part and what remained was to install python-MySQLdb package. Off I went to
easy_install MySQLdb
No luck there - package build failed with missing library -lmysqlclient_r. Turns out that the 'thread-safe' version of mysql client (mysqlclient_r) is preferred but the mysql build doesnt build this by default. What a shame!!!
Anyway, so I wasnt going to redo the whole mysql client library build again - more README files and googling later, grabbed mysql-python-1.2.2 tarball, got it into a folder, edited site.cfg and changed 'threadsafe=false'. The next run on python setup.py build worked properly, with the python mysql linking against the non threadsafe mysqlclient library.
Think troubles are over yet? No way.
Off I went to test - started the python interpreter and did a import MySQLdb, and got a Permission Denied in some 'egg' file!!! What the heck are these egg files anyway. Well, I did'nt have much of a clue and more google later got educated that these are install packages used by the easy install system. The more I looked, the more it seems that the easy install is anything but easy :(. Anyway, this one had me floored - since I couldnt get to the line of source where the error was and had no clue how to view the contents of an 'egg' (they're zips - but I didnt know that and very helpfully there's hardly anyplace where they say that they're zips with the extension of egg!!! Baaah!! - why couldn't they just use .zip?)
More and more hard googling - and this time the info's really sketchy till eventually found a post from a guy who asked the exact same question. Guess what, the easy_install system unzips the eggs to some folder (pointed by PYTHON_EGG_CACHE env var) and there I needed to do a chmod a+x on the _mysql.dll. Well so I did echo $PYTHON_EGG_CACHE and the var isnt set!!! Admittedly at this point, I'm not looking sharp either - what started out as a quick spin has become a quagmire of installation issues - but I'll be damned if I let it sink me!!! Eventually, had the Eureka moment and checked ~/.python-eggs and sure enough found the truant _mysql.dll. Quick chmod a+x and presto - import MySQLdb worked!!! YAHOOOOOOOOOOOOOOOOOOOO!!!!
And now back to where I started - went back to turbogears, did a tg-admin quickstart, setup a mysql database and started with python start-testproject.py. Guess what - no luck yet - turns out that mysql cant client to my windows server with a socket.
More googling - and this time its really desperate - and more enlightenment - the windows mysqld doesnt do unix sockets - how do I force tcp/ip? Simple - use 127.0.0.1 as hostname in the connection settings instead of localhost!! Finally something that was easy to fix. Finally after 8 hours of on and off hacking away at installation issues I'm glad to see a turbogears web page.
Bottomline: Python's great, from the looks of it, turbo gears seems well designed. Mysql is a great database - a cygwin native server would be great - or atleast a client package. But if one has to run through all these hoops just go get a 'quick spin' then adoption's going to be difficult.
I havent tried RoR - but has someone tried a similar thing on cygwin (cygwin, ruby, RoR, mysql)? How does the experience compare - is it any easier to get off the ground?
First things first - decided to use mysql as the database (already have it on my machine and didn't want to install one more database (postgresql/sqlite). Now it turns out that MySQL doesnt have a cygwin package. More googling - mysql server can't run on cygwin due to something to do with pthreads. You can compile the mysql client on cygwin though.
That's what I decided to do - grabbed the Linux tar.gz source from mysql.com, got it into a directory and ran ./configure --without-server, followed by make && make install. All went through fine -other than the fact that it was time consuming and pretty boring (more so since I had to download and install gcc, bintools first in cygwin)
I thought I'd got through the hard part and what remained was to install python-MySQLdb package. Off I went to
easy_install MySQLdb
No luck there - package build failed with missing library -lmysqlclient_r. Turns out that the 'thread-safe' version of mysql client (mysqlclient_r) is preferred but the mysql build doesnt build this by default. What a shame!!!
Anyway, so I wasnt going to redo the whole mysql client library build again - more README files and googling later, grabbed mysql-python-1.2.2 tarball, got it into a folder, edited site.cfg and changed 'threadsafe=false'. The next run on python setup.py build worked properly, with the python mysql linking against the non threadsafe mysqlclient library.
Think troubles are over yet? No way.
Off I went to test - started the python interpreter and did a import MySQLdb, and got a Permission Denied in some 'egg' file!!! What the heck are these egg files anyway. Well, I did'nt have much of a clue and more google later got educated that these are install packages used by the easy install system. The more I looked, the more it seems that the easy install is anything but easy :(. Anyway, this one had me floored - since I couldnt get to the line of source where the error was and had no clue how to view the contents of an 'egg' (they're zips - but I didnt know that and very helpfully there's hardly anyplace where they say that they're zips with the extension of egg!!! Baaah!! - why couldn't they just use .zip?)
More and more hard googling - and this time the info's really sketchy till eventually found a post from a guy who asked the exact same question. Guess what, the easy_install system unzips the eggs to some folder (pointed by PYTHON_EGG_CACHE env var) and there I needed to do a chmod a+x on the _mysql.dll. Well so I did echo $PYTHON_EGG_CACHE and the var isnt set!!! Admittedly at this point, I'm not looking sharp either - what started out as a quick spin has become a quagmire of installation issues - but I'll be damned if I let it sink me!!! Eventually, had the Eureka moment and checked ~/.python-eggs and sure enough found the truant _mysql.dll. Quick chmod a+x and presto - import MySQLdb worked!!! YAHOOOOOOOOOOOOOOOOOOOO!!!!
And now back to where I started - went back to turbogears, did a tg-admin quickstart, setup a mysql database and started with python start-testproject.py. Guess what - no luck yet - turns out that mysql cant client to my windows server with a socket.
More googling - and this time its really desperate - and more enlightenment - the windows mysqld doesnt do unix sockets - how do I force tcp/ip? Simple - use 127.0.0.1 as hostname in the connection settings instead of localhost!! Finally something that was easy to fix. Finally after 8 hours of on and off hacking away at installation issues I'm glad to see a turbogears web page.
Bottomline: Python's great, from the looks of it, turbo gears seems well designed. Mysql is a great database - a cygwin native server would be great - or atleast a client package. But if one has to run through all these hoops just go get a 'quick spin' then adoption's going to be difficult.
I havent tried RoR - but has someone tried a similar thing on cygwin (cygwin, ruby, RoR, mysql)? How does the experience compare - is it any easier to get off the ground?
Thursday, May 24, 2007
Maven Emma plugin - filters dont work
Using v0.5 of the Maven emma plugin. Found that if you specify an include/exclude filter, no classes are instrumented....On digging a little deeper under the covers, it seems that there's a bug with the Maven emma plugin - I've edited the plugin's jelly script and using it here.
Posted the bug and the fix to Maven's sourceforge developer forum. You can find it here
The fixed plugin.jelly's here - Maven emma plugin 0.5 fixed
Posted the bug and the fix to Maven's sourceforge developer forum. You can find it here
The fixed plugin.jelly's here - Maven emma plugin 0.5 fixed
Friday, May 04, 2007
Building a FC6 server for Java development
It has come upon me to build a FC6 server with a Java build environment that is going to serve as the build machine. This is the first time I'm coming to Fedora - always having used Debian in the past.
That's all there is to it. Just make sure that you do everything as the 'application' user - dont do it with your account or the root account.
- OS installation was smooth. I installed with KDE since some folks prefer the GUI.
- During the installation, I chose Java Development - and all the gcj components got installed. Then post installation, I downloaded JDK from Sun and installed it too. The (gcj) tomcat of course didnt work very well in this mess. Fixing it was easy - just fired up yum via Add or Remove Programs and removed gcj - which removed all the other gcj tools and libraries too.
- Setting the host name - unexpectedly, had trouble doing this - finally used system-config-network.
- Environment variables - again a little digging around to figure out how to set environment variables so that they are effective for all users on the system - essentially JAVA_HOME, JRE_HOME, CATALINA_HOME, MAVEN_HOME etc. Finally found that this is best set in /etc/environment where it works for all users.
- Downloaded java tools and libraries - housed tomcat and maven under /usr/lib/java/maven-xxx and /usr/lib/java/apache-tomcat-x.xx and created symlinks. Placed symlinks to tomcat's startup.sh and shutdown.sh in /usr/bin. Placing a symlink to maven doesnt work - but an alias works as well. To make it work for all users, simply put it in /etc/profile
alias maven=/usr/lib/java/maven/bin/maven - Created a group java and added users to the group. Set permissions on the maven installation folder so that java group has write access (this is so that maven plugin:download for additional plugins works properly and can write to the maven plugins folder)
That's all there is to it. Just make sure that you do everything as the 'application' user - dont do it with your account or the root account.
Tuesday, May 01, 2007
Ant - Debugging classpath
Here's a nice tip on debugging classpaths in Ant: http://www.javalobby.org/java/forums/t71033.html
Essentially,
Essentially,
<?xml version="1.0"?>
<project name="project" default="default">
<property name="lib" value="web/WEB-INF/lib"/>
<property name="src" value="src"/>
<property name="dist" value="dist"/>
<path id="classpath">
<fileset dir="${lib}">
<include name="**/*.jar"/>
</fileset>
</path>
<target name="default" description="--> description">
<javac srcdir="${src}" destdir="${dist}">
<classpath refid="classpath"/>
</javac>
</target>
</project>
Sunday, April 22, 2007
Cobertura vs EMMA
This is a rant against the Cobertura maven integration - and to a certain extent on Cobertura also - and hopefully an objective one ;). Hopefully this'll help someone select between the two big (open source) coverage tools out there.
We've been using Cobertura at work till now and its done its job nicely - the reports look great and the maven 1.x integration, while not neat, is functional. While we knew that someday we were going to have to merge coverage data and have a single report across multiple test methods (junit, selenium tests and manual), cobertura documentation stated that this was possible and so we weren't really bothered.
Thought I'd give it a whirl and set it up - and that's when the trouble started. Atleast, with the maven integration.
First of all - there's no way to just instrument code. You can generate the report (which will instrument classes and run the tests) but if you just want to instrument classes so that the final deployable contains instrumented classes, its a no go with the maven plugin tools.
Obviously, no point giving it up there - so thought I'd just include the ant tasks and go the ant way in my maven goal. Turns out that there's no 'plugin init' kind of goal that can be called post build:start that will set up dependencies and import the cobertura ant tasks. You have to do it all yourself. Fine - went that way too - so now my maven.xml uses the cobertura ant tasks and finally I'm able to generate an instrumented build. YIPPPPPPPEEEEEE.... or wait...lets' just make sure that this thing works...
Does it?
Turns out - no - it doesnt - so I dropped the WAR into tomcat and accessed the login page of the application and then shut down tomcat nicely. There's even a cobertura.ser created in the tomcat bin folder and I'm thinking that probably this will all work together finally...
So I go ahead, tweak my Maven.xml further with a coverage task that will merge the data from the junit runs and servlet container runs. Turn the switch on... and lo and behold...Exception reading the merged data file. Back to google and after hunting around for sometime found this Bug while merging reports
So finally I'm ready to give up cobertura and give Emma a try...and it couldnt have been better...
1. Goals are nicely setup
2. You can init the emma system with the emma:init goal and then use ant tasks if you want flexibility for doing things like merging reports.
3. The merging works :))
One sticky issue that I did come up with was that the for the same source and test cases the coverage reported by cobertura and emma differ widely. With Cobertura, we were at 40% coverage while with EMMA, the number's up to 60% coverage - and while EMMA has some literature on how it does things - I'd be glad if someone did explain why or how the reported numbers could be so different for the same base code and unit test suite?
We've been using Cobertura at work till now and its done its job nicely - the reports look great and the maven 1.x integration, while not neat, is functional. While we knew that someday we were going to have to merge coverage data and have a single report across multiple test methods (junit, selenium tests and manual), cobertura documentation stated that this was possible and so we weren't really bothered.
Thought I'd give it a whirl and set it up - and that's when the trouble started. Atleast, with the maven integration.
First of all - there's no way to just instrument code. You can generate the report (which will instrument classes and run the tests) but if you just want to instrument classes so that the final deployable contains instrumented classes, its a no go with the maven plugin tools.
Obviously, no point giving it up there - so thought I'd just include the ant tasks and go the ant way in my maven goal. Turns out that there's no 'plugin init' kind of goal that can be called post build:start that will set up dependencies and import the cobertura ant tasks. You have to do it all yourself. Fine - went that way too - so now my maven.xml uses the cobertura ant tasks and finally I'm able to generate an instrumented build. YIPPPPPPPEEEEEE.... or wait...lets' just make sure that this thing works...
Does it?
Turns out - no - it doesnt - so I dropped the WAR into tomcat and accessed the login page of the application and then shut down tomcat nicely. There's even a cobertura.ser created in the tomcat bin folder and I'm thinking that probably this will all work together finally...
So I go ahead, tweak my Maven.xml further with a coverage task that will merge the data from the junit runs and servlet container runs. Turn the switch on... and lo and behold...Exception reading the merged data file. Back to google and after hunting around for sometime found this Bug while merging reports
So finally I'm ready to give up cobertura and give Emma a try...and it couldnt have been better...
1. Goals are nicely setup
2. You can init the emma system with the emma:init goal and then use ant tasks if you want flexibility for doing things like merging reports.
3. The merging works :))
One sticky issue that I did come up with was that the for the same source and test cases the coverage reported by cobertura and emma differ widely. With Cobertura, we were at 40% coverage while with EMMA, the number's up to 60% coverage - and while EMMA has some literature on how it does things - I'd be glad if someone did explain why or how the reported numbers could be so different for the same base code and unit test suite?
Tuesday, April 17, 2007
Tulip festival at Skagit county
We visited the Tulip festival at Skagit valley. Its hard to describe in words - and a picture's worth a thousand words - so here's a bunch of pics - let them do the talking :).Anyway, had a lovely weekend driving up to Mount Vernon with friends and had a memorable weekend.
 |
| TulipFestival |
Subscribe to:
Posts
(
Atom
)