Framework Description: The framework essentially comprises 4 parts.

1.) TreeNode and DataPoint classes to hold hold data and define tree nodes: -

DataPoint contains an attribute array to hold the values for a particular node and is contained within each TreeNode instance. The structures for these classes are as follows.

class TreeNode {

public double entropy;
public Vector data;
public int decompositionAttribute;
public int decompositionValue;
public TreeNode [] children;
public TreeNode parent;

public TreeNode () {
data = new Vector ();
}
}

class DataPoint {

public int [] attributes;
public String label;

public DataPoint (final int numattributes) {
attributes = new int [numattributes];
}
}

2.) DecisionTree: -

This is the base class for decision-tree implementation and has a containment relation with the TreeNode class. It encapsulates all the implementation details for decision tree creation and entropy calculation. It uses the max-gain criteria for splitting over nodes. Right now, pruning hasn’t been implemented.

The field-definitions for decision-tree is as follows.

public class DecisionTree {

private int numAttributes;
private String [] attributeNames;
private Vector [] domains;
private TreeNode trainingRoot;
private TreeNode testingRoot;
private String namesFile;
private String trainingDataFile;
private String testingDataFile;
private int correctCount;
private int inCorrectCount;
private boolean debug;
private int maxDepth;
….

}

The decision-tree class invokes the InputProcessor (explained below) , which populates the training and testing Roots with input data. After this, it invokes createDecisionTree and induce modules to the create the actual tree.

The overall structure of induce is as follows.

public void induce (final TreeNode node) {

double bestEntropy = 0;
boolean selected = false;
int selectedAttribute = 0;
int numdata = node.data.size ();
int numinputattributes = numAttributes - 1;
node.entropy = calculateEntropy (node.data);
if (node.entropy == 0) {
return;
}

for (int i = 0; i < numinputattributes; i++) {
int numvalues = domains[i].size ();
if (alreadyUsedToDecompose (node, i)) {
continue;
}

double averageentropy = 0;

for (int j = 0; j < numvalues; j++) {
Vector subset = getSubset (node.data, i, j);
if (subset.size () == 0) {
continue;
}

double subentropy = calculateEntropy (subset);
averageentropy += subentropy * subset.size ();
}

averageentropy = averageentropy / numdata;

if (selected == false) {
selected = true;
bestEntropy = averageentropy;
selectedAttribute = i;
} else {

if (averageentropy < bestEntropy) {

selected = true;
bestEntropy = averageentropy;
selectedAttribute = i;
}
}
}

if (selected == false) {
return;
}

int numvalues = domains[selectedAttribute].size ();
node.decompositionAttribute = selectedAttribute;
node.children = new TreeNode [numvalues];
for (int j = 0; j < numvalues; j++) {
node.children[j] = new TreeNode ();
node.children[j].parent = node;
node.children[j].data = getSubset (node.data,
selectedAttribute, j);
node.children[j].decompositionValue = j;
}

for (int j = 0; j < numvalues; j++) {
induce (node.children[j]);
}
node.data = null;
}

The most important function call within induce is to calculate the entropy and the structure for that is given below.

public double calculateEntropy (final Vector data) {

int numdata = data.size ();
if (numdata == 0) {
return 0;
}

int attribute = numAttributes - 1;
int numvalues = domains[attribute].size ();
double sum = 0;

for (int i = 0; i < numvalues; i++) {
int count = 0;
for (int j = 0; j < numdata; j++) {
DataPoint point = (DataPoint) data.elementAt (j);
if (point.attributes[attribute] == i) {
count++;
}
}

double probability = ((double) count) / numdata;

if (count > 0) {
sum += -probability * Math.log (probability);
}
}
return sum;
}

3.) InputProcessor: -

This is used for all the input processing and in turn, for node data population and is utilized by DecisionTree to get the populated Node Objects from raw input data. This is a fairly complicated class with different modules to read the attributes and the actual data. Besides, it contains the functionality to do a random split between training and test in case only one data-set is provided & to discretize the linear data into bins.

The key module is readDataSets.

public int readDataSets (final String fileName, final TreeNode root, final String toSplit) throws Exception {

FileInputStream in = null;
/************************************************
* Reading the Training / Testing data set File *
************************************************/
try {
File inputFile = new File (fileName);
in = new FileInputStream (inputFile);
} catch (Exception e) {
System.err.println (”Unable to open file: ” + fileName + “n” + e);
return -1;
}

BufferedReader bin = new BufferedReader (new InputStreamReader (in));
String input;
int index = 1;
while (true) {
input = bin.readLine ();
if (input == null) {
break;
} else if (input.startsWith (”|”)) {
continue;
} else if (input.contains (”|”)) {
input = input.substring (0, input.indexOf (”|”));
}
if (input.equals (”")) {
continue;
}

StringTokenizer tokenizer = new StringTokenizer (input, “,”);
int numtokens = tokenizer.countTokens ();
if (skipCount > -1) {
if (numtokens != numAttributes + 1) {
return -1;
}
} else if (numtokens != numAttributes) {
return -1;
}

DataPoint point = new DataPoint (numAttributes);
// if there is no label to skip
if (skipCount == -1) {
point.label = “Example#” + index;
for (int i = 0; i < numAttributes; i++) {
point.attributes[i] = getSymbolValue (i, tokenizer.nextToken ());
}
} else if (skipCount > -1) {
int attributeIndex = 0;
for (int panditIndex = 0; panditIndex < numAttributes + 1; panditIndex++) {
// assign label to the data point and skip it as an attribute field
if (panditIndex == skipCount) {
point.label = tokenizer.nextToken ();
continue;
}

point.attributes[attributeIndex] = getSymbolValue (attributeIndex, tokenizer.nextToken ());
attributeIndex = attributeIndex + 1;
}
}

/************************************************
* Required 2/3-1/3 random data split follows *
************************************************/
if (toSplit.equals (”SPLIT”)) {
double randomNumber = 3 * Math.random ();
if (randomNumber > 2) {
testingRoot.data.addElement (point);
} else {
root.data.addElement (point);
}
} else {
root.data.addElement (point);
}
index = index + 1;
}
in.close ();
bin.close ();
return 1;
}
4.) DisplayProcessor:-

This class provides the key implementation for handling the display functionality. The output formatting is similar to what you see here. The key module is displayTree which displays each node-level recursively.

public void displayTree (final TreeNode node, final String tab, final int numAttributes, final String [] attributeNames, final Vector [] domains, final int depth) {

int outputattr = numAttributes - 1;
if (node.children == null) {
int [] values = getAllValues (node.data, outputattr, domains);
if (values.length == 1) {
System.out.print (” ” + domains[outputattr].elementAt (values[0]));
return;
}

System.out.print (” {”);
for (int i = 0; i < values.length; i++) {
System.out.print (”"” + domains[outputattr].elementAt (values[i]) + “”");
if (i != values.length - 1) {
System.out.print (” , “);
}
}

System.out.print (” };”);
return;
}

int numvalues = node.children.length;
for (int i = 0; i < numvalues; i++) {
System.out.print (”n” + tab + attributeNames[node.decompositionAttribute] + ” = ” + domains[node.decompositionAttribute].elementAt (i) + “:”);
if (depth <= this.MAX_DEPTH) {
displayTree (node.children[i], tab + “: “, numAttributes, attributeNames, domains, depth + 1);
} else {
System.out.print (”…”);
}

if (i != numvalues - 1) {
System.out.print (”t”);
} else {
System.out.print (”");
}
}
}

Again, this can be easily modified or extended as needed since the display functionality is cleanly decoupled from decision-tree creation logic and other parts explained above. Finally, in case you require the complete framework implementation alongwith the data-sets and instructions, please drop me a note explaining the purpose and need for it and I shall get back to you.

After a lot of hassle, I was finally able to resolve it and everything seems to be working alright since then. Nehow, I thought of putting down my observations here for anyone who may face similar issue with their installations or otherwise.

- The obvious things first - I checked the permissions on the installation folder (for me, <root>/blog) and found it to be perfectly ok (755 for read / execute to world).

- Next I checked the .htaccess in the root and so I found the problem. I had an earlier rails (Ruby on Rails) installation and following lines were present in it: -

RewriteEngine on

RewriteCond %{HTTP_HOST} ^sushain.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.sushain.com$
RewriteRule ^contest http://127.0.0.1:12009%{REQUEST_URI} [P,QSA,L]
DirectoryIndex preload.html index.html
# BEGIN WordPress

# END WordPress

I immediately knew that the url rewriting module was causing some issue here, but I didn’t want to tinker with this since I may want my Ruby app (although dormant for now) to come alive again in future. So, what I did instead was that I changed the .htaccess file in the /blog folder from -

DirectoryIndex index.php
AuthUserFile “/home/sushainp/.htpasswds/public_html/blog/passwd”

to -

DirectoryIndex index.php
AuthUserFile “/home/sushainp/.htpasswds/public_html/blog/passwd”
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /blog/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /blog/index.php [L]
</IfModule>

# END WordPress

This overrode the url rewriting mechanism for /blog level addresses and thus, solved the issue.

Btw, for those of you interested in knowing more about url rewriting module, more information is available here.

Cheers !

Solution options: Well, I tried out many libraries and finally used none. In any case, here is my experience.

1.) Ziplib library from phpclasses - Very powerful library, can be used for recursively adding folders to an archive, etc. Just seemed to be too complicated for my task.

2.) Lib by devco.net - Simple usage and fairly good lib. This is how I tried using it: -

require (”pclzip.lib.php”);
$exp = “63″;
$zipfile = new PclZip(”$exp”.”_pack.zip”);
error_reporting(E_ALL);
$output = shell_exec( “cp index.php 63″ );
$v_list = $zipfile->create(”$exp”);

The issue I had with this one was that it didn’t allow me to add subdirectory structures selectively, or at least I couldn’t fig it out rightaway.

3.) Yet another library, this one was by Drupal - Seemed similar to 1.) and again, I just realized mid-way that I needn’t have that much functionality at all and wanted something simpler.

4.) Another one by granthinkson - Neat library, esp if you want to get all dirty with the actual file-creation and understanding process. I just wanted my zip :sigh:.

5.) Yet another by Techrepublic - Again, simple usage and I would’ve almost gone with this, but for point (6.) below. But, in any case, fairly easy to use this -

$obj = new Archive_Zip(’63.zip’);
$files = array(’63.soft’,'63.chp’, ‘63.cel’);
$obj->create($files);

6.) Well, dunno why, but I had my usual whim and I suddenly decided not to go with any of these and resorted back to plain old ‘jar’ util. So, this is what I did finally, esp since our server is configured with a jre anyways.

/* Modified: 1-1-09 | code for zip creation - pandit */
…..
$zipfilename = “$exp”.”_pack.zip”;
……
if(!file_exists(”$tmp_dir/$zipfilename”)) {
$src = “$root_dir/ftp/pub/$exp/*”;
$dest = “$tmp_dir/$exp”;
exec( “cp $src $dest” );
exec (”jar cfM $tmp_dir/$zipfilename -C $dest .”);
}
……
$zip_link = $base_url . “/$zipfilename”;

But, thats so trivial, said Watson.. Yup, everything becomes trivial after explanation, sighed Holmes.

Anyhow, I implemented it here

In this piece, I try to analyze the refutations (against various arguments) given by Turing in his classic article on the ‘Imitation Game’, and determine what still holds given the present state of AI. The following arguments and analysis are with reference to the original Turing Article located at - http://www.loebner.net/Prizef/TuringArticle.html Towards the end, I present my views on the current state of success at passing the Turing Test and what future warrants on the subject.

1.) Theological objection: - To me it seems that the theological arguments have always banked on faith rather than being based on logic and reason. However, this objection may still carry some weight to those who can define and clarify the idea of a soul and then be able to link this idea to thinking process. Without these definitions, linking the thinking process with man’s immortal soul is a highly generalized assumption based on the notion that an all-powerful God is able to somehow put this into effect. However, this very assumption is contradicted later on by claiming that the same God can’t put those souls somewhere else, in this instance - machines. Overall, this objection seems too pretentious to carry substantial weight now. Moreover, Turing’s refutations in the original paper seemed to be quite rational to put this case to rest.

2.) The “Heads in the Sand” objection: - This objection in itself (as stated in the original paper) carries no weight now. However, it is somewhat related to the ethical issues that are now often connected with AI, when we pause considering whether we can develop AI, and ask ourselves - Should we create AI and ponder upon the various implications of advanced / strong AI if it were to co-exist with humankind. But, since the current discussion is about ‘Can machines think’, we would rest this case here.

3.) The Mathematical objection: - Actually, this argument doesn’t really seem like a reason to object the test since it tentatively puts machines only at a disadvantage w.r.t humans and thus this test may be a decent criteria to determine the validity of that objection. Anyhow, as far as computing capacities are concerned, we’ve reached very high orders from what was available back in Turing’s time. Thus, ideally, such machines can be build which would decently respond to the objections in this point.

4.) The argument from consciousness: - In his original paper, Turing has handled this argument from the perspective of feeling as it relates to thinking and he describes this objection as - ‘whether machines can feel that they are thinking‘. Within this context, his refutations seem valid since he links these objections to solipsism (claim that only self can be verified), which he then claims is noone’s win since it makes idea communication difficult.

However, there is a broader context to this argument now, especially after thought experiments like Chinese Room by Searle - http://en.wikipedia.org/wiki/Chinese_Room . These experiments broaden the scope of the argument by relating consciousness to mind, bringing the notion of intentionality & states, and illustrating that machines can’t have minds irrespective of their intelligent behavior.

One may argue that Turing would have reasoned that even the neurons within human brains aren’t conscious in the sense of understanding what they are doing, and it somehow appears that the overall system as a whole is behaving intelligently and there is nothing more to it than appearance. However, since these ideas are not the part of his original paper, these objections would still hold.

5.) Arguments from various disabilities: - This argument is essentially a combination of various objections and expectations each of which are discusses below. For many of these, we would require an exact definition of the word to relate it to AI and for this purpose, we would refer this source - http://www.answers.com : -

- Being kind, friendly, loving: These are abstract words that require contextual definitions. If these are implied by being helpful and useful, then certainly this objection will not hold since there are many machines, softwares etc, that help people automate their work. Applications of AI to advanced robotics has produced many helpful robots - http://www.helpfulrobots.com/ .

However, if we think of kindness, friendship and falling in love from a conscious point of view, then it would again require a mind and intention. It would thus fall into the consciousness argument discussed above.

- Being resourceful: As per our source, resourceful means - able to act effectively or imaginatively, especially in difficult situations. Clearly, most of the existing softwares meet the criteria of efficiency. There are various agent programs that deal with different situations intelligently as well - http://en.wikipedia.org/wiki/Windows_Live_Agents . Imagination is again dependent on how we define it, but overall this criteria seems to be met now.

- Being beautiful: If this concerns external beauty, there are certainly beautiful machines created by careful manufacturing, etc. Internal beauty is again abstract and would belong to the consciousness argument, as in being aware of an internal state.

- Having initiative: Most AI machines or softwares have been created with an ultimate goal to take the initiative from humans and automate processes.

- Having a sense of humour: There have been some efforts towards artificial comedy - http://www.clone3d.net/ .

- Telling right from wrong and learning from experience and doing something really new: Machine learning and causalreasoning are two of the core fields in AI research and have been implemented in many systems. Doing something new can be a direct consequence of machine learning or learning from experience. So, this objection doesn’t hold now.

- Making mistakes: Since AI programs are written by humans, they can be buggy and hence their actual output can deviate from expected. This can be classified as making mistakes.

- Making someone fall in love: This is quite achievable using meticulous and aesthetical design, and giving a pleasing appearance to a useful machine.

- Using words properly: NLP and voice recognition programs are a case in point.

- Having as much diversity of behavior as a man: This objection still holds and would continue to hold until certainmilestones are reached in brain and mind simulation .

- Being the subject of its own thought: Thought requires a clear contextual definition here, but, in general, this could be similar to providing a program source as an input to itself and observe the output. Again, doable and hence the objection doesn’t hold.

6.) Lady Lovelace’s Objection: The gist of this objection can be summed up in one sentence - machines can never do anything really new so as to take us (humans) by surprise and Turing gave a classic refutation to this by claiming that this is equivalent to saying that there is nothing new under the sun since every new discovery or invention is linked to some past event or chain of events. So, basically he tried to give a context to what should be considered as really new and within that context, this objection is clearly not true today. There are programs capable of giving surprises as well as creating unique and new scenarios. A case in point is the origin of Butterfly effect - http://en.wikipedia.org/wiki/Butterfly_effect .

7.) Argument from Continuity in the Nervous System: This argument needed refutation on grounds of fairness in the original paper at the time Turing wrote it. However, now with neuro-computing models in existence, this argument can be made redundant in principle.

8.) The Argument from Informality of Behavior: The crux of the argument is - If each man had a definite set of rules of conduct by which he regulated his life, he would be no better than a machine. But there are no such rules, so men cannot be machines. Turing refutation on this still holds since we still haven’t found the absolute laws of behavior that regulate life, something similar to the M-theory, or the theory of everything - http://en.wikipedia.org/wiki/M-theory .

9.) The Argument from Extrasensory Perception: The argument itself seems pretty weak and containing very less substantiation. There are varied opinions on phenomena like telepathy, precognition, etc, and like the argument the refutation is also pretty weak (telepathy-proof room). Anyhow, some people may still have this objection, however it remains a very subjective issue.

New Issues: -

Inherent problems with the Turing Test and Comprehensive Turing Test:

With the advancement of AI to fields like machine vision and perception, one may demand a more comprehensive test. This is since humans generally don’t use question-answering as a means to find whether someone is able to ‘think‘. We use more simplistic (or elaborate, if we consider machines) methods like sight, sound of voice, etc and our notion of thinking is implied by many other apparent things than just plain response. A case in point are the systems like CAPTCHA - http://www.captcha.net/ . This brings back the old issue on whether the Turing Test is a real test of intelligence at all. It seems that the inherent problem with the test is that a program can fool an interrogator by playing on his psychology. This task is much more easier and doesn’t definitely require intelligence equivalent to the mind, or thought process. A simple re-use of the question-phrases can be used as an effective way of letting an interrogator know that the program is actually a person belonging to a particular genre. A case in point is Eliza - http://en.wikipedia.org/wiki/ELIZA . Of late, there have been similar concerns raised over expertise of judges in the Loebner competition. As far as the current state of success with passing the Turing Test is concerned, it may be noted that Loebner is the only official competition for judging this and most of the effort in AI is actually not focused on passing this test. This current state of AI may not have a one-to-one co-relation with the whether this test has been passed or not. However, current state of AI does give us an idea regarding whether this test can be passed if all effort was directed towards it. The 2007 winner for Loebner award was UltraHal, which is basically a software assistant and was re-programmed to enter the competition so that it could specifically fool the judges (ref the interview here - http://aidreams.co.uk/forum/index.php?page=67 ) . So, again the chance of fooling a judge is subjective and dependent on the skill of judge as described above in the case of Eliza bot. It may be argued that the Turing test has already been passed by some programs since they fooled a set of judges in Loebner, but the credibilty of judges would always be in question and so will be the exact rules and regulations under the competition was held. In all, I would say there’s a chance of about 25% for a machine to pass the test, considering strict Turing guidelines and maybe in next 50 years, we would make substantial advances in allied fields of AI to eventually pass the test satisfactorily, however, this again depends on how much effort is put in actually solving this problem in specific.

- Indexing: Most hyped is the size of its massive index (~120 billion pages), as also quoted in their info page - “Cuil searches more pages on the Web than anyone else—three times as many as Google and ten times as many as Microsoft”. However, I didn’t find any conclusive proof for this in that Google still listed more total hits on most of my test-searches. [Example SearchKey = 'Sushain': Cuil - 662 | Google - 929]

Now, this may still improve with time, so we’ll let this one pass.

- Renewal rate: Now, they might have indexed more pages than Google but their index still doesn’t contain recent links. Indexing recent content is really critical to search engines and again, they should improve on this front with time.

- Page ranking: The info page also says - “Rather than rely on superficial popularity metrics, Cuil searches for and ranks pages based on their content and relevance. When we find a page with your keywords, we stay on that page and analyze the rest of its content, its concepts, their inter-relationships and the page’s coherency.”

The issue I found was that sometimes the searches listed ‘not-so-important’ results before more-relevant ones. For instance, listing the pages linked to someone’s friend (via a social network) before actually putting the person’s site-listing etc. In my case, it shows my name being present in my friend’s HiFi n/w, before pointing anything relevant to me (and I haven’t put a robots.txt file on my website yet to stop crawlers :-)).

- Infrastructure: Many a times, during my experimentation, Cuil would just give a ‘We’ll be back’ message, or wouldn’t repeat the results that were generated for the same query, some time ago. Again, there’s room for improvement and they would do well to evolve with time.

- Layout: I’ve personal hots for dark (energy-saving) backgrounds and Cuil scores there. Even otherwise, the results are laid out in a better way and are easy to spot, but again, I think I’ve seen this before on proprietary search-engines.

- Privacy: Cuil does claim to be the good-boy search engine n not track the data et al, but the bottom-line still is how good the search capability is.

Final verdict: Well, I’m pretty much in favor of any new-comer into a partially monopolized domain (Though I love Google as much as anyone :-)). The only issue I’ve is that they really could’ve done some more ground-work before making this live (Links on Info and Privacy weren’t working till yesterday) since the first impression really goes a long way. All in all, I think, we need to give more time to Cuil; No enterprise gets build in a day and Cuil should be appreciated to have at least taken an initiative in an otherwise one-firm show.

Description: The framework is developed as a javascript service that defines vertical / horizontal hierarchical layout method handlers. There are different handlers based on whether one wants to organize divisions (div tags), separate nodes, or node hierarchies. Apart from the structuring logic, the file also includes the rendering logic that paints the node(s) and their associated relations to build the actual hierarchy.

Now, the layouts are provided out of the box and all one requires is a nodelist (supporting composite pattern), which will then be consumed to draw the actual hierarchy. Such hierarchies can be utilized for a particular filtered view within a social networking graph.

The node definition(s) should comply to the following generic template to be able to fed and subsequently consumed properly by the framework: -

var Node = function(id) {
this.id = id;
this.coords;
this.shape;
this.text;
this.group;
this.childList = new dojox.collections.ArrayList();
}

Illustration: -

- Get a hierarchy object (pass the ultimate parent of the node hierarchy and specify the alignment [horizontal / vertical]: -

HierarchyLayout newLayout = new HierarchyLayout (document, ultimateParent, alignment);

- Invoke addNodeHierarchy: -

newLayout.addNodeHierarchy (ultimateParent, “”); // 2nd argument is yet to be utilized

This invocation would assign a value to the node coordinates (node.coords.x, node.coords.y) for each node in the hierarchy according to the passed alignment.

- Invoke renderHierarchy on a node: -

node.renderHierarchy ();

This invocation would render the immediate child nodes considering “node” as the parent.

The library can be downloaded from - Layout Lib

You can see a demo-usage @ SoNet home under “Demo” –> “Layout service demo”.

Please feel free to drop a note in case you’ve comments / feedback.

Thus, a generic XML-generation framework is desirable that could return precise XML files by consuming a schema definition(specifying generic XML structure) and an input configuration (containing information specific to a particular XML-generation instance).

Input(s) to the framework: Schema Definition (xsd), Configuration (.xml), Input Parameters (HashMap ), Resource Specification [Specifies the system-specific entity for which the xml is to be generated].

Solution Design: The core idea is pretty straight-forward and comprises of the following steps: -

[1]. Create an in-memory parse-tree out of the given .xsd

[2]. Use the resource specification to refer the configuration and find out ‘the information’ specific to that particular XML-generation instance

[3]. Use the above ‘information’ to find out the scope of the current instance of xml generation from within the entire parse-tree

[4]. Adorn that scope with the input parameters and recursively apply ‘the information’ to generate the actual XML file

Description:

[1]. Create an in-memory parse-tree out of the given .xsd - For this, one can either go with any of the open-source xml-parsers, or write from scratch. Writing from scratch would generally require Element, Node, Attribute and ComplexType classes along with their respective handlers. Then, you would’ve a Schema Parser, which would utilize instances of those classes and generate the final tree object.

I won’t elaborate further on this here; if required, please drop me a note. In essence, the only contract to be followed is that the return object should contain a parse-tree structure.

[2, 3 & 4]. To understand these, let us delve a bit into the source.

Our generator would generally need to follow something similar to the following contract: -

/**
* IXMLGenerator represents the contract that all XML generators must adhere to
* in order to support the default logic of XML genertion. Default XML generation
* contract is based on the consumption of the starting XML element
* and the consolidatedMap of input and configuration parameters to
* generate the populated response XML for that particular element.
*
* @author sushain
*
* @version 1.0
*/

public interface IXMLGenerator
{ /**
* This method takes a resource name, corresponding method and a
* parameter map and builds a request xml
* which can be consumed by the a request handler.
*
* @param resource
* @param method
* @param paramMap
*
* @return Generated request xml
*/

public static String generateXml(final String resource,
final String method, final HashMap paramMap)
throws SchemaParsingException;
}

Here, generateXml would be the main method, inherently calling the tree-creation module, and effectively completing step (1). This method would utilize the input-parameter map (HashMap containing the name-value pairs for the elements to be populated in the xml), resource (the entity for which the xml is desired) and method (the type of xml desired {system-specific}) by referring the ‘Configuration’ in the following manner: -

Now, I’ve repeatedly referred to a ‘Configuration’ above, lets see how that actually looks like: -

<configuration>
<resource name=”<system-defined name>”>
<method name=”<resource access method / CRUD operation>”>

<param name = “aaa” value=”val1″ />

<param name = “bbb” value=”val2″ />

<param name = “ccc” value=”val3″ />

<!–param name = “<complex_aaa>” value=”<attr_val1>,<attr_val2>” /–>

<!–param name = “<complex_bbb>” value=”<attr_val3>,<attr_val4>” /–>

</method>

.
.
.

Some key-points about the above conf are: -

- Each resource-name has a list of parameters associated with it that would define what exactly its associated XML would consist of.

- In case of complex-type params, the value consists of a list of comma-separated attributes.

- Such a definition is readable, easily extensible and configurable.

We would also have a corresponding Config class to read the above XML configuration into a Document (org.w3c.dom.Document) and build a hashMap out of it.

The structure of such a class would be similar to: -

public class Configuration {

private File configFile;

private Document configDoc;

private String resultNodeName;

public Configuration(String configText){

this.configFile = new File (configText);

this.configDoc = getConfigDocument();
}

public Document getConfigDocument() {

Document XMLDoc = XMLutils.getXMLDocument(configFile);

if (XMLDoc == null) {

System.out.println(”XMLDoc null”);

System.exit(-1);
}

String query = “/configuration”;

NodeList nl = (NodeList) XMLutils.runXPathQuery(XMLDoc, query);

if(nl.getLength() !=1 ) {

return null;
}

Document responseObject = XMLutils.getNodeAsDocumentRoot(nl.item(0));

return (responseObject == null) ? null : responseObject;
}

public String getResultNodeName() {

return resultNodeName;
}

public HashMap getConfigurationMap (final String resourceName, final String methodName) {

HashMap configObject = new HashMap ();

Node temp = this.configDoc.getDocumentElement().getFirstChild();

while(temp != null) {

if(temp.getNodeType() == Node.ELEMENT_NODE) {

if (temp.getAttributes().item(0).getNodeValue().trim().equals(resourceName)) {

NodeList methodList = temp.getChildNodes();

int methodListIndex = 0;

while (methodListIndex < methodList.getLength()) {

if (methodList.item (methodListIndex).getAttributes() == null) {

methodListIndex = methodListIndex + 1;

continue;
}

if (methodList.item (methodListIndex).getAttributes().item(0).getNodeValue().trim().equals(methodName)) {

NodeList paramList = methodList.item(methodListIndex).getChildNodes();

int paramListIndex = 0;

while (paramListIndex < paramList.getLength()) {

if (paramList.item(paramListIndex).getAttributes() == null) {

paramListIndex = paramListIndex + 1;

continue;
}

configObject.put(paramList.item(paramListIndex).getAttributes().item(0).getNodeValue().trim(),
paramList.item(paramListIndex).getAttributes().item(1).getNodeValue().trim());

paramListIndex = paramListIndex + 1;
}
}

methodListIndex = methodListIndex + 1;
}
}
}

temp = temp.getNextSibling();
}

return configObject;
}

}

- Once such a setup is in place, the generateXml method would then build a ‘Consolidated Map’ by appending ParamMap to HashMap.

- It would then read the xsd definition and get the in-memory parse tree ready using the module discussed in step (1).

- After that the actual xml generation logic would be invoked, passing the above built Consolidated Map and the resource specifiction.

- The generation logic would iterate through the tree, until it locates the resource spec

- It would then start building the XML, simultaneously referring the Consolidated Map and populating elements whenever it finds a corresponding entry in the Map. The population logic would be written in the corresponding handlers for ’simple’ as well as ‘complex’ type nodes.

This would be something similar to: -

if(attList.size() > 0) {

XSDAttribute att = (XSDAttribute)attList.get(0);

if (consolidatedMap.containsKey(getName())) {

attributeString = consolidatedMap.get(getName()).toString();

StringTokenizer stok = new StringTokenizer (attributeString, “,”);

while (stok.hasMoreTokens()) {

buffer.append(”<”);

buffer.append (getName() + ” “);

buffer.append (att.getName());

buffer.append(”=”);

buffer.append(”"” + stok.nextToken() + “”");

buffer.append(”>”);

if (consolidatedMap.containsKey(att.getName())) {

buffer.append (consolidatedMap.get(att.getName()));
}

buffer.append(”

buffer.append(getName());

buffer.append(”>”);
}

}

return buffer;

Finally, this XML would be returned to the caller and subsequently consumed by the original framework. For extension purposes, the only file to be modified later would be the configuration, which could be easily achieved due to the uniformity and structured nature of the definition. Due to a generic contract specification that only mandates an input parameter-map with name-value pairs of population-parameters, such a framework could be utilized in applications with a consuming UI-front as well as headless command-based apps. We thus saw how an effective configuration-design simplifies a seemingly difficult extension & integration problem.

If you have any suggestions, please drop a note / post.

- alphaworks.ibm.com/tech/mashuphub

- services.alphaworks.ibm.com/damia

- services.alphaworks.ibm.com/qedwiki

^ Excellent assembly canvasses and services to create holistic aggregations and their respective interface components, and in turn giving us yet another instance of an effort towards the unification of the random segments.

IDE used: -

Rational Software Architect (RSA) and Rational Application Development (RAD) Platform – built on Eclipse 3.0

Plugins: -

- Team – As a repository tool for consistent change management
- JDT Core - Just touched it once to re-install the code assist support
- org.eclipse.ant.core
- org.eclipse.pde.core
- org.eclipse.tomcat
- Most other preinstalled plugins in the .metadata/plugins folder in RSA (/plugins folder in Eclipse)

File-types usually touched: -

.java, .xml, .properties, .jsp, .js, .sh, .bat, etc and some project-specific formats

Process followed when a fresh workspace is given: -

- To start with, shift the dock from the bottom position to leftmost.

- After this, dock all the development-related views like (pakage explorer, navigator, outline, hierarchy et al) onto this.

- Doing this (above explained docking) gives the entire width of screen for code-writing purposes. Also, this enables toggling any other view by single mouse-click.

- Keep server, problems, and console views docked at the bottom, since they’re needed most of the time.

- After this, open the required perspective – java, j2ee, or debug.

- Un-check the ‘build automatically’ option in the Project-menu because you don’t want the ws to build every-time you change a single line.

- If required, go to Preferences – Java – Code Style and change the formatting to what you usually practice, or to the standard that is followed at the organization / client working for.

- The next activity is creating the required CVS repositories and checking-out the code.

- Follow this by a build check to see if all the required projects are checked-out and resolved all dependencies. A clean build would ensure that.

- Next, do the database setup. Run the associated scripts for schema generation and data population.

- After this comes the server configuration part. Depending on whether the server is Tomcat, Websphere Application Server or JBoss, follow different set of steps.

- In case there is an automation plugin available, last four steps could be made redundant.

- After this, prepare the main Ear or War for deployment, if required, by right-clicking and selecting ‘Prepare for deployment’. This would generate deployment code for all the projects specified within the application.xml / web.xml.

- Following this, do a clean-buildall again and finally publish the .ear onto the server.

- Run the sanity-test bucket to see that the code hasn’t broken any existing stuff.

- Follow this by unit-test creation according to a test-case matrix provided by a business-analyst or yourself, and then running those.

- After this, if lucky, you would do the code sync-merge-checkin, or would’ve to switch to the debug-mode :-)

- After code check-in, ideally, sync the workspace again with Head-stream and do the above steps till sanity-check to ensure everything is working perfectly.

- If any new plugins are installed, start RSA with –clean option to clear the registry et al.

There are many other things to be done and points to follow in order to ensure that the development process is smooth, but the above description shall give a good enough idea to save me.. in case the thunder strikes.

[Disclaimer: If I goofed-up any of the steps, blame me not.. it would simply imply that thunder struck early !

Let us start with an example scenario whence a top-notch scientist working in a world-class firm makes a ground-breaking discovery, leading to an equally precious idea and associated methodology to implement the idea.

He knows that ideas are meant to be patented and thus he files a disclosure with his local Invention Development Team (IDT). Now, this IDT consists of equally capable guys, with a decent patent-portfolio and a healthy disdain for jack-in-the-office patent wannabes. They begin to scrutinize the disclosure as usual.

They go through the existing and easily-accessible prior-art and find some seemingly similar implementations. Thus, they immediately publish a questionnaire soliciting a response. Now, our dear scientist, being quite busy with putting new ideas and draft disclosure in pipeline, misses the solicitation deadline given, and gets a note with the subject - ‘Disclosure Rated Close’ in his mailbox after some days.

Panic struck, he immediately writes to IDT expressing his distress and feeling of injustice et al. Soon after, he realizes his sloppiness and writes another apologetic note asking for a re-consideration of deadline and disclosure. He spends that night sorting out the core idea and drafting the answers to the questionnaire.. with his mind already messed-up with new ideas. He finally strikes gold and publishes his detailed explanation requesting IDT to revoke the ‘close decision’ on the application. After much deliberation, he finally succeeds and the disclosure status again changes to ‘under evaluation’. Rejuvenated with his cock-a-doodle of victory, he starts following-up with IDT at regular intervals, dropping notes and giving an impression as if he is next-in-line after Euler, Newton and Einstein.

Anyhow, the IDT guys, after many more comparisons and explanations aren’t satisfied with whatever prior-art they evaluated the idea against, and give the decision as ‘Search’. Now, this doesn’t go down too well with the scientist guy. Afterall, if you ask him, he had manually searched every single entry on Goog, and found nothing even remotely similar.

Anyhow, he starts working with the IDT; IDT guys giving him a new paper every other day to evaluate and he making sure that he points out sufficient differences between that and his own disclosure. This process goes on for sometime, after which, the IDT folks start to give that one-in-a-blue-moon feeling of getting impressed. But now, they face another dilemma. They were pretty convinced that the idea was unique and the implementation novel, unrelated to anything done before, but the patent, if filed, wouldn’t have business value.

Now, the scientist guy is inconsolable.. “Business Value ? But this is a beautiful innovation.. Novelty, Uniqueness, Rigor.. everything in perfect proportions.. just patent it, it would create a new business for you.”

It would not however benefit the existing business in any way. “The revenue that the patent, if filed, would generate on an average after ‘n’ number of years and after going through ‘t’ lawsuits doesn’t weigh well against the cost of the patent application (cost of filing + cost of effort + cost of time spent) and the cost of those ‘t’ lawsuits and the cost of establishing a new business to accommodate your invention.”

In the hindsight, these explanations seem pretty rational. But the scientist guy still reasons - “Okay, so you don’t want to file an application now, no problem.. I can wait till we get a suitable business, and we’ll then file an application.”

“Ah well, your invention is valuable and has got all the limelight and hype because ‘it is unique right now’. We can wait, but then you’ll risk someone else making the same discovery and filing a patent application, in which case, your efforts will be rendered useless.”

The scientist snapped and took a moment to think about 6 billion people, who at that moment, seemed all capable of making the par and taking away his prized possession. “So, what is the way out ?”, asked the Scientist guy.

“Well.. lets publish this defensively, which would mean publishing this in an open database accessible to all patent attorneys. That way, we preserve our right of using this and no one else can patent this in future. So, from then on, this becomes a piece of un-patentable, public knowledge. You will of course retain the major authorship and your organization will become the disclosing entity. The only catch is that since it’ll be public, we won’t be able to generate any revenue through lawsuits.”

To the scientist, this is a revelation - Unification of business and technological brilliance. Feeling of maintaining the celibacy of a brain-child forever. An equilibrium. And thus, he decides to rest his case.

[Disclaimer: All the above images are copyright of phdcomics.com and have been referred for non-commercial use]