Framework Description: The framework essentially comprises 4 parts.

1.) TreeNode and DataPoint classes to hold hold data and define tree nodes: -

DataPoint contains an attribute array to hold the values for a particular node and is contained within each TreeNode instance. The structures for these classes are as follows.

class TreeNode {

public double entropy;
public Vector data;
public int decompositionAttribute;
public int decompositionValue;
public TreeNode [] children;
public TreeNode parent;

public TreeNode () {
data = new Vector ();
}
}

class DataPoint {

public int [] attributes;
public String label;

public DataPoint (final int numattributes) {
attributes = new int [numattributes];
}
}

2.) DecisionTree: -

This is the base class for decision-tree implementation and has a containment relation with the TreeNode class. It encapsulates all the implementation details for decision tree creation and entropy calculation. It uses the max-gain criteria for splitting over nodes. Right now, pruning hasn’t been implemented.

The field-definitions for decision-tree is as follows.

public class DecisionTree {

private int numAttributes;
private String [] attributeNames;
private Vector [] domains;
private TreeNode trainingRoot;
private TreeNode testingRoot;
private String namesFile;
private String trainingDataFile;
private String testingDataFile;
private int correctCount;
private int inCorrectCount;
private boolean debug;
private int maxDepth;
….

}

The decision-tree class invokes the InputProcessor (explained below) , which populates the training and testing Roots with input data. After this, it invokes createDecisionTree and induce modules to the create the actual tree.

The overall structure of induce is as follows.

public void induce (final TreeNode node) {

double bestEntropy = 0;
boolean selected = false;
int selectedAttribute = 0;
int numdata = node.data.size ();
int numinputattributes = numAttributes - 1;
node.entropy = calculateEntropy (node.data);
if (node.entropy == 0) {
return;
}

for (int i = 0; i < numinputattributes; i++) {
int numvalues = domains[i].size ();
if (alreadyUsedToDecompose (node, i)) {
continue;
}

double averageentropy = 0;

for (int j = 0; j < numvalues; j++) {
Vector subset = getSubset (node.data, i, j);
if (subset.size () == 0) {
continue;
}

double subentropy = calculateEntropy (subset);
averageentropy += subentropy * subset.size ();
}

averageentropy = averageentropy / numdata;

if (selected == false) {
selected = true;
bestEntropy = averageentropy;
selectedAttribute = i;
} else {

if (averageentropy < bestEntropy) {

selected = true;
bestEntropy = averageentropy;
selectedAttribute = i;
}
}
}

if (selected == false) {
return;
}

int numvalues = domains[selectedAttribute].size ();
node.decompositionAttribute = selectedAttribute;
node.children = new TreeNode [numvalues];
for (int j = 0; j < numvalues; j++) {
node.children[j] = new TreeNode ();
node.children[j].parent = node;
node.children[j].data = getSubset (node.data,
selectedAttribute, j);
node.children[j].decompositionValue = j;
}

for (int j = 0; j < numvalues; j++) {
induce (node.children[j]);
}
node.data = null;
}

The most important function call within induce is to calculate the entropy and the structure for that is given below.

public double calculateEntropy (final Vector data) {

int numdata = data.size ();
if (numdata == 0) {
return 0;
}

int attribute = numAttributes - 1;
int numvalues = domains[attribute].size ();
double sum = 0;

for (int i = 0; i < numvalues; i++) {
int count = 0;
for (int j = 0; j < numdata; j++) {
DataPoint point = (DataPoint) data.elementAt (j);
if (point.attributes[attribute] == i) {
count++;
}
}

double probability = ((double) count) / numdata;

if (count > 0) {
sum += -probability * Math.log (probability);
}
}
return sum;
}

3.) InputProcessor: -

This is used for all the input processing and in turn, for node data population and is utilized by DecisionTree to get the populated Node Objects from raw input data. This is a fairly complicated class with different modules to read the attributes and the actual data. Besides, it contains the functionality to do a random split between training and test in case only one data-set is provided & to discretize the linear data into bins.

The key module is readDataSets.

public int readDataSets (final String fileName, final TreeNode root, final String toSplit) throws Exception {

FileInputStream in = null;
/************************************************
* Reading the Training / Testing data set File *
************************************************/
try {
File inputFile = new File (fileName);
in = new FileInputStream (inputFile);
} catch (Exception e) {
System.err.println (”Unable to open file: ” + fileName + “n” + e);
return -1;
}

BufferedReader bin = new BufferedReader (new InputStreamReader (in));
String input;
int index = 1;
while (true) {
input = bin.readLine ();
if (input == null) {
break;
} else if (input.startsWith (”|”)) {
continue;
} else if (input.contains (”|”)) {
input = input.substring (0, input.indexOf (”|”));
}
if (input.equals (”")) {
continue;
}

StringTokenizer tokenizer = new StringTokenizer (input, “,”);
int numtokens = tokenizer.countTokens ();
if (skipCount > -1) {
if (numtokens != numAttributes + 1) {
return -1;
}
} else if (numtokens != numAttributes) {
return -1;
}

DataPoint point = new DataPoint (numAttributes);
// if there is no label to skip
if (skipCount == -1) {
point.label = “Example#” + index;
for (int i = 0; i < numAttributes; i++) {
point.attributes[i] = getSymbolValue (i, tokenizer.nextToken ());
}
} else if (skipCount > -1) {
int attributeIndex = 0;
for (int panditIndex = 0; panditIndex < numAttributes + 1; panditIndex++) {
// assign label to the data point and skip it as an attribute field
if (panditIndex == skipCount) {
point.label = tokenizer.nextToken ();
continue;
}

point.attributes[attributeIndex] = getSymbolValue (attributeIndex, tokenizer.nextToken ());
attributeIndex = attributeIndex + 1;
}
}

/************************************************
* Required 2/3-1/3 random data split follows *
************************************************/
if (toSplit.equals (”SPLIT”)) {
double randomNumber = 3 * Math.random ();
if (randomNumber > 2) {
testingRoot.data.addElement (point);
} else {
root.data.addElement (point);
}
} else {
root.data.addElement (point);
}
index = index + 1;
}
in.close ();
bin.close ();
return 1;
}
4.) DisplayProcessor:-

This class provides the key implementation for handling the display functionality. The output formatting is similar to what you see here. The key module is displayTree which displays each node-level recursively.

public void displayTree (final TreeNode node, final String tab, final int numAttributes, final String [] attributeNames, final Vector [] domains, final int depth) {

int outputattr = numAttributes - 1;
if (node.children == null) {
int [] values = getAllValues (node.data, outputattr, domains);
if (values.length == 1) {
System.out.print (” ” + domains[outputattr].elementAt (values[0]));
return;
}

System.out.print (” {”);
for (int i = 0; i < values.length; i++) {
System.out.print (”"” + domains[outputattr].elementAt (values[i]) + “”");
if (i != values.length - 1) {
System.out.print (” , “);
}
}

System.out.print (” };”);
return;
}

int numvalues = node.children.length;
for (int i = 0; i < numvalues; i++) {
System.out.print (”n” + tab + attributeNames[node.decompositionAttribute] + ” = ” + domains[node.decompositionAttribute].elementAt (i) + “:”);
if (depth <= this.MAX_DEPTH) {
displayTree (node.children[i], tab + “: “, numAttributes, attributeNames, domains, depth + 1);
} else {
System.out.print (”…”);
}

if (i != numvalues - 1) {
System.out.print (”t”);
} else {
System.out.print (”");
}
}
}

Again, this can be easily modified or extended as needed since the display functionality is cleanly decoupled from decision-tree creation logic and other parts explained above. Finally, in case you require the complete framework implementation alongwith the data-sets and instructions, please drop me a note explaining the purpose and need for it and I shall get back to you.

After a lot of hassle, I was finally able to resolve it and everything seems to be working alright since then. Nehow, I thought of putting down my observations here for anyone who may face similar issue with their installations or otherwise.

- The obvious things first - I checked the permissions on the installation folder (for me, <root>/blog) and found it to be perfectly ok (755 for read / execute to world).

- Next I checked the .htaccess in the root and so I found the problem. I had an earlier rails (Ruby on Rails) installation and following lines were present in it: -

RewriteEngine on

RewriteCond %{HTTP_HOST} ^sushain.com$ [OR]
RewriteCond %{HTTP_HOST} ^www.sushain.com$
RewriteRule ^contest http://127.0.0.1:12009%{REQUEST_URI} [P,QSA,L]
DirectoryIndex preload.html index.html
# BEGIN WordPress

# END WordPress

I immediately knew that the url rewriting module was causing some issue here, but I didn’t want to tinker with this since I may want my Ruby app (although dormant for now) to come alive again in future. So, what I did instead was that I changed the .htaccess file in the /blog folder from -

DirectoryIndex index.php
AuthUserFile “/home/sushainp/.htpasswds/public_html/blog/passwd”

to -

DirectoryIndex index.php
AuthUserFile “/home/sushainp/.htpasswds/public_html/blog/passwd”
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /blog/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /blog/index.php [L]
</IfModule>

# END WordPress

This overrode the url rewriting mechanism for /blog level addresses and thus, solved the issue.

Btw, for those of you interested in knowing more about url rewriting module, more information is available here.

Cheers !

Solution options: Well, I tried out many libraries and finally used none. In any case, here is my experience.

1.) Ziplib library from phpclasses - Very powerful library, can be used for recursively adding folders to an archive, etc. Just seemed to be too complicated for my task.

2.) Lib by devco.net - Simple usage and fairly good lib. This is how I tried using it: -

require (”pclzip.lib.php”);
$exp = “63″;
$zipfile = new PclZip(”$exp”.”_pack.zip”);
error_reporting(E_ALL);
$output = shell_exec( “cp index.php 63″ );
$v_list = $zipfile->create(”$exp”);

The issue I had with this one was that it didn’t allow me to add subdirectory structures selectively, or at least I couldn’t fig it out rightaway.

3.) Yet another library, this one was by Drupal - Seemed similar to 1.) and again, I just realized mid-way that I needn’t have that much functionality at all and wanted something simpler.

4.) Another one by granthinkson - Neat library, esp if you want to get all dirty with the actual file-creation and understanding process. I just wanted my zip :sigh:.

5.) Yet another by Techrepublic - Again, simple usage and I would’ve almost gone with this, but for point (6.) below. But, in any case, fairly easy to use this -

$obj = new Archive_Zip(’63.zip’);
$files = array(’63.soft’,'63.chp’, ‘63.cel’);
$obj->create($files);

6.) Well, dunno why, but I had my usual whim and I suddenly decided not to go with any of these and resorted back to plain old ‘jar’ util. So, this is what I did finally, esp since our server is configured with a jre anyways.

/* Modified: 1-1-09 | code for zip creation - pandit */
…..
$zipfilename = “$exp”.”_pack.zip”;
……
if(!file_exists(”$tmp_dir/$zipfilename”)) {
$src = “$root_dir/ftp/pub/$exp/*”;
$dest = “$tmp_dir/$exp”;
exec( “cp $src $dest” );
exec (”jar cfM $tmp_dir/$zipfilename -C $dest .”);
}
……
$zip_link = $base_url . “/$zipfilename”;

But, thats so trivial, said Watson.. Yup, everything becomes trivial after explanation, sighed Holmes.

Anyhow, I implemented it here

In this piece, I try to analyze the refutations (against various arguments) given by Turing in his classic article on the ‘Imitation Game’, and determine what still holds given the present state of AI. The following arguments and analysis are with reference to the original Turing Article located at - http://www.loebner.net/Prizef/TuringArticle.html Towards the end, I present my views on the current state of success at passing the Turing Test and what future warrants on the subject.

1.) Theological objection: - To me it seems that the theological arguments have always banked on faith rather than being based on logic and reason. However, this objection may still carry some weight to those who can define and clarify the idea of a soul and then be able to link this idea to thinking process. Without these definitions, linking the thinking process with man’s immortal soul is a highly generalized assumption based on the notion that an all-powerful God is able to somehow put this into effect. However, this very assumption is contradicted later on by claiming that the same God can’t put those souls somewhere else, in this instance - machines. Overall, this objection seems too pretentious to carry substantial weight now. Moreover, Turing’s refutations in the original paper seemed to be quite rational to put this case to rest.

2.) The “Heads in the Sand” objection: - This objection in itself (as stated in the original paper) carries no weight now. However, it is somewhat related to the ethical issues that are now often connected with AI, when we pause considering whether we can develop AI, and ask ourselves - Should we create AI and ponder upon the various implications of advanced / strong AI if it were to co-exist with humankind. But, since the current discussion is about ‘Can machines think’, we would rest this case here.

3.) The Mathematical objection: - Actually, this argument doesn’t really seem like a reason to object the test since it tentatively puts machines only at a disadvantage w.r.t humans and thus this test may be a decent criteria to determine the validity of that objection. Anyhow, as far as computing capacities are concerned, we’ve reached very high orders from what was available back in Turing’s time. Thus, ideally, such machines can be build which would decently respond to the objections in this point.

4.) The argument from consciousness: - In his original paper, Turing has handled this argument from the perspective of feeling as it relates to thinking and he describes this objection as - ‘whether machines can feel that they are thinking‘. Within this context, his refutations seem valid since he links these objections to solipsism (claim that only self can be verified), which he then claims is noone’s win since it makes idea communication difficult.

However, there is a broader context to this argument now, especially after thought experiments like Chinese Room by Searle - http://en.wikipedia.org/wiki/Chinese_Room . These experiments broaden the scope of the argument by relating consciousness to mind, bringing the notion of intentionality & states, and illustrating that machines can’t have minds irrespective of their intelligent behavior.

One may argue that Turing would have reasoned that even the neurons within human brains aren’t conscious in the sense of understanding what they are doing, and it somehow appears that the overall system as a whole is behaving intelligently and there is nothing more to it than appearance. However, since these ideas are not the part of his original paper, these objections would still hold.

5.) Arguments from various disabilities: - This argument is essentially a combination of various objections and expectations each of which are discusses below. For many of these, we would require an exact definition of the word to relate it to AI and for this purpose, we would refer this source - http://www.answers.com : -

- Being kind, friendly, loving: These are abstract words that require contextual definitions. If these are implied by being helpful and useful, then certainly this objection will not hold since there are many machines, softwares etc, that help people automate their work. Applications of AI to advanced robotics has produced many helpful robots - http://www.helpfulrobots.com/ .

However, if we think of kindness, friendship and falling in love from a conscious point of view, then it would again require a mind and intention. It would thus fall into the consciousness argument discussed above.

- Being resourceful: As per our source, resourceful means - able to act effectively or imaginatively, especially in difficult situations. Clearly, most of the existing softwares meet the criteria of efficiency. There are various agent programs that deal with different situations intelligently as well - http://en.wikipedia.org/wiki/Windows_Live_Agents . Imagination is again dependent on how we define it, but overall this criteria seems to be met now.

- Being beautiful: If this concerns external beauty, there are certainly beautiful machines created by careful manufacturing, etc. Internal beauty is again abstract and would belong to the consciousness argument, as in being aware of an internal state.

- Having initiative: Most AI machines or softwares have been created with an ultimate goal to take the initiative from humans and automate processes.

- Having a sense of humour: There have been some efforts towards artificial comedy - http://www.clone3d.net/ .

- Telling right from wrong and learning from experience and doing something really new: Machine learning and causalreasoning are two of the core fields in AI research and have been implemented in many systems. Doing something new can be a direct consequence of machine learning or learning from experience. So, this objection doesn’t hold now.

- Making mistakes: Since AI programs are written by humans, they can be buggy and hence their actual output can deviate from expected. This can be classified as making mistakes.

- Making someone fall in love: This is quite achievable using meticulous and aesthetical design, and giving a pleasing appearance to a useful machine.

- Using words properly: NLP and voice recognition programs are a case in point.

- Having as much diversity of behavior as a man: This objection still holds and would continue to hold until certainmilestones are reached in brain and mind simulation .

- Being the subject of its own thought: Thought requires a clear contextual definition here, but, in general, this could be similar to providing a program source as an input to itself and observe the output. Again, doable and hence the objection doesn’t hold.

6.) Lady Lovelace’s Objection: The gist of this objection can be summed up in one sentence - machines can never do anything really new so as to take us (humans) by surprise and Turing gave a classic refutation to this by claiming that this is equivalent to saying that there is nothing new under the sun since every new discovery or invention is linked to some past event or chain of events. So, basically he tried to give a context to what should be considered as really new and within that context, this objection is clearly not true today. There are programs capable of giving surprises as well as creating unique and new scenarios. A case in point is the origin of Butterfly effect - http://en.wikipedia.org/wiki/Butterfly_effect .

7.) Argument from Continuity in the Nervous System: This argument needed refutation on grounds of fairness in the original paper at the time Turing wrote it. However, now with neuro-computing models in existence, this argument can be made redundant in principle.

8.) The Argument from Informality of Behavior: The crux of the argument is - If each man had a definite set of rules of conduct by which he regulated his life, he would be no better than a machine. But there are no such rules, so men cannot be machines. Turing refutation on this still holds since we still haven’t found the absolute laws of behavior that regulate life, something similar to the M-theory, or the theory of everything - http://en.wikipedia.org/wiki/M-theory .

9.) The Argument from Extrasensory Perception: The argument itself seems pretty weak and containing very less substantiation. There are varied opinions on phenomena like telepathy, precognition, etc, and like the argument the refutation is also pretty weak (telepathy-proof room). Anyhow, some people may still have this objection, however it remains a very subjective issue.

New Issues: -

Inherent problems with the Turing Test and Comprehensive Turing Test:

With the advancement of AI to fields like machine vision and perception, one may demand a more comprehensive test. This is since humans generally don’t use question-answering as a means to find whether someone is able to ‘think‘. We use more simplistic (or elaborate, if we consider machines) methods like sight, sound of voice, etc and our notion of thinking is implied by many other apparent things than just plain response. A case in point are the systems like CAPTCHA - http://www.captcha.net/ . This brings back the old issue on whether the Turing Test is a real test of intelligence at all. It seems that the inherent problem with the test is that a program can fool an interrogator by playing on his psychology. This task is much more easier and doesn’t definitely require intelligence equivalent to the mind, or thought process. A simple re-use of the question-phrases can be used as an effective way of letting an interrogator know that the program is actually a person belonging to a particular genre. A case in point is Eliza - http://en.wikipedia.org/wiki/ELIZA . Of late, there have been similar concerns raised over expertise of judges in the Loebner competition. As far as the current state of success with passing the Turing Test is concerned, it may be noted that Loebner is the only official competition for judging this and most of the effort in AI is actually not focused on passing this test. This current state of AI may not have a one-to-one co-relation with the whether this test has been passed or not. However, current state of AI does give us an idea regarding whether this test can be passed if all effort was directed towards it. The 2007 winner for Loebner award was UltraHal, which is basically a software assistant and was re-programmed to enter the competition so that it could specifically fool the judges (ref the interview here - http://aidreams.co.uk/forum/index.php?page=67 ) . So, again the chance of fooling a judge is subjective and dependent on the skill of judge as described above in the case of Eliza bot. It may be argued that the Turing test has already been passed by some programs since they fooled a set of judges in Loebner, but the credibilty of judges would always be in question and so will be the exact rules and regulations under the competition was held. In all, I would say there’s a chance of about 25% for a machine to pass the test, considering strict Turing guidelines and maybe in next 50 years, we would make substantial advances in allied fields of AI to eventually pass the test satisfactorily, however, this again depends on how much effort is put in actually solving this problem in specific.

Managing information is one of the most fundamental activities done in any enterprise, irrespective of the size, scale, or purpose. Thus, it becomes an obvious and essential need to find ways and develop notions & techniques that would make this activity easier, intelligible, coherent and accessible. There are scores of issues that come under the information management umbrella, right from generation / aggregation through transformation to consumption / presentation. In essence, Info N.0 tries to capture and in turn define information management using contemporary XXX N.0 technologies, like Web N.0, Enterprise N.0, RSS N.0.

Now, Info 3.0 group is meant to explore the notions, possibilities and issues related to the next generation of information management, encompassing Web 3.0 / 2.0 technologies, Enterprise 2.0 and the role that Artificial Intelligence may play in the eventual evolution of Info N.0. It would be pertinent to quote that right now the definitions for this are very fuzzy, specifications are absent and thats why the purpose of this group is to evolve a framework, through a series of blog-posts and retrospection, such that one could look at it and get an idea on the scope and possibilities of Info N.0

I always believe that a shared-blog or a wiki is the perfect place to evolve ideas; so let us come up with whatever thoughts, ideas, or prior experience we have had related to Information Management in general, or specifically related to Info N.0. As is the case with any community, I won’t be posting here all the time, so just drop me a note at - sushain.pandit@gmail.com and I would be more than glad to add you to the author’s list.

Lastly, pl let me know in case you’ve any comments or sugg on how this should be done in a better way, (for instance start a wiki instead of a blog et al). So, once again welcome to the group and rock on !

- Indexing: Most hyped is the size of its massive index (~120 billion pages), as also quoted in their info page - “Cuil searches more pages on the Web than anyone else—three times as many as Google and ten times as many as Microsoft”. However, I didn’t find any conclusive proof for this in that Google still listed more total hits on most of my test-searches. [Example SearchKey = 'Sushain': Cuil - 662 | Google - 929]

Now, this may still improve with time, so we’ll let this one pass.

- Renewal rate: Now, they might have indexed more pages than Google but their index still doesn’t contain recent links. Indexing recent content is really critical to search engines and again, they should improve on this front with time.

- Page ranking: The info page also says - “Rather than rely on superficial popularity metrics, Cuil searches for and ranks pages based on their content and relevance. When we find a page with your keywords, we stay on that page and analyze the rest of its content, its concepts, their inter-relationships and the page’s coherency.”

The issue I found was that sometimes the searches listed ‘not-so-important’ results before more-relevant ones. For instance, listing the pages linked to someone’s friend (via a social network) before actually putting the person’s site-listing etc. In my case, it shows my name being present in my friend’s HiFi n/w, before pointing anything relevant to me (and I haven’t put a robots.txt file on my website yet to stop crawlers :-)).

- Infrastructure: Many a times, during my experimentation, Cuil would just give a ‘We’ll be back’ message, or wouldn’t repeat the results that were generated for the same query, some time ago. Again, there’s room for improvement and they would do well to evolve with time.

- Layout: I’ve personal hots for dark (energy-saving) backgrounds and Cuil scores there. Even otherwise, the results are laid out in a better way and are easy to spot, but again, I think I’ve seen this before on proprietary search-engines.

- Privacy: Cuil does claim to be the good-boy search engine n not track the data et al, but the bottom-line still is how good the search capability is.

Final verdict: Well, I’m pretty much in favor of any new-comer into a partially monopolized domain (Though I love Google as much as anyone :-)). The only issue I’ve is that they really could’ve done some more ground-work before making this live (Links on Info and Privacy weren’t working till yesterday) since the first impression really goes a long way. All in all, I think, we need to give more time to Cuil; No enterprise gets build in a day and Cuil should be appreciated to have at least taken an initiative in an otherwise one-firm show.

Description: The framework is developed as a javascript service that defines vertical / horizontal hierarchical layout method handlers. There are different handlers based on whether one wants to organize divisions (div tags), separate nodes, or node hierarchies. Apart from the structuring logic, the file also includes the rendering logic that paints the node(s) and their associated relations to build the actual hierarchy.

Now, the layouts are provided out of the box and all one requires is a nodelist (supporting composite pattern), which will then be consumed to draw the actual hierarchy. Such hierarchies can be utilized for a particular filtered view within a social networking graph.

The node definition(s) should comply to the following generic template to be able to fed and subsequently consumed properly by the framework: -

var Node = function(id) {
this.id = id;
this.coords;
this.shape;
this.text;
this.group;
this.childList = new dojox.collections.ArrayList();
}

Illustration: -

- Get a hierarchy object (pass the ultimate parent of the node hierarchy and specify the alignment [horizontal / vertical]: -

HierarchyLayout newLayout = new HierarchyLayout (document, ultimateParent, alignment);

- Invoke addNodeHierarchy: -

newLayout.addNodeHierarchy (ultimateParent, “”); // 2nd argument is yet to be utilized

This invocation would assign a value to the node coordinates (node.coords.x, node.coords.y) for each node in the hierarchy according to the passed alignment.

- Invoke renderHierarchy on a node: -

node.renderHierarchy ();

This invocation would render the immediate child nodes considering “node” as the parent.

The library can be downloaded from - Layout Lib

You can see a demo-usage @ SoNet home under “Demo” –> “Layout service demo”.

Please feel free to drop a note in case you’ve comments / feedback.

Thus, a generic XML-generation framework is desirable that could return precise XML files by consuming a schema definition(specifying generic XML structure) and an input configuration (containing information specific to a particular XML-generation instance).

Input(s) to the framework: Schema Definition (xsd), Configuration (.xml), Input Parameters (HashMap ), Resource Specification [Specifies the system-specific entity for which the xml is to be generated].

Solution Design: The core idea is pretty straight-forward and comprises of the following steps: -

[1]. Create an in-memory parse-tree out of the given .xsd

[2]. Use the resource specification to refer the configuration and find out ‘the information’ specific to that particular XML-generation instance

[3]. Use the above ‘information’ to find out the scope of the current instance of xml generation from within the entire parse-tree

[4]. Adorn that scope with the input parameters and recursively apply ‘the information’ to generate the actual XML file

Description:

[1]. Create an in-memory parse-tree out of the given .xsd - For this, one can either go with any of the open-source xml-parsers, or write from scratch. Writing from scratch would generally require Element, Node, Attribute and ComplexType classes along with their respective handlers. Then, you would’ve a Schema Parser, which would utilize instances of those classes and generate the final tree object.

I won’t elaborate further on this here; if required, please drop me a note. In essence, the only contract to be followed is that the return object should contain a parse-tree structure.

[2, 3 & 4]. To understand these, let us delve a bit into the source.

Our generator would generally need to follow something similar to the following contract: -

/**
* IXMLGenerator represents the contract that all XML generators must adhere to
* in order to support the default logic of XML genertion. Default XML generation
* contract is based on the consumption of the starting XML element
* and the consolidatedMap of input and configuration parameters to
* generate the populated response XML for that particular element.
*
* @author sushain
*
* @version 1.0
*/

public interface IXMLGenerator
{ /**
* This method takes a resource name, corresponding method and a
* parameter map and builds a request xml
* which can be consumed by the a request handler.
*
* @param resource
* @param method
* @param paramMap
*
* @return Generated request xml
*/

public static String generateXml(final String resource,
final String method, final HashMap paramMap)
throws SchemaParsingException;
}

Here, generateXml would be the main method, inherently calling the tree-creation module, and effectively completing step (1). This method would utilize the input-parameter map (HashMap containing the name-value pairs for the elements to be populated in the xml), resource (the entity for which the xml is desired) and method (the type of xml desired {system-specific}) by referring the ‘Configuration’ in the following manner: -

Now, I’ve repeatedly referred to a ‘Configuration’ above, lets see how that actually looks like: -

<configuration>
<resource name=”<system-defined name>”>
<method name=”<resource access method / CRUD operation>”>

<param name = “aaa” value=”val1″ />

<param name = “bbb” value=”val2″ />

<param name = “ccc” value=”val3″ />

<!–param name = “<complex_aaa>” value=”<attr_val1>,<attr_val2>” /–>

<!–param name = “<complex_bbb>” value=”<attr_val3>,<attr_val4>” /–>

</method>

.
.
.

Some key-points about the above conf are: -

- Each resource-name has a list of parameters associated with it that would define what exactly its associated XML would consist of.

- In case of complex-type params, the value consists of a list of comma-separated attributes.

- Such a definition is readable, easily extensible and configurable.

We would also have a corresponding Config class to read the above XML configuration into a Document (org.w3c.dom.Document) and build a hashMap out of it.

The structure of such a class would be similar to: -

public class Configuration {

private File configFile;

private Document configDoc;

private String resultNodeName;

public Configuration(String configText){

this.configFile = new File (configText);

this.configDoc = getConfigDocument();
}

public Document getConfigDocument() {

Document XMLDoc = XMLutils.getXMLDocument(configFile);

if (XMLDoc == null) {

System.out.println(”XMLDoc null”);

System.exit(-1);
}

String query = “/configuration”;

NodeList nl = (NodeList) XMLutils.runXPathQuery(XMLDoc, query);

if(nl.getLength() !=1 ) {

return null;
}

Document responseObject = XMLutils.getNodeAsDocumentRoot(nl.item(0));

return (responseObject == null) ? null : responseObject;
}

public String getResultNodeName() {

return resultNodeName;
}

public HashMap getConfigurationMap (final String resourceName, final String methodName) {

HashMap configObject = new HashMap ();

Node temp = this.configDoc.getDocumentElement().getFirstChild();

while(temp != null) {

if(temp.getNodeType() == Node.ELEMENT_NODE) {

if (temp.getAttributes().item(0).getNodeValue().trim().equals(resourceName)) {

NodeList methodList = temp.getChildNodes();

int methodListIndex = 0;

while (methodListIndex < methodList.getLength()) {

if (methodList.item (methodListIndex).getAttributes() == null) {

methodListIndex = methodListIndex + 1;

continue;
}

if (methodList.item (methodListIndex).getAttributes().item(0).getNodeValue().trim().equals(methodName)) {

NodeList paramList = methodList.item(methodListIndex).getChildNodes();

int paramListIndex = 0;

while (paramListIndex < paramList.getLength()) {

if (paramList.item(paramListIndex).getAttributes() == null) {

paramListIndex = paramListIndex + 1;

continue;
}

configObject.put(paramList.item(paramListIndex).getAttributes().item(0).getNodeValue().trim(),
paramList.item(paramListIndex).getAttributes().item(1).getNodeValue().trim());

paramListIndex = paramListIndex + 1;
}
}

methodListIndex = methodListIndex + 1;
}
}
}

temp = temp.getNextSibling();
}

return configObject;
}

}

- Once such a setup is in place, the generateXml method would then build a ‘Consolidated Map’ by appending ParamMap to HashMap.

- It would then read the xsd definition and get the in-memory parse tree ready using the module discussed in step (1).

- After that the actual xml generation logic would be invoked, passing the above built Consolidated Map and the resource specifiction.

- The generation logic would iterate through the tree, until it locates the resource spec

- It would then start building the XML, simultaneously referring the Consolidated Map and populating elements whenever it finds a corresponding entry in the Map. The population logic would be written in the corresponding handlers for ’simple’ as well as ‘complex’ type nodes.

This would be something similar to: -

if(attList.size() > 0) {

XSDAttribute att = (XSDAttribute)attList.get(0);

if (consolidatedMap.containsKey(getName())) {

attributeString = consolidatedMap.get(getName()).toString();

StringTokenizer stok = new StringTokenizer (attributeString, “,”);

while (stok.hasMoreTokens()) {

buffer.append(”<”);

buffer.append (getName() + ” “);

buffer.append (att.getName());

buffer.append(”=”);

buffer.append(”"” + stok.nextToken() + “”");

buffer.append(”>”);

if (consolidatedMap.containsKey(att.getName())) {

buffer.append (consolidatedMap.get(att.getName()));
}

buffer.append(”

buffer.append(getName());

buffer.append(”>”);
}

}

return buffer;

Finally, this XML would be returned to the caller and subsequently consumed by the original framework. For extension purposes, the only file to be modified later would be the configuration, which could be easily achieved due to the uniformity and structured nature of the definition. Due to a generic contract specification that only mandates an input parameter-map with name-value pairs of population-parameters, such a framework could be utilized in applications with a consuming UI-front as well as headless command-based apps. We thus saw how an effective configuration-design simplifies a seemingly difficult extension & integration problem.

If you have any suggestions, please drop a note / post.

- alphaworks.ibm.com/tech/mashuphub

- services.alphaworks.ibm.com/damia

- services.alphaworks.ibm.com/qedwiki

^ Excellent assembly canvasses and services to create holistic aggregations and their respective interface components, and in turn giving us yet another instance of an effort towards the unification of the random segments.

Title: A classification approach exploiting the rectangular topology of a floor-plan to determine the location of a mobile user inside premises using only 2 access points.

Detail: This disclosure contains two related ideas. One is the concept that tries to reduce the problem complexity by exploiting the underlying problem domain and the other provides an associated implementation with a classification approach using ANN (Artificial Neural-Networks).

The known solutions to these problems make use of a minimum 3 access-points to determine the location using triangulation. These solutions exploit the fact that the Signal strengths from an access point vary radially, i.e., the signal strength along an imaginary circle of radius ‘r’ from the location of the access point (assumed as the focus or center) is constant, say SS(r1), and decreases as we go radially away from the focus. So, if three such access points (APs) are placed randomly, then since 3 different circles can uniquely determine a point in 2-d space, we can locate a test-user based on his relative signal strengths w.r.t those 3 APs. [Refer Fig. 1]

Figure 1. Case when APs are randomly placed (Triangulation).

We try to take advantage of the fact that majority of physical structures, like buildings, organizations, etc, are rectangular in shape (top-view). Now, it is a fact that there is a near complete loss of signal strength when there is an obstruction thicker than 3cm, which is minimum wall thickness in almost all concrete buildings into which such a system may be installed. In theory, the signal strengths from an access point always vary radially and are constant along the circumference of signal strength circles. So, if we place the access-point on a corner, those circles reduce to quadrants (1/4th of a circle), because the other 3/4 of the circle is cut-off by the test-area walls.

Thus, by placing two access points at two of the adjacent corners in such rectangular structures, the “signal-strength circles” can be reduced to “Signal-Strength Quadrants” as shown in the figure. And thus, we can locate the test-user at the intersection of those two quadrants because any point in the 2d-plane of the building or test-lab can be represented by the set of Signal Strengths from the two Access Points. [Refer Fig. 2]

Figure 2. Case when APs are places at the adjacent corners of a rectangular test area.

The figure above shows the access point placement at the two ‘adjacent’ corners. Now, this figure is meant to portray that, theoretically, in a rectangular test-area, there is a possible arrangement of 2 access points, which would yield a unique intersection of signal strengths and this can serve as a possible premise for any further implementation.

Now, this premise serves as a foundation to prove that an implementation based on this can lead to decent results since, theoretically, we can see that unique points exist for any signal strength pair (SS1, SS2).

In most existing work on the problem of location determination with ‘n’ APs, such a foundation is existent for n >=3, since triangulation comes into picture in those cases. But, that foundation isn’t obvious when n = 2 when triangulation won’t apply. Most papers describe training their algorithms for different access points (varying from 2 onwards), which is bound to give some results. But, in most cases those results aren’t considerable when we talk of 2 access points, case of a single access points being highly improbable to yield even remotely precise results.

Implementation: Let us assume that the floor plan of a test building installed with two APs is as shown in Fig. 2. The methodology for determining the location of any Wi-Fi enabled device inside the test premises can then be covered in following phases: -

Phase 1: A set of ‘n’ locations (brown dots) is selected in the building and training data is collected at each of those n points. For each point, a value tuple of the form [SS1, SS2, x, y, Location No], is inserted in the training database, where, SS1 = Signal Strength from AP1, SS2 is signal strength from AP2 & x, y is the abscissa and ordinate respectively considering rectangular coordinate system with reference at the left-bottom of the figure. Here, we may try and filter out null or negative signal strength values. So, we have a database that can later be queried to get the signal strength from each access point at each training position. This process is called data-collection phase and is fairly common in any of the ANN techniques. This data is later used to train the Neural-model as explained below.

Phase 2: In this phase, we try to build a mathematical model for the test area using neural networks. Consider an equation of the form below: -

C = W0 * S1 + W1 * S2 + W2, —————————- (I), where C is the Class, W0, W1, W2 are the constant weights (initially they are unknown) and S1, S2 are the variable signal strengths from the two access points. Here [W0, W1, W2] is called the weight matrix and F [S1, S2] is a representation for a function of S1 and S2.

The above equation is a starting assumption that we make in any neural-modeling approach. Now, from phase 1 above, we would have ‘n’ cases (1 for each test-location), where we know the Signal strengths (S1, S2) as well as the class (since we know the location of those 12 training points). So, using these 12 cases, we can try to obtain the value of the Weight matrix in Equation (I). This process is called training of a neural model and this may be thought of as being equivalent to solving 12 simultaneous equations to arrive at the values of 3 variables W0, W1 and W2.

After this training phase, we would get some values of [W0, W1, W2] which would satisfy all the ‘n’ points, i.e., given any signal strength pair (S1, S2), it would locate it properly in the correct class. Then we reason from the fact that since our model is trained enough to classify points in the lab in question, so given a signal strength value pair from any other point as well (different from those ‘n’ training points), it would still locate it correctly in the appropriate class. This approach essentially tries to find and exploit any pattern that is present in the signal strength distribution for this particular test-lab.

Now, we divide the floor plan into topological regular classes (rectangular, hexagonal, etc). Let me explain these models a bit in detail. Consider Fig. 3.

Figure 3. Basic classification model

As shown, let us assume that our first model (M1) is such that it classifies the test-floor into three classes (Class1, Class2 and Class3). Now, let us consider a test-point P. For this point, we would compute the signal strength tuple (SS1, SS2). Now, substituting these in the corresponding equation (like the one described in (I) above) for this model (M1), we will get a value for the Class, ‘C’. For this point, lets assume the class comes out as Class1 as shown in the figure. So, this way the floor-area in classes - Class2 and Class3 is eliminated and we can now focus on area under Class1.

Now, let us consider a second model (M2) being superimposed over M1, which is as shown in fig. 4.

Figure 4. Superimposed classification models

This model classifies the test-floor in a different way as shown by the class-names in red. So, by feeding the signal strength value pair for the test-point P, we would get the class as Class 3. So, our investigations can further reduce to the intersection of Class 3 of model 2 and Class 1 of model 1, i.e., essentially the shaded region. Going on applying 2-3 more models, we can pin point the location of the test-point (P) to a very narrow corridor within an acceptable error-margin.

So, the idea is to have these models, i.e., these equations (of form (I)) prepared in phase 2 and then just use the appropriate or all models for a test-point to narrow down its location. This overall approach was found to locate a test-user within a 3-meter error-margin in the average case with only 2 access points placed as indicated in Fig. 2.

You can view this on ip.com @ IPCOM000168722