Project in Hybernation

This post is announcing that I'm no longer actively developing ScenConnect. The project will remain up, and I'm very happy to answer questions about it, work on supporting various aspects of it, and advise others who would like to work on it.

There are a number of reasons for why I'm ending development, but the upshot is that it wasn't clear that the project would go anywhere. I'm now going to be working on projects concerning ubiquitous sustainability and the interfaces between urban informatics and material/industrial systems. If you have any references, please post them here.

I'm also going to be focusing more on working with existing projects.

Score Networks

After a lot of thought, I've finally figured out how to do the kind of default environment scoring. That is, independent of any query, how do you score a given bit of content. There are many issues that compounded the difficulty of this problem, but I think I have answered them temporarily.

• Many different kinds of factors should can influence the score, and some should take precidence over another (in the most trivial example, if the user gives a score to an item, the item will always show that score instead of being influence by the tag of an item).
  

To handle this, I explicitly built a matrix which specified which kinds of nodes would dominate other kinds.

• How much state about a given score should be retained? What should be present, and what should be recomputed?

Essentially I keep all of the connectivity information stored in a network. Since I'm currently allowing a small number of influence types, so storing everything saves massive hassles.

• How much should learning come into play?

I don't think it should come into play at all when constructing from items, nor when computing it. Instead, I think learning steps can augment the network with low-precidence links that are removed if high ones come along.

Intercommunication

I've decided that the right way to handle intercommunication is to abstract it out. There are too many good resources with restrictive licenses to pass up. I'm going to create an abstract CommunicationsManager interface, and use reflection to discover them. Then, I'll create a project for every intercommunications library, each with two releases: one with ScenConnect, and one without. Then, people can either choose their favorite, or mix and match, as their preferences dictate. No muss, no fuss.

I'm going to start with OpenSpime as an initial target. If it works well, I'll try to convince the SourceForge folks to create a new project for it. It'll be a nice little Java/Python interface, so if anybody has suggestions about Java/Python interfacing strategies, I'd be very happy to listen.

OpenSpime Libraries Released

The OpenSpime folks have released their first libraries. It could be pretty good. If it looks really fantastic after closer examination, I'll convert the project over, even if it means a Python rewrite.

Ranking

Recently, I've been working on the ranking part of the query system, which will be responsible for ordering the results. The exciting part of this is that this is the part of the system where I'm beginning to fold Clojure in, to actually execute the ranking parts of the query. Unfortunately, amidst all this fun and good cheer, I have a couple of concerns.

1. First of all, I'm not sure Clojure is buying me any advantage here, since I'm already building the parse tree anyway. I'm not too worried about this, since I hope to use Clojure to do ranking-like tasks in the background, so it's useful experience.
   
2. The bigger conflict is that I'm not sure how the current query ranking language fits with the standard interpretation of scores (as documented here earlier, and in the specs). This isn't surprising, since they were developed concurrently. Fortunately, I think this could be resolved by simplifying the rank language, until it can quietly be interpreted as standard. On the downside, I'm not sure how to implement the standard.

I've also discovered that some features, such as aggregates for the default rank, don't make much sense when applied to streaming arrivals (which I already knew, but I didn't know that I was making this mistake at the time). This is why implementation is good, because it finds inconsistencies in the implied implementations.

And then I hit continue

So, after a job search (which I've finished successfully, but I fear will be orthogonal to the scenario connector work for some time) and a vacation, I've started back to work on the project. I'm working on the query system and other internals because they are the easiest to pick up on. There are FIX ME's everywhere, and all I need to do to keep working is just look for them. I have a week where I have no employment commitments this summer, so I might hype this project a little during that time.

Bruce Sterling on the spime payoff for usability

I admit I feature bruces videos way too much, but Peter Moreville recommended it, so what the hey.

Bruce Sterling from Innovationsforum on Vimeo.