Adding Shindig to a .NET Site

The OpenSocial API is quite large and to try and implement it from scratch would be very expensive and next to impossible. Fortunately we have Shindig (which I expect anyone reading this blog is familiar with), the reference implementation for OpenSocial, which we can be used to make a web site open social compliant.

However, this still left us with a few difficult issues. One was that we were trying to do this in 2009. Shindig was very young, and still in incubator mode with the Apache foundation that manages its code base. Profiles was also very young, and still in beta mode. The other issue is that Shindig is implemented in Java or PHP, whereas the Profiles web site that we were making OpenSocial is written in .NET.

There is only so much we could do to resolve the maturity issues with the products. Waiting for them to mature happens naturally of course, but that can take time and is not really in our control. The other defense is to learn as much about the products as possible. With both Shindig and Profiles, we personally found the products and code base to be strong but the technical documentation was extremely light. Diving into the code base did slowly resolve these issue. To anyone reading this, I would highly recommend browsing through the Shindig code when the documentation seems wrong or altogether absent. We've answered many questions about the product in this way, and resolved a number of issues by just seeing exactly how Shindig wants to work.

For the .NET to Java issue we started by looking for .NET ports of Shindig. We did find one, but it was maintained by a single individual and that represented too much risk for us to follow. We ultimately decided to go with a hybrid deployment model. Shindig in Java/Tomcat working with Profiles in C#(.NET)/IIS. There are a number of ways to do this from a web hosting perspective, but to minimize Javascript issues we decided to host both IIS and Tomcat on the same server. We used the IIS/Tomcat connector so that all web traffic would come in to IIS on port 80. We deployed Shindig as non-root on Tomcat (shindig.war so /shindig, and the documentation for doing a non-ROOT deployment of shindig is not complete by the way, but if you know what you are doing a few changes in shindig.properties and container.js will do the trick) and set up the IIS/Tomcat connector to receive all /shindig web traffic.

At first, this did not work for reasons that were not obvious. After much snooping around we found that IIS (and probably the IIS/Tomcat connector) could not handle the long colon delimited URL's that Shindig builds in concatenating Javascript file request). We searched around and found that this was configurable in both IIS (with us it was HTTP.SYS http://support.microsoft.com/kb/820129) and with the IIS/Tomcat connector. A few changes and we had everything working!

Getting the Shindig server and .NET to use the same DB (SQL Server 2005 in our case) was not an issue. A plethora of drivers exists to hook Java and/or .NET to just about any DB you can find. So, we overcame the technical issues in getting .NET to work with Shindig. Perception issues, unfortunately, sill do remain. Not everyone is comfortable in having both .NET and Java running in the same application and same hosting environment. But when they see us demo what we have built, the do start to become convinced :)

Industry and Academia: A Foot in Each World

I started working at UCSF as a Technical Architect in February 2009. This was a new direction for me, as my professional background has not been with education or health sciences. Sure, like many in my field I'd had the occasional contract for cranking out code for a health related company and even once developed a system for UCOP as a contractor, but finance and marketing were primarily the industries I knew. However, academia is sort of it's own separate world. I found that as a developer in finance, or marketing, or whatever-idea-you-want.com'ing, that being a developer put you in a common group of developers and technologist that were not defined by their various industries. A java developer in finance and a java developer in telecommunications are still both java developers and they tend to go to the same conferences, follow the same advances in the technology, and do somewhat similar things.

At UCSF, things were different. The language was even different. Instead of being engineers or IT staff, we're now Informatics. The conferences are aligned primarily with health science and secondarily with technology. You see this with the AMIA conferences as well as the CTSA IKFC conferences (and you can look those terms up yourself).

So..., why does any of that matter to this blog? Because, it is the primary source of both opportunity as well as conflict in moving ideas from one arena (such as industry) over to the other arena (academia), and this can flow in both directions. Ideas from industry can sometimes take a while to be adopted by academia even though they add seemingly obvious value. On the other hand, what works for industry does not always work for academia because academia is just a different world. The users are different, and best practices elsewhere can sometimes backfire with researchers and professors.

We think OpenSocial is in the first camp: it's a great idea from industry that just makes too much sense not to work for academia. Especially with the recent work in building and connecting "research networking tools" at various institutions.