Bernardo Huberman is a busy man. When he’s not working on models of computational economics and agent-based systems (remember them?), he’s researching ways to automate social network discovery through email pattern analysis. I met Bernardo at a conference at HP Labs in Bristol a few years ago. Over dinner, he explained the thought at the core of much of his work — that the Web is an ecology, but with the singular characteristic that, being digital in essence, it is very easy to instrument, and hence a very useful experimental environment.

This idea has historically been underexplored — it is really only with the advent of blogging, with its associated cilia of trackbacks and pingbacks, aggregators and so on, that the exploitation of metadata generated by social media tools has really hit the mainstream.

But the ideas are applicable in many other areas. One of the core thoughts behind HowellHenryLand was the idea of consensual surveillance. Our observation was that an ad agency is an environment where the most valuable knowledge is usually tacit rather than explicit. HHCL has constructed a working environment designed to encourage serendipitious meetings, chance conversations, random interactions that (hopefully) lead to emergent insights. It’s a difficult environment on which to attempt to impose formal structure.

Our thought was that the ideal method for knowledge capture would be to allocate every key person their own personal amanuensis, trailing behind them with a notepad, capturing every important thought as soon as it was spoken — ‘a Boswell for every Johnson’ as we put it. Impractical in the physical world, but possible online (of course, the possibilities are limited by the degree to which people interact using electronic media — in a purely digital world, such as a MOO, every interaction can be monitored by agents such as Cobot. In most business environments you have to make do with what’s available). At HHCL, we set up a number of mailing lists for collaboration on key areas of the business, and made sure all the archives fed into a searchable ‘Soup’ of data. We set up email addresses, again feeding into the Soup — if you are emailing a good idea or an important snippet of research data to someone else, you can just Cc the Soup and your insight is stored away, ready to be found whenever needed. And we built instrumentation into our online chat areas, so that online meetings could be Souped for posterity. Very simple ideas, but ones that work, and still work in other environments — there has been a recent profusion of bots lurking in online chat services, busily keyword searching and informing their masters when there are people online discussing interesting things.

Assuming your people use digital communications, and are not adverse to such consensual surveillance, grabbing the data is easy — its much harder getting that data in front of someone who could benefit from it, in the moment, especially if that moment is most likely when they are sitting in front of a stack of notes trying to write a presentation. I wonder if anyone made any progress on developing a Remembrance Agent-style tool that runs inside MS Office watching for keywords, and which pops up something slightly more useful than Mr Clippy when a background search of your Soup turns up a useful snippet. Oh I forgot — wasn’t it called Autonomy? It’s a shame no one in the open source community is tackling this head-on — many knowledge-based businesses are too small to afford something on that scale, but could benefit hugely from this form of knowledge implement — especially if the data sources could be distributed blog-like systems rather than carefully-controlled document stores jealously guarded behind corporate firewalls: there’s a potential model somewhere in there for micropayment-driven knowledge sharing within distributed communities of practice — creating another ecological web which itself can be mined for meaning…