Yay! After months of pushing ideas, planning the concept and just being lazy as usual, the work on the Topify web site has finally kicked off “for real”. It’s not a lot yet, but we have a design and some good ideas, mostly aiming towards being user-friendly, intuitive and easily usable. The technical stuff is nearly complete as well, so it’s now a matter of combining and adjusting the algorithms and the web stuff. This might take some time still, since we’re only a 2-man-team with studies and part-time work to attend to as well, but we’re doing all things possible to bring this project to life as soon as possible! :) Cheers.

I apologize for not being able to write a lot these last few weeks. I’ve been mostly busy working on a chapter about the theory of public opinion for my final thesis. This was a very enlightening experience, since it opened my eyes on the different ways one can take when measuring the “voix publique” and the corresponding problems. The question I raised at the end of the chapter in my thesis is the one I ask here: do blogs represent the public opinion? This is indeed a very tricky question, but if one takes a look at what the public opinion actually is, there is hope still, that the question can be answered!

The concept of a public opinion comes from the 18th century France, where it was represented by a group of similarly educated, well-read intellectuals. Due to the similarity in the education and the philosophy of the members of the group, the critique they expressed tended to come in unison, hence the initial singular in “public opinion”. Later on, as more differently educated social groups appeared, the “one” public opinion dispersed into the opinions of these groups. By the 20th century, every person was considered bright enough to think on his own, so the notion of an “opinion expressed in social groups” was dropped for a more “public” definition of the term.

However, ever since the public opinion was considered to be a sum of all the personal opinions in the public, no one has been really able to grasp it anymore. Multiple institutions for research of public opinion exist all over the world, but all they do is question the people on the street (at home, on the Web, whatever.. keyword is “at random”). The biggest problem with this approach is that though not every person has an opinion to every subject, they’re still browbeaten by the interviewers to say something..

On the other hand, most people have opinions to certain subjects. And most of us wouldn’t mind sharing them, provided that our anonymity is guaranteed. These are thus exactly the thoughts and opinions that one should gather in order to get an idea of the so called “public opinion”. How do we find these people? Look on the Web! Find all posts that deal with a topic that you want to examine and see what people are saying. This is being done already by specialized brand-monitoring companies like Trackur, which was just recently reviewed on ReadWriteWeb. Unfortunately, however, such services still run for a fee and not for all the random subjects around us.

Thus I would argue that a service that gives its’ users an overview of the “hot” topics and the opinions about these should be of great value and interest to everyone from a common man to a scheming politician. It’s time to put a stop to the “A”-blogs dominating the agenda like the social groups of the 19th century. There is much more information out there that is just as exciting and valuable, we just tend to underestimate the long tail. To get an impression of the matter just click over to this post over at the Data Mining blog and take a look at the odds for a “normal” blogger getting a headline at a site like TechMeme compared to those of an “A”-blogger. Impressive huh? Now just try to imagine the diversity of the topics and opinions in the public blogs.. it’s IMMENSE!

Just a couple of weeks ago I was trying to set things straight in the rather troublesome relationship between tags and topics. The bottom line was that tags are just the same words we use in the content and are hence just as arbitrary and hard to generalize as the content itself. No perfect solution for this problem exists as yet, although I proposed to topify the content in order to keep the generalizing terms (tags.. topics..) as close to the content as possible.

Now I get reminded that there is a third member of the family that may be used in this way, but rather tends to go astray and entertain people. I’m of course talking about the ever present headlines that set the tone to a piece of content long before we actually get to read the whole thing. So what is the function of this, rather distinct, member of the meta-family?

One might have thought that titles should provide the reader a short taster of what is coming next, so that he or she would be able to filter and prioritize a piece of content among others. However, certain media actors don’t like to play fair, especially those who don’t have much of worthy content to offer. This is first and foremost the case with the so called Yellow Press, which has invented and perfected the art of indirect headline for the last century or so.

The idea of an indirect headline is to use a quality of human nature - a humans curiosity - to drive readers to some potentially utterly worthless content, as was the case in a blog entry in The Blog Herald that inspired this here post. The evil thing about this is that we can’t help it but take the bait over and over, even after getting disappointed time and time again.

So what can we do to tame this extravagant member of the meta-family? Well, frankly not much. One could check out the site’s tags to get some additional information, but the problem with tags is that they are becoming more and more like headlines as well. If you tag your content in detail, you risk losing readers who know for sure they don’t care about certain stuff. General tags on the other hand don’t allow you to filter and prioritize the content properly, so you have to check out at least the first few lines of the text. This is where, I think, a more intrinsic way of meta-describing content by topics e.g. can come in handy, from the user point of view anyway.

Since the birth of the Internet, we are being told that in order to build a successful web site, one maxim has to be strictly followed: “Content is King”. It doesn’t matter, what the site is about or who is the target audience, if it doesn’t have any content, well, then it’s just some useless HTML code and not a web site. There have been some claims over the recent years that community has taken over this pivoting role, but all in all, communities are just a highly advanced way of producing more.. duh.. content. Well, maybe apart from one, silently praying for a Killer App to come and save us all from this flood of all this content. Where can I join??

The reality is that content has become the tyrant of our everyday lives, as so brilliantly described in a post by Sarah Perez on ReadWriteWeb. And this doesn’t apply to the people who have to check all this content-hugging services out for a living; each and every one of us who uses the web in one way or the other is affected by it. Just try finding a single news article on some topic or a corresponding conversation. Yes, you can Google, or look up the relevant tags on Delicious, or check the buzz out on Digg, or.. But each of these options would give you an overload of relevant articles from which you still have to choose. And don’t forget that you have to choose one of the myriad of searching/exploring services in the first place.

So do we need a Killer App to save us? The general opinion is while such an app would be useful from an ease of use point of view, it’s still far from a great solution. For one, bringing all of the content together in one place doesn’t solve the problem of it being rather unsorted and unfiltered, as pointed out in a recent Techrunch article on Web 3.0. One could argue that we should just stop producing all this duplicate content, which has actually already been proposed rather seriously on Publishing 2.0. But while such a movement is a great initiative, what I really feel we need is a sophisticated filtering system.

There is probably no way we could filter all the unwanted content out automatically without losing something potentially interesting (or new!). But there must be a way of filtering out duplicates that deal with the same topics from slightly different angles. One could then adjust such a filtering algorithm personally over time by giving it feedback so that it would learn our reading preferences to a certain threshold where we get a minimum of redundant news and also minimize the risk of having something new discarded. Might take some more serious thinking and doing, but doesn’t sound like rocket science to me.

This is a challenging question that comes up all the time in our everyday life. This includes anything from the “normal” face-to-face communication when two people meet on the street to blog entries and instant messages or tweets on the Web. As a matter of fact, this is the main topic of a great ReadWriteWeb post titled “What People Say When They Tweet“. I’m really glad to see that the survey performed by the RWW blog and Summize doesn’t stop on calculating the most frequent words and phrases, since those only reflect our emotions and current activities (e.g. lol, ;), working, sleeping). Instead, they try to extract the topics of conversations, further dividing these into short-term and long-term discussions (e.g. about a “Lost” episode vs. US election).

I find the results truly fascinating, since this is a kind of information that is really hard to obtain directly. What these results show is an extremely simplified reflection of the matters that in one way or the other bother us at a certain point in time. It’s a bit sad that the results show less than 10 topics per day, since there must be tens and hundreds of others that have a big enough population as well. It’s just normal that some themes that are common to all the population of a country, like the US election, will dominate all the discussions for a long time. This can be compared to a cloud of 100 topics, where one, the election, completely dominates the rest, but there are hundreds of others around it, that shouldn’t be neglected, considering that there are millions of potential topics.

But no need to be sad! Since this is exactly what Topify is aiming to provide all its’ users - an overview of the topics that are burning the fingertips of all those who are ardently blogging away in an effort to make oneself heard. We hope to make your everyday experience at this project’s site as fascinating as that one post on RWW, with the difference that we are aiming to concentrate on the more structural and constructive communication area - the blogosphere. Excited already? :)

I think I don’t have to explain to anyone what tags are, or? But to make sure, Wikipedia states that tags allow anyone on the Web to annotate and categorize content and then roam around and discover similarly tagged content. Sounds like a blast to a lot of people, all one has to do is find some content one finds interesting (a lucky search will do), tag it accordingly and then sit back and enjoy the thrill of discovery via his or her RSS reader.

Now, suppose we stumble upon one of those Microsoft-Yahoo merger articles (well, you couldn’t avoid running into one even if you wanted to) and want to tag it. It’s surely about.. Microsoft.. and Yahoo.. and business deals/mergers/acquisitions.. the “Future of the Web as we know it”.. then technology.. media.. ads.. revenues.. oh and the ubiquitous Google.. I just crawled these from 5 blog entries or so, if I took a week off and went through the other thousand, I’m sure the number of tags used on this one subject would reach a century.

What is then the problem with tagging? Is that different people use different words for the same topics? Or that everyone categorizes differently? Or that people read in different topics into stories? Obviously it’s all of the above and this is what makes tagging, a practice targeted to be used by communities, so hard to actually employ this way. There is a great post by Ms. Rashmi Sinha on the cognitive processes behind concept building and tagging, which shows that it’s a very tough task to actually employ tags that would make the tagged content be easily discovered by others.

So what are the possible solutions to this here mess? The “Tagging” blog describes the approach of ‘tag description’ where users can make a note for themselves, what they actually understand under a selected tag. “Tag gardening” is a great definition for this approach, and I assume that most of you won’t be to keen on having to spend hours on growing your favorite tag plants. Another attempt to make tags more generalized is the idea of semantic tagging, e.g. as provided by Zigtag. I’m sure it will improve the tagging experience by a country mile, but it still doesn’t reduce the number of tags nor the number of words we can theoretically use.

Another solution would be to actually extract the main topics from a piece of content so that the story is naturally categorized. Then, depending on if you want to read a story about just one of these topics or explore in similar topics, you can either search directly or use natural language semantic relationships to find similar stuff. This might take the social component out of inventing tags that are understood by a possibly large part of the community, but it would bring in more related content discovery and allow people with similar thoughts to find each other and start talking, rather than tagging. Or?

Hello and welcome to the official blog of Topify!

What is Topify you ask? No? Well, we’ll tell you anyway. Topify is a project aiming to extract and provide you with the intrinsic subject for all those news articles and blog posts that appear en masse everyday on the Web. This problem occurred to me while working on my Master thesis on political communication online, when I noticed that while there is a lot of content out there that you know deals with the topics you want to examine or inform yourself on or just read about, it is not an easy task to filter it out properly.

Sure, there are blog directories and tagging communities, which, don’t take me wrong, are a great thing to have, but sometimes when you’re just looking for some facts and thoughts on.. say rubber stamps (the political ones) you end up with blog entries just calling someone a rubber stamp without much of an explanation or something similar to that. What is much harder to come by is a piece of writing like this, really dedicated to those politicians who, according to the new Safire’s Political Dictionary “take orders from a political leader or [are] dominated by orders from higher up.“

So while the world of online news and blog posts is prospering like never before, the options on how to browse all this data remain rather scant. At the moment, if you don’t tag your content right, you run the risk of being missed by a significant part of your potential target audience because sometimes people just want to read about one exact topic and not all the posts tagged with a generalized concept. This is where we at Topify hope to be able to build a system that can extract those natural topics from news items and blog posts for everyone to discover and talk about.

If I could get you interested, come around again to this blog for further thoughts and discussions on this and related topics. The project itself is still in development, but with your suggestions it could become great addition to your favorite social web sites as soon as this Summer. Stay tuned! :)