The end of piracy?

(via ORG-discuss list) Pirates, pirates, pirates. it's all there is in the news lately. Pirates in Somalia, pirates in Sweden, pirates online, pirates in culture. We have pirates jumping out of our breakfast cereals it seems. Last week saw the pinnacle of hyperbolic coverage of piracy of the Internet sort when the Pirate Bay operators were sentenced to a year in jail. We were promised by talking heads on TV that the ruling would be the beginning of the end of internet piracy, and that a new and happy future where everyone respects copyright law is finally within reach.

But then reality got on the way again. First it was a report from Norway that informed us of something we already knew anecdotally, and that is that teenagers who download music (legally or illegally) are 10 times more likely to pay for music than those who do not. In other words, the next generation of music consumers is downloading. This is the market, deal with it.

Then Bram Cohen gave an interview to The Register (I know, I know), where I think he gave the most insightful view of why there will always be a surplus in music production:

"Music has a bigger problem, it's that people want to make it. It's the peacocks tail. The reason guys make music is that they want to get laid. So men are usually willing to pay a lot of money in the hope of getting laid. Anything that helps you get laid with amazing regularity is something you would expect a tremendous oversupply of. So we have unbelievable amounts of music. People pay a lot to learn how to play music and it's ridiculous to expect people to make money off it. Normally if you want to make money you do something no one wants to do."

Must learn to play an instrument... but I digress. So, there will always be an offer surplus, and those who consume the most are also the ones you have been fighting during the last 10 years. Moreover, another study shows that users view P2P subscription services favourably, particularly if it comes with their ISP. This at the same time as yet another effort to try to sneak in three-strikes into European regulations have been unsuccessful.

A picture is starting to emerge, so here is some free advice for the music industry. Instead of spending an inordinate amount of money in pursuing a failed business model, why not look for other avenues? Copyright owners have been pursuing P2P providers for too long, and there is no indication whatsoever that they will be able to eliminate file-sharing through the courts, as another service will simply take the place of the previous one. This is why copyright industries have been pushing for the unpopular "three-strikes-and-you're out" strategy, because it leaves enforcement in the hands of the intermediaries. However, why not make them partners in your business and allow some form of blanket licensing scheme by which ISPs charge a higher broadband premium service with a download package. The artist gets the money, the ISPs geta cut (and do not have to enforce a nightmarish policy), and the user gets what they want.

Too much of a happy ending? Unrealistic utopian wishful thinking? To quote a great man, you may say I'm a dreamer, but I'm not the only one.

Phorm launch in the horizon

Phorm (also known as Webwise) has declared that it is finally set to launch its service in the UK with BT's ISP service. Phorm is a service that allocates web users a unique number, it then monitors websites that the user has visited and looks for recurring keywords related to its advertisers. It then associates those keywords with the user's number, so that when the user visits a site containing a web advert, it will display relevant commercials. For a more detailed explanation, read Richard Clayton's excellent report.

As a founding supporter of ORG, I do not like Phorm. I understand their claims of anonymity and security, but I am still not convinced that this level of surveillance is either useful or necessary. Phorm seems to add an unwanted layer of interception to the internet, and I am not sure that users will like to have such records made of their browsing habits. Phorm claims that it destroys those records, but I am still uneasy. Who is to say that in this difficult financial climate, they won't change their data retention practices to sell more information about web users?

Something that is perhaps unsurprising, yet still disappointing, is that Phorm seems intent in misrepresenting and hiding what it does to the public. There was a public outcry when they conducted a series of test runs with unsuspecting BT customers. BT is also intent in claiming that Phorm's main function is to "increase protection against online fraud". They also seem to brush all consumer concerns under the rug. They claim:

"That being said, what we did not take into account was the fact there would be a very small number of very determined people who would do their very best to make it appear in the worst popular light. I am surprised by the fact, after it has been repeatedly explained how the technology works, they seem to be very keen on misunderstanding what it does."

No, thanks to people like Richard Clayton, we do understand how it works quite well, and we are still very concerned about it. As a BT customer I won't be opting in of the system, as the Information Commissioner has declared that Phorm can only operate with the consent of the user.

If you don't like Phorm, join the Facebook campaign. Phew, I managed to reach the end of the post without any clever "eph" replacement, ephing well done. Doh!

Book Review: Rainbows End

Before being attacked by the Lynne Trusses of the world, the title of Vernor Vinge's Rainbows End is not missing an apostrophe. The book is named after a retirement community which was chosen by either "an everyday illiterate or someone who really understood the place.” Vinge is part of a wave of technologically sophisticated authors depicting the post-internet world, this cadre includes Charles Stross, William Gibson and Cory Doctorow, yet I am reluctant to call this movement cyberpunk. Post-cyberpunk maybe?

The book is very technical in nature, and includes dialogue on instant messaging (thankfully not on chatspeak); it also introduces several new acronyms for neuroscience technologies that we may have in the future: JITT (Just In Time Training), and YGBM (You Gotta Believe Me). It is possible that all of this SMing and techno-speak may put-off some people, but they do work in the context of the hyper-connected world described by Vinge. While the text conversations get clunky at times, the story makes up for it, and the characters jump out of the page, particularly the main character, erstwhile poet and recovered Alzhaimer sufferer Robert Gu.

The world of Rainbows End is what makes the novel memorable, and I beleive that it showcases three new technologies that make it an excellent read for cyberlaw classes. In this world, medicine has not only managed to expand lifespans considerably, it has begun reversing the process of ageing, particularly neurological debilitating diseases such as Alzheimers. Vinge's real triumph is to describe what would a world inhabited by rejuvenated elderly would look like, particularly because medicine cannot cure everything. So there are people like Robert Gu who have made astounding recoveries, while there are other lucid yet infirm elderly on wheelchairs. This world is also one where wireless broadband is ubiquitous, but the internet looks nothing like ours. Computers are wearable and include contact lenses that over-impose an information layer to reality, so the future web is a vast world of augmented reality where people can wear avatars in everyday life, but also where everything and everyone is tagged with information. The third technological feature of this world is that it is a strictly controlled hardware environment, where every piece of gear has embedded protection that connects with both Homeland Security and with certificate authorities. This allows a level of control that we cannot dream of, but it is also a rather vulnerable world.

There are some really interesting considerations of this post-geriatric world. Leaving aside the bioethics of the technologies described, I found the social implications staggering. We already live in an era sharply divided along generational lines. The wired generations live in a different world than that of the analog ones. While the number of older people adopting some of the technologies is growing (my mother has joined Facebook and frequents internet forums), there is still a large disconnect with the possibilities presented by the user-generated world, so there are entire generations left behind. This is not really explored in social research in cyberspace, but Vinge presents us with an important question. What if lifespans keep getting longer, and older generations have the mental learning capabilities that they had when they were younger? The answer is both shocking and elegant. Send them to school! In Rainbows End, schools are filled with the very young and the very old, both learning how to use and navigate the datastreams. This makes a lot of sense, but it also presents some challenges for a society where the old and the new clash in such manner. There is an endearing connection between teenagers and the old geezers that I found both believable and desirable. Technology could bring back respect for one's elders. What a liberating idea!

Rainbows End is also the ultimate user-generated world. Users can modify themselves, but also their environment. There is one fan-generated layer covering the entire world which has mapped Pratchett's Discworld into our own, so if you were in China you would be in the Agatean Empire, if you were in London you would see Ankh-Morpork, and presumably if you were in Australia you would see XXX. In fact, the world is filled with these "belief circles" where fans choose which reality to inhabit, and sometimes they clash and fight for recognition and space.

Finally, Rainbows End depicts a tightly controlled network environment, where every piece of hardware must have a valid certificate issued by an authorised certificate authority. This is simply the endgame of Trusted Computing, and the embedding of technological protection measures not only into media, but into every single piece of equipment that has a chip inside. While in theory, this allows a level of surveillance unmatched even by our CCTV crazy environment, it can also be circumvented. Vinge's genius is that he recognises that even if those pushing for secure hardware environments get their wishes fulfilled, hackers would still be able to circumvent the technologies by using illegal gear. But also, Vinge describes the dangers of such a world, which would be vulnerable to cascading failures if someone was able to attack a single ceritficate authority and void a whole lot of permissions.

To conclude, Rainbows End is a wonderful look at the near future, but one that leaves some questions for us to decide. I like his view of the post-geriatric world, and I think that augmented reality will soon be here. The book is filled little gems and witticisms that make it highly enjoyable and readable if you get past the texting.

And yes, rainbows do end.

What is digital content?

Continuing with the coverage of the interim Digital Britain report, something has been bothering me since I read it, so I went back and browsed through it again until I realised what it was. According to the UK's chief technology policy-makers, we still seem to be living in the 20th century. Why? Several reasons: the only mention to Web 2.0 is in the glossary; some of the technologies being pushed are proved failures with the public; it believes DRM offers a solution to piracy; it blatantly ignores the content delivery revolution that is about to take place; but most importantly, it ignores user-generated content by insisting on the outdated view of the top-down content provider.

There are few things that can drive me to anger, but chiefly amongst those is the insistence by the copyright industry and regulators to ignore that nowadays content is much more diverse to what it once was. The first sign that the Digital Britain report has some very outdated ideas about content creation and distribution is the baffling emphasis it places on Digital Audio Broadcasting (DAB). DAB radio is a broadcasting standard that replaces FM, offering better quality, larger amount of metadata, and more stations in the same spectrum.You may be thinking what is wrong with this? I'm quite fond of DAB in a way, I have a receptor in my kitchen, but what I object to is the fact that DAB has not proved popular with the public because it is what I call a mistimed technology, much similar to the DAT tape in the 90s. Mistimed technologies are those which are a clear improvement over their predecessors, but they come to the market almost at the same time as a much better technology comes along and renders the mistimed technology obsolete as well. There is therefore no incentive for users to upgrade to the inferior technology, so many stay with the older and inefficient one until they upgrade fully. In the case of DAB radio, most people still use FM radio, very few have bothered switching to DAB, and a large sector of the market, particularly younger audiences, are using online delivery instead. DAB is the new DAT.

But as I have already hinted at, the biggest problem when it comes to digital content is the fact that the interim report practically ignores the user-generated revolution. Yes, there is the obligatory mention to UGC and Youtube, but then the drafters have no idea what to do with it other than to mention that digital technologies lower barriers to new providers such as "the wide range of services now catering to ethnic minority communities and to specialist interest, the development of community services, of user-generated content whether on YouTube or on social networking sites". In one dismissing paragraph the UGC revolution is relegated to fringe status akin to Gaelic stations, World Music and train-spotting. So, UGC is on the radar, but the report makes sure that whenever it talks about content, it is talking about institutional content. In a Pratchean use of the word, it is clear that whenever they talk about traditional Content, we are supposed to read it with a capital C, while what the rest of us do can be written without pressing the shift key.

What I find disheartening is that the report insists that the solution to UK-based content creation is through more funding for the BBC, ITV and Channel 4. We can then expect more failed digital channels from the BBC, as well as more publicly supported mind-numbing reality shows, when the real innovation lies elsewhere. BBC 6 Music struggles to top 300,000 listeners; BBC 3 struggles with similar viewing figures, while the high-brow BBC 4 has been criticised for not offering good value for money. In contrast, a live recording of DiggNation in London attracted ten thousand screaming fans; Facebook boasts 8.5 million UK users, Youtube has been visited by 20 million Brits in 2008, while Wikipedia had 9.6 million users last year. There is a generational shift that policy-makers seem to completely ignore. Their idea of digital content seems to extend to whether they can listen online to the Today programme on Radio 4. The turning of the century completely missed them.

It is perhaps unfair to compare hits between media, but this is a point that has to be made. The next generation of content users has no interest in the BBC and DAB, they want Youtube, Twitter and Facebook. It is about time policy should be drafted to reflect that nowadays the word content goes further than Radio 2.

Censorship UK

Today Britons have woken up to the fact that they are also browsing the Internet behind a firewall similar in some ways to the much maligned Great Firewall of China. Every news outlet is reporting that Wikipedia has been censored in the UK. Several UK ISPs are accused of filtering an article featuring the cover art for the Scorpions 1976 album Virgin Killer (pictured non-offending cover, unless you sufer from extreme hairtyle consciousness). Customers of several UK ISPs found out that they would get a 404 error when trying to access the page (a list of affected ISPs here). I'm in Costa Rica, so I can see the offending image, but plenty of users have complained they cannot.

As it is common with any hint of censorship on the internets, the virtual post-digestive refuse has hit the rotational air-propelling device, and the collective British geekdom has risen up in arms to protest the action. To me one of the most interesting aspects of the whole thing is that it has uncovered the underlying architecture of control that rests underneath the everyday workings of the UK's network connection. There is a little known UK watchdog called the Internet Watch Foundation (IWF), which is the "UK Hotline for reporting illegal content specifically: Child sexual abuse content hosted worldwide and criminally obscene and incitement to racial hatred content hosted in the UK". What a remit! The IWF does not have direct power to censor the web, but it can issue notices to participating ISPs, which can voluntarily add the offending website to a blacklist (called Cleanfeed). If a user tries to access any of the blacklisted URLs and IP addresses, he/she is redirected to an HTTP proxy server that blocks the offending content. Any other traffic goes through the proxy filter. The IWF has admitted doing the filtering:

"A Wikipedia web page, was reported through the IWF’s online reporting mechanism in December 2008. As with all child sexual abuse reports received by our Hotline analysts, the image was assessed according to the UK Sentencing Guidelines Council (page 109). The content was considered to be a potentially illegal indecent image of a child under the age of 18, but hosted outside the UK. The IWF does not issue takedown notices to ISPs or hosting companies outside the UK, but we did advise one of our partner Hotlines abroad and our law enforcement partner agency of our assessment. The specific URL (individual webpage) was then added to the list provided to ISPs and other companies in the online sector to protect their customers from inadvertent exposure to a potentially illegal indecent image of a child."

Nice of them to decide for us. However, this type of hamfisted approach tends to have some other consequences. ORG reports that because one page in Wikipedia has been blacklisted, all of the customers of the filtered ISPs appear to Wikipedia to come from the same IP address (that of the proxy server). The other consequence of course is that censorship online usually backfires spectuacularly (the so-called Streisand Effect), as web users are made aware of the attempt to censor, the offending content goes viral and becomes more popular than it was before the actions. As evidence of this, the Virgin Killer page averaged 500 visits during the previous months (with a peak view of 4,000 views during November. However, the page received 126,000 views yesterday.

So, panGloss warned us about the potential problems of Cleanfeed some time ago in a SCRIPTed editorial. I hate slippery slope arguments, but in this case they are justified. Today it is one page in Wikipedia, tomorrow Google? Facebook? Our much-touted freedoms are slowly being eroded, and the comparison with China is not an exaggeration. We tend to look at the Chinese web and feel a misplaced sense of superiority and smugness. The words mote, beam and eye come to mind.

So, now I leave you with the flowchart that regulates what you can view on the internet in the UK. Enjoy your freedoms.

Update: Interesting bump in sales for the album.

Googleverse: Utopia or Dystopia?

It is a bit tired to say that Google's stranglehold on the search engine market seems unshakeable. To offer a couple of ironic factoids, I found the accompanying image using Google Image Search, and this blog is published in Blogger, a Google-owned service.

Many people have been warning us about Google's dominance for a while, yet we continue to use it for a variety of reasons, from convenience to the fact that it works well. Spiegel Online has published a very interesting article on the growing backlash against some of Google's more controversial services, such as StreetView, but also building a disturbing picture of the amount of data mining going on at the Mountain View company. Gmail users have become accustomed (and immune) to warnings about the practice of Google searching for keywords in their messages to target contextual advertising. iPhone users can also be tracked when using Google Maps in their mobile devices, which tell Google exactly where you are.

The practices at Google cut right to the heart of the modern debate about Internet privacy that is usually the topic of discussion in various conferences I attend. Those who do not see privacy breaches as a problem, tend to offer three arguments. One is whether there is safety in numbers. With the amount of data that Google gathers, is it possible for any person to actually misuse it? Your personal information is lost in petabytes of meaningless noise available online, so it is unlikely that someone will actually access it. The other argument is, so what? If the user gets a free and useful service, who cares if the company makes a buck by providing context advertising to the consumer? The third is of course the issue of privacy itself. If you have nothing to hide, then why bother about what information is held by Google or any other company?

These are legitimate answers to the issue of privacy, but I cannot help being slightly worried by the far-reaching power of Google. I am also concerned about the environment of unaccountability in which Google is allowed to operate. In Europe we have data protection law for a reason, and perhaps it is time for regulators to look at Google's practices and start asking questions about lines crossed.

It is clear that privacy is a growing concern in the media due to the widely advertised privacy blunders by government officials. We also have a public debate prompted by the Daily Mail editor riling about growing privacy protection due to human rights legislation. If privacy is back on the menu, then the largest online perpetrator should certainly come under regulatory scrutiny, regardless of whether one believes that its practices are not such a big threat to our rights.

A progressive IP and IT policy in the US?

One of the defining characteristics of U.S. IP policy in the last 16 years has been an inevitable move towards stronger protection, while in IT the move has been towards laissez-faire deregulation of media. Many people politically classed as either liberal or progressive have come to expect nothing but bad news coming from the United States government. From the forceful push towards maximalist IP protection pursued by the USTR, to idiotic misunderstanding of the Internet, every time the U.S. passes new legislation in these topics, we brace ourselves for another round of corporatist encroachment.

IP Watch is reporting on some names as the potential advisors to the Obama administration. I have to say that I like the list so far. Arti Rai, Beth Noveck, and Larry Lessig are amongst those mentioned, which seems to fare really well for the next American administration. The top items on the agenda would be network neutrality, bilateral trade agreements, WTO's negotiations, patent reform, and copyright enforcement. It is possible that placing the right people at the top of imporant public policy insitutions would help to steer the United States into a more progressive IP and IT policy. A more liberal administration could also be more friendly to open source initiatives, much like some parts of the European Union are.

Having said that, I have to be a realist and accept that the copyright and telecomms industries have a lot of power regardless of who's in charge, and that it is likely that things won't change that much.

The predictive power of computers

I have always been interested in the depiction of computers in popular fiction. One early computer meme represented the percentage crunching computer/robot that would helpfully tell the protagonists the percentage rate of survival for X situation ("You have a 2% chance to survive that jump").

While computer predictions are not as reliable as their fictional counterparts, they are still used in complex simulations involving anything from climate change to traffic flow. I have been thinking about this for several reasons, including research into networks and because of the latest release of iTunes. The system comes with a recommendation system called Genius. You select a song, and Genius will generate a playlist that "matches" it. I am not sure about the parameters used to generate the lists, but I tried seeding lists with Pat Benatar's "Love is a Battlefield" and New Order's "Blue Monday", and I was quite impressed with the resulting lists. However, trying anything other than Anglo Saxon-centric pop and rock will slice Genius' IQ in half. I tried "Bongo Bong" by Manu Chao, and even several Soda Stereo songs, and all I got was a message claiming that Genius could not generate a list. I was also underwhelmed by the Classical music lists. Apparently, I am not the only person who thinks that Genius is not particularly clever (a hint to Apple: do not name any system with synonyms for intelligence, it will only generate derision from the blogosphere). If you want to try to go into Microsoft world and purchase a Zune, the recommendation system appears to be rather good (I do not know anyone who would willingly buy a Zune though).

Recommendation systems are not new. Amazon's recommendation algorithm has been with us for years, and it really works. It seems like every week there is talk of a new startup that promises to bring social intelligent systems to make our lives easier by letting our friends recommend stuff to us; the Guardian's Technology podcast seems to be almost entirely about social recommendations lately. Similarly, talk about the semantic web and intelligent systems has been around for ages.

I remain sceptical about many of these predictive services, particularly about social applications. While I find it interesting that I share musical tastes with people in my Facebook friends list, I will not make many decisions based on that. I am also sort of troubled about the fact that our tastes can be so easily predicted. I like fooling the system from time to time! Nevertheless, with the advent of the petabyte age, some pictures are starting to emerge. It seems like we make decisions in a more predictive manner than we previously thought. Serendipity and inventiveness are actually rare, and most of us are remarkably simple creatures, responding to the world in a predictable manner.

Can we code these deterministic responses into law? Although I would like to answer no, my logical brain says yes. But what do I know, I have some shameful songs in my iPod.

It's all about the data, stupid!

Electronic evidence and data gathering are becoming an important part of modern litigation. When Viacom requested data from Google pertaining their massive litigation, they did not just ask for all the usage information related to their own content, they requested everything. All information about every single video uploaded to YouTube ever, and they got it, together with all removed videos. That is a lot of information (reportedly, the user database itself is 12TB worth). Now some poor sod at Viacom is going to have to go through that...

Welcome to the Terabyte age: gazillions of data stored in servers all over the world. What is useful and what is not? Is this even a relevant question any more? Chris Anderson, of Wired and Long Tail fame, thinks that the deluge of data heralds the end of theory as we know it. All of our preconceptions should be re-written in order to cope with an increase in storage and processing power that will not peak any time soon.

He uses Google as the typical example of where things are headed when you have seemingly unlimited storage space. Google gathers 1 Petabyte of information every 72 minutes, an amount so humongously large, so mind-bogglingly huge, that all comparisons break down. No individual can sift through that amount of data to find useful information about it, so Google relies on "intelligent" sorting software in order to go through the data and answer questions about it. Google then doesn't need to have people reading web pages in order to know what they contain, all it needs is indexing and sorting software that will correlate one set of data with another. So, Google "knows" that this blog is probably related to some vague legal issues not because they have read the small blurb on the blog, or because I classified it as such, but because there is a high incidence of certain words, as well as the tags I use. This also allows the reasonably accurate allocation of AdWords and other paid search material.

All of the wealth of data means that we are finding it more and more difficult to write theories that account for all of the stored facts. Physics has become a statistical science where cats can be alive and dead at the same time. Biology has become a statistical science of billions of combinations of DNA. Our theories are simply not general enough to describe reality any more, only statistical approximations of reality. Anderson claims that all of the above heralds the end of theory:

"Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot."

This may sound awfully reductionist. Can we forget about analysis, and simply look for correlation about everything? If it works, why not? Think of how we study the law. Most of our legal theories work on the assumption that human beings are rational individuals acting on the pursuit of social/selfish/rational goals and interests. But what if what we are is simply a collection of responses that can be mapped and correlated at statistical level? We are increasingly leaving trace of how we think, not on what we write, but on what we click.

It is my reductionist claim that once we start going through the average clicking path of a person through a day, we will discover some unsavoury truths about our own nature.

Travel and "open" wifi

It's been quiet here in TechnoLlama for a while after knit-gate and the story about gold spammer executions being picked up by the always excellent Terra Nova. I am now off to Brazil for a conference, and as a reluctant traveller and Internet addict I am always very interested in finding adequate wifi coverage, which is not always easy and rather expensive.

For the last year I have been relying on BT Openzone, which came as part of my home broadband package. However, the free service was discontinued, and it has now been replaced with something called BT Fon. Fon is a really interesting experiment, an international network of users that agree to share their broadband connection with other Fon users. In their own words:

FON is the largest WiFi community in the world. FON is a Community of people making WiFi universal and free. Our vision is WiFi everywhere made possible by the members of the Community, Foneros. We share some of our home Internet connection and get free access to the Community’s FON Spots worldwide!

The idea is ground-breaking, yet amazingly simple when you think about it. Road warriors will be used to picking up all sorts of closed networks when roaming the wireless space. The idea is to let you connect to those wireless networks in exchange for making your own service available to others. This seems to me to be a logical step-up of the sharing open ethic exemplified by tools like Wikipedia, Creative Commons and open source licensing. By sharing your resources you also benefit by being able to connect to a network anywhere in the world.

Those legally inclined will certainly get suspicious right now. What are the legal implications? What are the liabilities involved if someone uses your network to download porn, terrorist manuals, unlicensed copyright works, or boot-leg Rick Astley videos? Is there a chance that sharing your connection will make you vulnerable to attacks? Fon say that the connection is secure, and because all users need to user their login in order to connect, a record will be left somewhere that access to that dodgy site was not yours, but it will identify which username did it.

I have to admit that I really like the idea, it has all of the sharing elements that I like about the participatory web. However, I am sort of sceptical about BT's reasons for joining the Fon network. Call me cynical, but I am certain that BT realised that revenue from roaming wifi is not what it should be, and that it was partly to blame on people like me who got access as part of a package. Now they can charge me whenever I need to go online on the road and I cannot find a Fonero. Hopefully, the network will become big enough soon enough.

Who broke the Internet?

The Internet is broken. You may have noticed some of the signs: slow connections, sluggish downloads, constant disconnections, lost emails, tons of spam. Who broke it, and how can we fix it?

This was part of a very interesting talk by Chris Marsden at BILETA 2008, an earlier version of which can be found on SCRIPT-ed. At the heart of the topic of network neutrality lies the very real fact that there is a serious problem with current architecture. Bittorrent protocol is a wonderful way of transferring large files, but it is resource-heavy. Small number of users can hog as much as 70-80% of the network traffic at any given time.

How can this be stopped? Many ISPs have started working on traffic-shaping, throttling, and using other restrictive tools. The justification for these practices is that by throttling bittorrent protocols, it is possible to make sure that the rest of users will not see their service affected. In an unfair situation, placing caps on protocols guilty of hogging the network would be a good idea. To me the "gotcha" moment came when I saw a chart by George Ou, presented at the Network Neutrality hearings in Washington. To me, this is incontrovertible proof that Something Must Be Done:

The problem is of course that current solutions to the problem, such as rate limiting, are inherently unfair, as they castigate the legitimate and occasional bittorrent user as well as the hardcore seeder. It also can be partially circumvented through encryption, although I have heard it on good authority that encryption can be easily shaped.

Bob Briscoe, a researcher at BT, is proposing a change to the network architecture that has been in place since 1986. Current TCP/IP traffic is handled through the additive increase/multiplicative-decrease (AIMD) algorithm, which shapes current congestion through the network. Briscoe and others argue that the current algorithm is unfair as it allows a bittorrent user to open many streams for sharing the same content, while someone browsing will only use one session, which favours the bandwidth hog in detriment of the normal user. The proposed change is to allow for a weighted TCP algorithm, which will allocate similar bandwidth to users, regardless of the number of streams they have open at the same time.

I find Broscoe's proposal elegant and logical, but most importantly, I like the idea of fixing an architectural problem through architecture. Amongst all of the regulatory debates about the subject, I find that the best solution is not to legislate, but to fix the broken protocol.

Now, if I can only get my BT Vision box to work, I will be a happy man.

YouTube v Pakistan: a test on resilience

Technology news sites have been reporting this story, but I think that its importance has been downplayed. Last Friday, Pakistan Telecom blocked access to YouTube inside the country as a measure to censor the site because it contained criticism to the Prophet Mohamed (PBUH). However, this blocking exercise went slightly awry, as the blockade resulted in a two-hour global outage of YouTube.

In order to understand what happened, one has to understand how things get to us online. Sites are nothing but documents hosted in machines around the world. Those servers have IP addresses, which are also connected to the domain name. Browsers find out how to connect to those servers via a Domain Name Server, which gets to resolve addresses into domain names by updating them through the root nameserver system. There are thirteen such root servers, which know that www.youtube.com is hosted in servers with a certain IP address. What Pakistan Telecom did on Friday was to locally hijack that domain name, and told its local DNS servers that YouTube was hosted at a machine that was not the one where the content is normally hosted. This was a server in Pakistan setup to handle the requests to YouTube. The problem occured when the fake information was mistakenly communicated to one of the root nameservers hosted in Hong-Kong, and once there it was replicated to all of the other nameservers.

Depending on how often your ISP updates its tables, you would not have had access to YouTube for a period of time on Friday. YouTube and Google technicians found the source of the problem, and corrected it.

This case has several interesting IT Law implications. Firstly, there is the question of regulation through choke-points, the firewall system. This has proved to be rather more efficient than it was previously believed, but it is still easily bypassed if you know what you're doing. Pakistan operates inside a firewall, as I experienced when I visited the country in 2006, but because of VPN access, I was able to easily circumvent the ban.

The other issue is that this incident has unearthed a glaring vulnerability to the global domain name system. If you want to bring down a site, all you need to do is to trick one of the root nameservers into passing incorrect information to the others in the network. This in my opinion has tremendous governance implications.

Another question, could YouTube sue Pakistan Telecom?

Update: Seems like the guilty party was not the root nameservers, but routing tables.

New media: old wine in new bottles?

I fear to utter the word "paradigm shift" because it is overused (and often misunderstood). However, 2008 is promising to be a year of true changes in the media environment. If the markets shift, will we get the regulation of the new market right?

Why do I say that the media environment is changing? Change has been in the making for a while, all of the techno-shamans feel it in the air, casting their digital bones to try to see in which direction it will go. The download generation is coming of age (next year will be a decade after Napster, a decade!) This means that there is an entire market of people who have never and will never own a CD. This generation watches YouTube more often than they watch TV, and are more likely to have the attention span of a guppy, and therefore are notably promiscuous with their tastes. Witness the dawn of the Long Tail generation.

This was evident in the very good article in The Guardian about the stars of video bloogging. When thousands of UK teenagers show up for the recording of DiggNation, you know that there is something big going on. The new media environment is a nebulous, niche market place, where no two people have the same likes, and talking to someone from outside your area of interest is akin to talking to someone from Mars (if there were such thing as people from Mars, but I digress...)

So, the new economy is driven by IP protection, right? After all, the creative industries are all about intellectual property maximalism and its twin objectives: protection, regulation, and control (fine, not twins, more like triplets, but I digress again...) The problem for IP maximalists is that the last decade has seen the rise of the free society, or as Chris Anderson calls it, freenomics. The creative industries have to compete against free competitors, as the cost of copying digital works approaches zero. Think of the problem using the old goods vs services dichotomy. We have different systems of law to govern provision of goods or services, because they are very different in nature. Intellectual property sat uneasily between both worlds. While copyright works were sold as goods, they displayed a lot of the features of services. We even had case law which dealt precisely with the question of whether software was a good or a service.

However, digital technology has shifted that duality towards services. As we see more and more methods of providing creative works online at negligible cost, these works become services. Amazon still sells goods, but it is also providing an aggregated service through which I can download books and music (or I could, if I lived in the U.S., grumble, grumble...) When the copyright industry finally realises that we have shifted almost fully into the services economy, then we might see some real change. In the meantime, we are stuck with laughable attempts at controlling old media through outdated regulation methods, such as copyright term extension for performances, or the much-maligned ISP liability reform.

I'll now be off to punish myself because I used the phrases "paradigm shift" and "long tail" in the same post (at least I withstood the temptation of using the word "zeitgeist"). Forgive me reader, for I have sinned.

Chinese search engine wins copyright infringement case

Various news sources have been reporting that the Beijing High Court has ruled in favour of popular search engine Baidu.com in a copyright infringement suit brought by major record companies. The dispute arose by the existence of a dedicated music search function called Baidu MP3, which allowed users to find infringing copies hosted by third parties. Four major record companies (Universal, EMI, Warner, Sony BMG) and their subsidiaries sued Baidu in 2005 for copyright infringement, alleging that the search engine allowed the download of 137 of their songs. A court ruled in favour of Baidu in November, but the music companies appealed.

This seems like a reasonable decision which lends credit to those of us who believe that linking to infringing material is not infringement as such.

This seems like a good story with which to start 2008. Happy New Year!

ISPs acting on Bittorrent

(via Ars Technica) EFF has released a report outlining several practices at Comcast, a popular American ISP. This is controversial, because some ISPs sell themselves as high-speed with no restrictions, which would be violated by bandwidth throttling and traffic shaping. Comcast claims that it delays bittorrent traffic, and does not block it.

What is traffic shaping? It is a method on imposing speed restrictions on certain types of traffic, such as Bittorrent, under the assumption that it slows down the quality of service to those who are not downloading at the moment. Regardless of who you believe, bittorrent hogs a lot of bandwidth in what appears to follow a Pareto distribution where few users use up a disproportionate amount of bandwidth. If that is the case, why should we get upset by ISPs imposing some form of traffic restriction on users that use up a lot of resources? One answer is that the assumption that bittorrent traffic is illegal is not always warranted. Just this week I used bittorrent to download Open Office, Knoppix and SuSE Live CD, all perfectly legitimate open source software. I also downloaded a copy of XP SP2 as my (also legitimate) CD was giving me problems when installing it in my MacBook.

Having said that, it is clear that the vast majority of traffic that uses bittorrent protocols is infringing. If we start with that assumption, it seems fair that other types of traffic be given priority in the network. As an online gamer, I hate network lag and poor quality of service, and any technical issue that can be used to solve such issues should be applauded.

Downloading TV shows made easy

(via David Berry) This is an awesome guide to setting up a show downloading service on your computer. With NBC terminating their contract with iTunes in the USA, and with iTunes UK TV shows being sold at a prohibitive price, consumers are voting with their feet and choosing to download their favourite telly from BitTorrent sites.

The British have welcomed download technology with open arms. UK viewers are the top TV downloaders with 38.4% of the market, and larger numbers of Brits are finding entertainment in the small YouTube box.

By the way, if you are looking for tips, download the Miro player, and bookmark tvRSS.

Clogging the Internets can land you in trouble

The Washington Post reports on action being taken by some ISPs in the United States to curb high traffic. Comcast is reportedly terminating users' accounts based on high bandwidth usage, although it seems like there is no specification of how much is too much. I'm surprised by the move, as it has the potential to get heavy-users to leave Comcast and choose other networks. Then again, this may be precisely what is intended.

The natural techie reaction to stories like this is to cry foul. In fact, every fibre in my body seems to be wanting to write "keep your hands off our Web!" However, the more I think about actions like these and about the whole net neutrality argument, the more I think that something needs to be done. Let's assume that bandwidth is a limited resource (which is in itself a contentious technical statement after all). If small numbers of users take up most of the bandwidth, then that would affect how others are able to browse at peak times. Why not place restrictions on how much can a single person download?

Appealing as this argument may be, I'm still troubled. How much is too much? Who gets to determine the thresholds? Is there a technical definition that we can all agree on?

Heady regulatory questions so early in the morning. I'm off to clog the web further by viewing The Internet is for Porn for the 100th time.

iTunes sells, Eminem sues

It has been a mixed week for iTunes. Earlier this week it was reported that the site has hit 3 billion downloads, and it is now the 3rd largest music retailer in the United States, ahead of the likes of Amazon. But now the BBC reports on a suit initiated by rapper Eminem against Apple for copyright infringement, alleging that the company makes his songs available to the public through iTunes without his permission.

If the case is not settled out of court, this could prove to be a landmark litigation. Music publishing and distribution agreements are famously complex legal matter, filled with tortuous clauses that try to foresee any eventuality. For average artists, the agreements involve a record label (nowadays likely to be a subsidiary of a large media conglomerate), the publishers and the distributors and retailers. Digital retailers like Apple have existing agreements with publishers in which they are licensed to distribute the work through iTunes.

The problem in this case seems to be that Eminem operates outside of the normal channels. He still releases his records (or more accurately, released) through an independent record label called Eight Mile Style. In the analogue economy of bricks, mortar and limited shelve space, such small labels need the complex distribution chains available to the corporations, so they will usually have an agreement with a publisher or distributor in order to get their product to the stores; in this particular case Eight Mile Style has a distribution agreement with Universal Music. However, in the digital realm of the long tail, such arrangements are not required because popular artists like Eminem can effectively cut out the middle person and deal directly with the retailer. In their complaint, the plaintiffs argue that Apple currently sells the music based on their existing agreement with Universal, but that they have never allowed Universal to licence their works to Apple. This is an extremely interesting claim, I'm curious to read the current publishing agreement between Eight Mile Style and Universal to see if this is correct.

This could have huge repercussions throughout the industry, as it would be possible for artists to obtain more per download.

Social network me that neutral Web 2.0

I never thought I would say this, but I've actually read something written by Andrew Orlowski which has not prompted me to throw things at my computer screen in anger. In an article on Net Neutrality, he has actually uncovered something that has been worrying me since I first heard about the debate. The big problem is, how do you define the term? You can get five different definitions from Wikipedia alone.

But this is not a post about net neutrality, I don't know enough about the subject to comment intelligently on it. However, this is a Monday morning rant against easy sound bites and complacent application of terms just because they sound good. Web 2.0 is one of those terms, similar to social networking and other neologisms prevalent in technology writing.

BitTorrent and traffic shaping

One of the problems of moving house is getting new providers for all kinds of services. Choosing a new broadband provider is a worry, particularly if you were happy with your old ISP. Telewest (now Virgin Media) were pretty good in general, even despite the fact that they were too quick to disconnect my web space and to get rid of all my files the very second after mys subscription ran out. Now I am with BT, and while I have been quite happy with some features of their service (free wireless roaming hours, yay!), I have found to my surprise that they seem to cap BitTorrent transfers. I found to my dismay that I could not get a BitTorrent load exceeding 30 kbps.

How could BT do this? And why? Welcome to the weird and wonderful world of traffic shaping. Even the most conservative estimates tell us that BitTorrent traffic makes up for 30% of all bandwidth used in the world at any given time. At peak hours, some have estimated the figure to go as high as 55%. This is a large chunk of very expensive bandwidth. Some ISPs have decided to regulate this by applying traffic shaping rules that will provide caps on the bandwidth allocated to P2P traffic, particularly BitTorrent. One Quality of Service (QoS, talk about euphemisms!) software provider describes the traffic shaping done by their software like this:

QoS Class	Protocols	Priority	Guaranteed	Maximum
VOIP	SIP, H323, Skype, MSN Messenger	High	1Mbit/s
P2P	eMule, EDonkey, KaZaA, Gnutella, BitTorrent, Direct Connect	Low		256Kbit/s
SHELL	ssh, telnet	High
BULK	ftp, smtp	Low
DEFAULT	Unclassified	Medium

This assigns a 256 kbps to all P2P traffic, which is very low indeed.

Should ISPs get away with this? Trying to see their point of view, one could argue that ISPs are entitled to make sure they provide an adequate service to all their users at all times, and that a few people using a large percentage of bandwidth to download their next episode of Lost is not a particularly efficient way to allocate resources. However, ISPs make a big point in their advertisement about how fast they are, and how you can get 8 mbps, or even 16 mbps in their ultra-fast network. Who are they trying to entice with these offers of speed? P2P traffic of course! After all, you don't need 16 mbps to read the email from Auntie Rita. It is highly unlikely that people will use it to download cookie recipes either.

So, if I pay BT a premium price for their fast connection, I very much expect to get top download speed for whatever protocol I choose to use. It should also be pointed out yet again that BitTorrent is used to share legitimate content. I downlaoded both OpenOffice and Open SuSE Linux with the protocol. Also, we must not forget "legitimate" file-sharing. After all, content providers keep advertising that we will be able to download movies in the future. Will these also be affected? Probably not, as most of this traffic will be offered through proprietary systems, like iTunes, or through sanctioned broadband systems like Cache Logic's Velocix.

So, what to do? Should I just sit and take the slow BitTorrent bandwidth? No way! There's always a technological solution for every technological attempt to regulate technology. Armed with encouraging previous experiences that someone somewhere will have an answer, I was able to find out that the best solution is to turn on encrypted connections in the BitTorrent client (in my case Azureus). This will not cover all downloads as the sending client needs to have encryption turned on as well, but it will drastically improve download speeds (from 30 kbps for three torrent files to almost 200 kbps). Most moderns clients will have encryption turned on.

So, if you are moving house, check out the list of "Bad ISPs" (BT is not listed, funnily enough).