Microsoft developing BitT alternative 'Avalanche'

[Chmee]

Microsoft readies BitTorrent alternative
Avalanche technology could make it easier to distribute big files over the Internet

By James Niccolai, IDG News Service
June 16, 2005

CAMBRIDGE, ENGLAND - Researchers at Microsoft's (Profile, Products, Articles) Cambridge, England, labs are developing a file-sharing technology that they say could make it easier to distribute big files such as films, television programs and software applications to end-users over the Internet.

Code-named Avalanche, the technology is similar to existing peer-to-peer (P-to-P) file swapping systems such as BitTorrent's, in the sense that large files can be divided into many smaller pieces to ease their distribution. End users request the file parts from other users' hard drives and reassemble them to create the original file.

Such systems can scale well to serve millions of users, and reduce the bandwidth and computing costs of sending content directly to users from central servers. Some have also irritated publishers who complain the services are used to share copyright works illegally.

The problem with existing systems, according to Microsoft, is that people sometimes have wait a long time to receive the last, "rare" pieces of a file. This is made worse when clients drop off line unexpectedly and creates bottlenecks when only a few clients have files that are in high demand.

Avalanche goes a long way to solving these problems, according to Peter Key, joint head of the systems and networking group at Microsoft's research labs in Cambridge, during an open day on Wednesday.

It does this by encoding the file pieces at the server with a special algorithm before they are distributed. Each encoded piece contains information about every other piece of the original file, so users don't have to collect every last piece in order to reassemble the whole, Key said.

"Each encoded piece has the 'DNA' of all pieces in the file," another Microsoft researcher wrote. "A given encoded piece can be used by any peer in place of any piece."

When PCs in the Avalanche network receive encoded files, they randomly create new encoded files from the ones they have collected, and these are sent to other peers. When a user receives enough encoded files, they assemble them to make the original.

The system differs from BitTorrent's eponymous software in a few ways, Key said. It does not depend on central servers, called "trackers," to orchestrate the download. The Avalanche client on each PC shares the files automatically among users; they do not look at other users' hard drives to find what they want. And the system works well in smaller networks, such as a corporate intranet, he said.

Perhaps more importantly for content creators, Microsoft claims its system prevents users from redistributing copyright material, because Avalanche will only forward files that have been signed by the publisher.

Microsoft has developed a prototype of Avalanche and is testing it by using it to distribute software applications to several thousand of its software beta testers, according to a research engineer demonstrating the software in Cambridge. The company has distributed a 4GB application in as little as a day, down from about two weeks when it sends a program directly, he said.

The software may also be interesting to TV broadcasters and movie studios. Microsoft has been in talks with both groups, and Avalanche may be introduced to users in the U.K. as early as next year, he said.

The British Broadcasting Corporation (BBC) began testing a service last month that lets people download TV and radio programs using a P-to-P system from Kontiki (Profile, Products, Articles). It is not looking at Avalanche currently, but will put the contract out for public tender before launching the service, said Chris Charlton, a BBC spokesman.

The concept behind Avalanche is impressive, according to Mike Thompson, principal research analyst with Butler Group, in Hull, England, who saw the technology demonstrated. But it faces two problems of perception, he said.

"Firstly, Avalanche is a mirror of P-to-P models that are coming under scrutiny for allowing illegal distribution. I believe this idea of 'good' and 'bad' P-to-P for file-sharing of copyright material will create a deal of confusion.

"Secondly, despite the 'pull' nature of the model and the security that should allow only the file to be accessed, Microsoft has had issues around security in the past -- IIS [Internet Information Server] being the clearest example of a secure solution that wasn't. I think the P-to-P network would be a prime target for the dissemination of viruses, despite Microsoft's assurances that it is 'safe.'"

Still, Avalanche is "an excellent take on P-to-P," Thompson said.

Link

[General Zod]

Amazing that Microsoft is developing the technology, but it's nice to finally see a bigger company getting behind it. Though I'll be genuinely surprised if it actually manages to wind up outperforming existing bittorrent clients.

[Seggybop]

The creator of BitTorrent calls BS on it.

[Xon]

If anyone was interested in a legitimate usage of P2P distribution with reduced bottlenecks, it sure as hell is Microsoft.

Just using this technology for patch distrubution could dramatically cut their bandwidth usage for the windows updates servers and other Microsoft downloads. Microsoft.com easily is the most bandwidth intensive site on the entire internet.

And if anyone can make this work, it is Microsoft. They have some of the best people in R&D, with a truely redicious budget and some hidiously talanted peole on hand to actually implement stuff.

Despite what you might think about various Microsoft products, there is considerable technological know-how put into them.

[Praxis]

The problem is that many BT users are swapping illegal files and M$ will be sure to crack down on that. As a result they'll alienate a lot of the market for it. The legitamite users are more likely to stay with BT since there will be more users.


I'm all for less piracy and don't do any illegal fileswapping, but IMHO unless Microsoft allows the pirates, they're going to have a hard time getting people to switch over.

[Uraniun235]

The problem is that many BT users are swapping illegal files and M$ will be sure to crack down on that.

How would they do that?

[Mr Bean]

Exactly, I'm seeing a reduced faster veirson of Bit-torrent from MS, the only cavenant is that only MS can create the torrents and all we get is a downloader client, thus letting US use it to get patches super fast but not for pOrn and the like.

[Xon]

The problem is that many BT users are swapping illegal files and M$ will be sure to crack down on that.

How would they do that?

By having the client only accept files which have been digitally signed by a short list probably in Microsoft's control.

Doesnt stop someone else from reverse engineering the protocal and writing their own client/server

[General Zod]

Exactly, I'm seeing a reduced faster veirson of Bit-torrent from MS, the only cavenant is that only MS can create the torrents and all we get is a downloader client, thus letting US use it to get patches super fast but not for pOrn and the like.

That won't necessarily be a problem. You can't make torrent files using bittornado either, however there's third party applications out there that let you make them in case you have a client like that. What I could see M$ doing is creating a proprietary .torrent encryption which only works with their client. That way you can't use .torrent files made with another program.