Burningbird: Centralization cont.

May 06, 2002

Centralization cont.

In the previous posting, Dave attached a comment that returns us to the conversation about centralization. However, I don't expect that we'll generate any definitive answers to "what is centralization", as the folks who are interested in distributed systems and P2P have been working this issue for years, with only qualified success.

Centralization hasn't as much to do with technical points of failure or with issues of deployment as it has to do with control. Centralization implies a single point of control residing in an authority other than yourself.

Is weblogs.com centralized? Yes, from both a control as well as a technology perspective.

Webloggers can automatically invoke the weblogs.com web services, or use the weblogs.com form to invoke the services manually. If the weblog has changed, the blogname and URL are added to the publicly accessible changes.xml file, and eventually to the weblogs.com HTML page. It remains on this page for three hours.

Both the HTML page and the associated XML file and supported services provide a single location to check for recently updated weblogs -- a useful technology. However, this single location also leads to the service's vulnerability.

If the server goes down, weblogs.com is no longer accessible. Dave provided a temporary backup location but if you're dependent on automated processes to look for the updated information the temporary location didn't work for you (not unless you wanted to modify your application to point to this new location).

The downtime with weblogs.com last week demonstrates a technical point of failure for a centralized application. There are mechanical methods one can take to avoid this such as the use redundant backup servers, as well as the use of banks of servers. However, as we've seen with DoS (Denial of Service) attacks, if there is a determined foe any centralized service can be brought down.

The possibility of technology failure doesn't concern me in regards to centralized services, as for the most part, this isn't an issue. As we've seen, weblogs.com has rarely been down in the past and the only reason we're more attuned to the issue now is because of the rather lengthy downtime of the service this last week. Redundant backup servers would have prevented this, but as Dave has said, Userland is a software development company not an ISP. Backup servers are expensive and weblogs.com is a free service.

What does concern me about weblogs.com is the control: Userland has complete control over who shows on this list. And Dave has written filters for this list, as he's discussed, openly, at Scripting News. (Though these filters may have been removed.)

That's the danger of centralization.

Are there alternatives? Sure, there are other centralized locations of weblog updates. However, these are also subject to the same technical point of failure as well as issues of control.

Trying to decentralize a service such as weblogs.com would require a new infrastructure overlayed on top of the existing Internet to support the concept of centralized services that are decentralized -- in other words to support supplying and consuming information about recently updated weblogs at a single point, the location of which can change from day to day, minute to minute.

Semi-decentralized applications such as Kazaa and Napster don't provide the technology to solve this problem; they aren't providing access to centralized resources, they're providing access to files that can be located on any number of machines.

Until such an infrastructure is in place, we'll continue to use weblogs.com and benefit from the service, while understanding the limitations inherent with centralized services such as this.

Returning to the comment in the previous post, Dave also mentioned the hypertext link. Now the simple hypertext link truly is a decentralized technology.

Anyone can put a link into their weblog. There is no authority controlling what you can and cannot link to unless you pay attention to the ridiculous and unenforcable "rules" that some web sites publish about deep-linking. Web sites may require permission to access certain pages, but you can place the link on your page -- it's up to the person clicking the link and the web site to negotiate actual viewing of the page.

And if you have a weblog, there is no authority controlling who links to you.

Weblog A links to you and you link to Weblog B, creating an indirect link from A to B. Continuing this process, weblog B links to weblog C and C links to D and D links to E and so on until you have an unbroken chain of weblogging circles forming a living, dynamic community that cannot be controlled and cannot be stopped -- not without taking down the Internet, itself. And though some have tried, the Internet is too vast now to be controlled by any one authority.

Centralization. It's all about control.

Posted by Bb at May 06, 2002 08:05 AM

Comments

Hi Shelley, again there's no disagreement here. I wonder if we can make this stuff concrete. Here's a topic I'm very interested in. A way to make weblog content accessible through Gnutella, with no centralized hosting at all. I post an item to my Gnutella-space, it has some unique identifier, and is instantly available to anyone who searches for that ID on Gnuetlla. Any ideas on how this might work?

Posted by: Dave Winer on May 6, 2002 10:38 AM

Shelley: If none of you after years of trying to figure it out can decide on what 'centralisation' even actually *is* - how can anyone possibly figure out what to do about it?

Posted by: Rogi on May 6, 2002 11:26 AM

I submit that question #1 be: 'What, Definatively, Is Centralisation?'

And then, and then only..

'Now What Shall We Do About It?'

And then onwards...

Posted by: Rogi on May 6, 2002 11:31 AM

Rogi, we do _know_ what centralization is; we just don't all agree (*grin*)

For instance, I don't consider Gnutella to be completely decentralized -- it doesn't have a golden gateway (a way of logging into the network without pre-knowledge, such as a static IP or community server). Only a completely decentralized solution can't be controlled, can't be shutdown -- centralization is control.

Dave, posting a file through Gnutella is a piece of cake, and searching for it is no different than searching for Brittney's latest inane warbling. However, the Gnutella architecture is based on an assumption that lots of people have copies of same, that it is static, and that it can be propogated throughout the system without said danger of divergence.

Consider the resource you would post --Gnutella's store and forward arch would quickly duplicate this information (not "instantly"), but then you would have distributed static copies of, what? changes.xml? A weblog page?

Returning to the concept of "static" data --Brittney's latest warbling is a static file, unchanging. She may release new warblings, but they become new resources, not replacements for existing resources.

Our weblogs and changes.xml, on the other hand, are dynamic. Store and forward generates divergent copies of the resource.

This returns to my statement earlier -- existing infrastructures don't support completely decentralized access of centralized services (or data that dynamically changes for that matter).

The most we can hope for with weblogs at this time is that we have token control over the hosting of said weblog -- in other words we post the pages to our own servers, or to servers maintained by ISPs responsive to our individual demands. And that the tools are autonomous of the tool vendor (even to updates if we choose not to pursue said updates).

Governments can force us to take the weblogs down, and so can a DoS, but it's a start.

Sorry, side rant. What specifically were you thinking of storing within the Gnutella infrastructure?

Posted by: Bb aka Shelley aka Weblog Bosswoman on May 6, 2002 12:00 PM

"Rogi, we do _know_ what centralization is; we just don't all agree (*grin*)"

So, as was suddenly very clear to me, you (all in general) *don't* know what it is then. I've got two very high level tech types here - and even *they* can't even 90% agree with each other. :-)

"Governments can force us to take the weblogs down"

Ours can't. So there.

So, what is, *definitively*, 'centralisation'? You and DW tell us.

*grin* back, as we know that niether of you can answer that. heh. ;-)

Posted by: Rogi on May 6, 2002 12:19 PM

And BTW - 99.99% of the connected ppl on the planet are going in via an ISP - and if that goes down, they go down. So almost everyone is 'centralised' before they even get to Birningbirds weblog to discuss 'centralisation'?

Hummm? :-)

Posted by: Rogi on May 6, 2002 12:26 PM

If "It's all about control", then we should look at the sitation using that term. The word "centralization" may be confusing the issue.

In terms of control, the deciding factor is: How much can a third party remove or impede my voice? With this criteria, tools like Blogger and Manila are worse, and tools like Movable Type and Radio are better.

Of course, worse and better are relative. How much control do you need to feel comfortable?

Posted by: Andy Chen on May 6, 2002 01:32 PM

IMHO centralization is more about reliability and delivery than about control.

Reliability/delivery.... if (God forbid) terrorists nuke Kansas City 90+% of the remaining internet should still be workable.

Control.... Dave Winer has every right to charge a fee for his products. He has every right to control how people access it. A decentralized _uncontrolled_ file sharing system immediately changes that. For everyone.

If a private entrepreneur decides to build something like weblogs.com they have every right to control it. 100%. And if they can't deliver the goods - reliably - then I and everyone else has the right to go elsewhere, or build it better.

Posted by: Dave on May 6, 2002 02:01 PM

Sorry I got side-tracked this morning into AIM stuff. Amazing how quickly stuff comes on-line.

Posted by: Dave Winer on May 6, 2002 02:53 PM

"Reliability/delivery.... if (God forbid) terrorists nuke Kansas City 90+% of the remaining internet should still be workable."

With respect Daveski - I think you mean 99.9999999999999999999% of the internet.

Posted by: Rogi on May 6, 2002 03:47 PM

Dave, welcome to Movable Type.

Posted by: Bb aka Shelley aka Weblog Bosswoman on May 6, 2002 03:54 PM

Thanks Shelley, it's an interesting piece of software. BTW, about centralized vs decentralized, it all depends on deployment. You can deploy Movable Type in a centralized way (the people who edit the weblogs don't have control of their data) or decentralized. Same with Manila. Try that out, see if that doesn't settle all the issues. It's about how you deploy.

Posted by: Dave Winer on May 6, 2002 06:26 PM

Your right Rogi! Being increasingly more conservative in my middle age I tend to understate things like that. I believe 99+% of the web remained intact during the morning EDT hours on 9/11/2001 but without firm numbers to back it I have to state it conservatively. Thanks for the correction!

Posted by: Dave on May 6, 2002 08:47 PM

Dave(W) - yes that makes sense to me. At last.

Posted by: Rogi on May 7, 2002 03:58 AM

Daveski - the only damaging effect that Sept 11th (jeez, it still makes me shudder the memories of that day) had on anything internet-wise was on ppl who were using NYC based servers. It didn't really have any impact at all on the big picture. Nuking Kansas (Dog forbid) would not even make a ripple on the net either.

It's a big ol' thing, and it isn't, by *far*, just American ;-) Although the US is a big player in its distribution and development.

The biggest impact 9/11 had on the 'net was to make ppl *use* it to get info - and this bought down or made impossible to access due to overload a lot, if not all, of the biggest newsites.

'tis far far far bigger than any one country, or any one event.

Posted by: Rogi on May 7, 2002 07:11 AM

Hi Dave W -- I'd still have to go with my original interpretation of centralization being an issue of control, though I imagine deployment could be a part of it. Perhaps we can say that we equitably agree to partially agree?

Posted by: Bb aka Shelley aka Weblog Bosswoman on May 7, 2002 10:32 PM