The Register Internet could be 500 times bigger than we think
"The true extent of the Internet is not widely known, and according to a study published this week, it could be more than 500 times larger than we think. The authors claim that as much as 7500TB of data exist in places on the Web that no search engine has mapped, as compared to the 19TB on the familiar 'surface' web."

"The material BrightPlanet has uncovered consists mostly of public information - 95 per cent of this is freely accessible, with more than half residing in topic specific databases. Of these, the 60 largest contain 750TB of information. This exceeds the capacity of the normal web by 40 times. All this hidden material is causing a great deal of frustration, the company says, since people can't find it using normal search engines. Even NorthernLight.com, widely reported to have the largest percentage of the web mapped at 16 per cent, covers only 0.03 per cent of the total content including the 'deep' web."

MSNBC Experts probe Net’s natural defenses
"The Internet’s organic structure explains why it’s so resistant to random failures, but researchers now say those same features make it vulnerable to cyberattacks. The findings could help security experts strengthen weak links in the Net’s chain.

"They found that samples of the World Wide Web didn’t have a random structure: Instead, the connections exhibited a hierarchy similar to that found in naturally occurring networks such as trees and living cells, with a small proportion of highly connected nodes branching off to a large number of less connected nodes. The structure was the same at different scales, meaning that the results could be extended to the Web as a whole, they said."

"Although the structure is particularly well-suited to tolerate random errors, it’s also particularly vulnerable to deliberate attack, they said. If just 1 percent of the most highly connected Internet routers or Web sites are incapacitated, the network’s average performance would be cut in half, said Yuhai Tu of IBM’s T.J. Watson Research Center.

“With only 4 percent of its most important nodes destroyed, the Internet loses its integrity, becoming fragmented into small disconnected domains,” he wrote in a commentary published in Nature."

Nature : Science Update The missing links
"In an exponential network, there is a well defined ‘average connectivity’ for the nodes: most are connected by a certain number of links, and only a very few differ substantially from this average.

Barabási’s team says that there is another common kind of network that has hitherto been neglected: the ‘scale-free’ network, in which there is no meaningful average number of links - no ‘scale’ to it, in other words. In a scale-free network the number of nodes with a given number of connections simply declines as that number of nodes increases. Many nodes are linked to the network via just one connection; fewer have two, even fewer have three, and so forth. Unlike an exponential network, there remain small but significant numbers of nodes with many connections."

"...a cyber-terrorist armed with a map of a scale-free network could deliberately focus their attack on the few most highly connected nodes. Knocking just a few of these out would disable just about all flow of or access to information for other users, breaking up the webs rapidly into isolated fragments. This is the Achilles’ heel of the net, say the researchers, and defences against e-terrorism need to concentrate on making key nodes invulnerable."

redux [03.22.00]
The Christian Science Monitor Wars of the future... today
“Take the NATO bombing of the Chinese embassy in Belgrade several weeks ago. Rage spread across China and hackers from the mainland attacked the Web sites of the US Departments of Energy and the Interior, and the National Park Service. A subsequent attack brought down the White House Web site for three days. The attacks generated headlines across the country.

What the news media didn't report was that the US government had known for a long time that someone had been in its computer systems - they just didn't know who. Then, in a fit of anger, the Chinese hackers caused some real damage - and gave away the hidden "location" of several "backdoors" they had built in US government networks."

"The US Government Accounting Office estimates 120 groups or countries have or are developing information-warfare systems. According to a report issued by the Center for Strategic and International Studies, 23 nations have cyber-targeted the US."

redux [03.22.00]
News.Com Researchers find Web divided into 4 regions
"A study by researchers from Compaq Computer, Web portal AltaVista and IBM concluded that the Web has distinct regions, including some that are inaccessible to one another. The layout, researchers said, resembles a bow tie with four sections: a "strongly connected core," "origination" pages, "termination" pages and "disconnected" pages.

Within the core, the knot of the bow tie, Web surfers can travel smoothly between sites through hyperlinks. One side of the bow contains origination pages that allow surfers to reach the central knot. The other side of consists of termination pages that can be accessed from the core but are not linked back to it.

The final region consists of disconnected pages, which are cut off from the core but are connected to other areas peripherally.

"Webmasters and people doing e-commerce need to understand how to position their sites," said Andrei Broder, vice president of research for AltaVista. "If you want to have more international traffic, you need to be in a better connectivity position. It's always better to be in the center of the town than far out.""

IBM Almaden Research Center Graph structure in the web
"The study of the web as a graph is not only fascinating in its own right, but also yields valuable insight into web algorithms for crawling, searching and community discovery, and the sociological phenomena which characterize its evolution. We report on experiments on local and global properties of the web graph using two Altavista crawls each with over 200M pages and 1.5 billion links. Our study indicates that the macroscopic structure of the web is considerably more intricate than suggested by earlier experiments on a smaller scale. "

"In a sense the web is much like a complicated organism, in which the local structure in a microscopic scale looks very regular like a biological cell, but the global structure exhibits interesting morphological structure (body and limbs) that are not obviously evident in the local structure. Therefore, while it might be tempting to draw conclusions about the structure of the web graph from a local picture of it, such conclusions may be misleading."

Mappa.Mundi A Shared Reality
" In the beginning, maps were fiction. We perceived our world as myths defined by belief not geography. Maps of these imagined worlds came in many shapes and sizes, but they all mixed the unreal with snippets of the real world. The process of mapping the real world was one of going from geographies of ideas to maps of real geography. On the Internet, we will pursue a reverse path: maps of the Internet will progress from our current maps of network topologies to maps of virtual worlds that we build, maps of ideas and thoughts."

"Maps help us navigate. On the Internet, finding things has become the big challenge. Death by a thousand clicks is the bane of any net user. The reason? We are attempting to shoe-horn the metaphor of maps–tools for navigating complex spaces–into existing metaphors, such as the infinite book that is the World Wide Web.

The Internet is a network of many metaphors. The core infrastructure supports many protocols, and each protocol adopts a metaphor. Electronic mail uses analogies taken from a postal service. Streaming media started with a radio metaphor before evolving into a unique medium. The World Wide Web is also a metaphor–pages in an infinite book.

What is missing today is a metaphor that helps us tackle the problem of meta-information: information about information. As we look at a page on the Web, the logical next step is to find other pages that are conceptually near. Near, of course, varies on your point of view. Meta-information is what helps the Internet become smarter about organizing itself. As we develop the tools to describe Internet resources, to manage meta-information, maps will happen. Until then, we are stuck in a world of many facts: all content, no context.