A low-power, open source, open hardware mesh access point

the DARKNET PROJECT
http://arstechnica.com/open-source/news/2011/11/the-darknet-plan-netroots-activists-dream-of-global-mesh-network.ars
Netroots activists dream of global mesh network
by Ryan Paul

A group of Internet activists gathered last week in an Internet Relay Chat (IRC) channel to begin planning an ambitious project—they hope to overcome electronic surveillance and censorship by creating a whole new Internet. The group, which coordinates its efforts through the Reddit social networking site, calls its endeavor The Darknet Project (TDP). The goal behind the project is to create a global darknet, a decentralized web of interconnected wireless mesh networks that operate independently of each other and the conventional internet. In a wireless mesh network, individual nodes can relay data for other nodes, ensuring that the routing of data remains robust as nodes on the network are added and removed. The idea behind TDP is that such a network would be resistant to censorship and shutdown because there would be no central point of control over the infrastructure. “Basically, the goal of the darknet plan project is to create an alternative, more free internet through a global mesh network,” explained a TDP organizer who goes by the Internet handle ‘Wolfeater.’ “To accomplish this, we will establish local meshes and connect them via current infrastructure until our infrastructure begins to reach other meshes.”

TDP seems to have been influenced in part by an earlier unofficial effort launched by the Internet group Anonymous called Operation Mesh. The short-lived operation, which was conceived as a response to the Anti-Counterfeiting Trade Agreement (ACTA) and its potential impact on Internet infrastructure, called for supporters to create a parallel Internet of wireless mesh networks. The idea is intriguing, but it poses major technical and logistical challenges, and it’s hard to imagine that TDP will ever move beyond the conceptual stage. The group behind the effort is big on ideas but short on technical solutions for rolling out a practical implementation. During the IRC meeting, they struggled to coordinate a simple discussion about how to proceed with their agenda. Still, despite TDP’s dysfunctional organizational structure and lack of concrete strategy, their message seems to resonate with an audience on the Internet. And enthusiasm for mesh networks and decentralized Internet isn’t isolated to the tinfoil hat crowd; serious government programs aim at producing similar technology. Earlier this year, the New York Times reported on a US government-funded program to create wireless mesh networks that could help dissidents circumvent political censorship in authoritarian countries. As repressive governments continue to get better at thwarting circumvention of their censorship tools, dissidents will need more robust tools of their own to continue propagating information. The US State Department seems to view decentralized darknets as an important area of research for empowering free expression abroad.

A growing number of independent open source software projects have also emerged to fill the need for darknet technology. Many of these projects are backed by credible non-profit organizations and segments of the security research community. Such projects could find a useful ally in the TDP if they were to engage with the growing community and help mobilize its members in a constructive direction. Unlike TDP, the original Operation Mesh coordinators had specific technologies in mind: they highlighted the I2P anonymous network layer software and the BATMAN ad-hoc wireless routing protocol as the best prospective candidates. Both projects are actively maintained and have modest communities, though the I2P website is currently down. Promising projects like Freenet develop software for building darknets on top of existing Internet infrastructure. Another group that might benefit from broader community support is Serval, a project to create ad-hoc wireless mesh networks using regular smartphones. The group has recently developed a software prototype that runs on Android handsets. They are actively looking for volunteers to help test the software and participate in anumber of other ways. TDP members who are serious about fostering decentralized Internet infrastructure could meaningfully advance their goals by assisting any of the previously mentioned projects. The growing amount of popular grassroots support for Internet decentralization suggests that the momentum behind darknets is increasing.

DIMNETs
http://arstechnica.com/tech-policy/news/2011/11/anonymous-bit-dimnet-tries-to-be-a-hedge-against-dns-censorship.ars
Anonymous “dimnet” tries to create hedge against DNS censorship
by Sean Gallagher

With concern mounting over the potential impact of the Stop Online Piracy Act and claims that it could make the Domain Name Service more vulnerable, one group is looking to circumvent the threat of domain name blocking and censorship by essentially creating a new Internet top-level domain outside of ICANN control. Called Dot-BIT, the effort currently uses proxies, cryptography, and a small collection of DNS servers to create a section of the Internet’s domain address space where domains can be provisioned, moved, and traded anonymously. So far, over 4,000 domains have been registered within Dot-BIT’s .bit virtual top level domain (TLD). Those domains are visible only to people who use a proxy service that draws address information from the project’s distributed database, or to those using one of the project’s two public DNS servers. While it’s not exactly a “darknet” like the Tor anonymizing network’s .onion domain, .bit isn’t exactly part of the open Internet, either—call it a “dimnet.” Just how effective a virtual top-level domain will be in preventing censorship by ISPs and governments—or even handling a rapidly growing set of registered domains—is unclear at best.

How it works
Dot-BIT is derived from a peer-to-peer network technology called Namecoin, derived from the Bitcoin digital currency technology. Just as with Bitcoin, the system is driven by cryptographic tokens, called namecoins. Tobuy an address in that space, you either have to “mine” namecoins by providing compute time (running client software that uses the computer’s CPU or graphics processing unit) to handle the processing of transactions within the network, or buy them through an exchange with cash or Bitcoins. All of those approaches essentially provide support to the Namecoin distributed name system’s infrastructure. You can also get an initial payout of free namecoins from a “faucet” site designed to help bootstrap the network. The cost of entry is pretty low: currently, registering a new domain costs about 1.6 namecoins, which can be had for about five cents. Your registration isn’t associated with your name, address, and phone number—instead, it’s linked to your cryptographic identity, preserving anonymity. Once you’ve registered a domain, you can assign it by sending out a JSON-formatted update request, mapping the domain to a DNS or providing IP addresses and host names to be distributed through Dot-BIT’s proxies and public DNS servers. That information is then spread across all of the network’s peer systems.

Simple, right?
Namecoin’s approach heavily favors early adopters, since once you’ve registered a domain, you can transfer it to someone else—or squat on it until someone pays you for it. That seems to be what a lot of early .bit adopters are counting on. For example, using Firefox and the FoxyProxy add-on to surf .bit-land to audi.bit lands you on a “this domain for sale” page. But while Dot-BIT may allow for an anonymous and relatively secure exchange of DNS information, it won’t necessarily prevent censorship by ISPs. If the .bit top-level domain becomes the target of laws like SOPA, it can be shut down pretty quickly by cutting off the head—its own internal DNS—either through port blocking or other filtering. And since it lacks the anonymizing routing abilities of “hidden” networks like Tor’s .onion domain, it won’t protect the identities of publishers and users who visit sites that use a .bit name. At the moment, then, it’s not certain what purpose .bit will actually serve, other than as an experiment in novel ways to create a DNS—or someplace for hackers to spend their illicitly earned Bitcoins.

the ‘INFORMAL SECTOR’
http://globalguerrillas.typepad.com/globalguerrillas/2011/12/bitcoin-finds-its-intrinsic-value.html
http://globalguerrillas.typepad.com/globalguerrillas/2011/12/the-proliferation-of-darknets.html

It’s time to update/widen the term to accommodate a wider range of modern activity.  A darknet:

is a closed, private communications network that is used for purposes not sanctioned by the state (aka illegal).

Darknets can be built in the following ways:

• Software.  A virtual, encrypted network that runs over public network infrastructure (most of the US government/economy uses this method).
• Hardware.  A parallel physical infrastructure.  This hardware can be fiber optic cables or wireless.  Parallel wireless infrastructures (whether for cell phones or Internet access are fairly inexpensive to build and conceal).
• In most cases, we see a mix of the two.

Examples of Darknets:

• The Zetas have built a huge wireless darknet (a private, parallel communications network) that connects the majority of Mexico’s states.  Most of the other cartels also have wireless darknets and there are also lots of local darknets.
• Hezbollah (in Lebanon) runs its own fiber optic network.
• TOR.  A voluntary, decentralized ad hoc network that anonymizes network connections.
• Botnets (up to 4 m computers strong) that can be used for global private communications.
• Etc.  The list goes on  and on….

The future?  Darknets that power alternative economies.  A network layer for accelerating the dark globalization of the $10 Trillion System D.

SYSTEM D
http://www.foreignpolicy.com/articles/2011/10/28/black_market_global_economy
The Shadow Superpower
Forget China: the $10 trillion global black market is the world’s fastest growing economy — and its future.
by Robert Neuwirth / 10.28.2011

With only a mobile phone and a promise of money from his uncle, David Obi did something the Nigerian government has been trying to do for decades: He figured out how to bring electricity to the masses in Africa’s most populous country. It wasn’t a matter of technology. David is not an inventor or an engineer, and his insights into his country’s electrical problems had nothing to do with fancy photovoltaics or turbines to harness the harmattan or any other alternative sources of energy. Instead, 7,000 miles from home, using a language he could hardly speak, he did what traders have always done: made a deal. He contracted with a Chinese firm near Guangzhou to produce small diesel-powered generators under his uncle’s brand name, Aakoo, and shipped them home to Nigeria, where power is often scarce. David’s deal, struck four years ago, was not massive — but it made a solid profit and put him on a strong footing for success as a transnational merchant. Like almost all the transactions between Nigerian traders and Chinese manufacturers, it was also sub rosa: under the radar, outside of the view or control of government, part of the unheralded alternative economic universe of System D.

You probably have never heard of System D. Neither had I until I started visiting street markets and unlicensed bazaars around the globe. System D is a slang phrase pirated from French-speaking Africa and the Caribbean. The French have a word that they often use to describe particularly effective and motivated people. They call them débrouillards. To say a man is a débrouillard is to tell people how resourceful and ingenious he is. The former French colonies have sculpted this word to their own social and economic reality. They say that inventive, self-starting, entrepreneurial merchants who are doing business on their own, without registering or being regulated by the bureaucracy and, for the most part, without paying taxes, are part of “l’economie de la débrouillardise.” Or, sweetened for street use, “Systeme D.” This essentially translates as the ingenuity economy, the economy of improvisation and self-reliance, the do-it-yourself, or DIY, economy. A number of well-known chefs have also appropriated the term to describe the skill and sheer joy necessary to improvise a gourmet meal using only the mismatched ingredients that happen to be at hand in a kitchen. I like the phrase. It has a carefree lilt and some friendly resonances. At the same time, it asserts an important truth: What happens in all the unregistered markets and roadside kiosks of the world is not simply haphazard. It is a product of intelligence, resilience, self-organization, and group solidarity, and it follows a number of well-worn though unwritten rules. It is, in that sense, a system.

It used to be that System D was small — a handful of market women selling a handful of shriveled carrots to earn a handful of pennies. It was the economy of desperation. But as trade has expanded and globalized, System D has scaled up too. Today, System D is the economy of aspiration. It is where the jobs are. In 2009, the Organisation for Economic Co-operation and Development (OECD), a think tank sponsored by the governments of 30 of the most powerful capitalist countries and dedicated to promoting free-market institutions, concluded that half the workers of the world — close to 1.8 billion people — were working in System D: off the books, in jobs that were neither registered nor regulated, getting paid in cash, and, most often, avoiding income taxes.

Kids selling lemonade from the sidewalk in front of their houses are part of System D. So are many of the vendors at stoop sales, flea markets, and swap meets. So are the workers who look for employment in the parking lots of Home Depot and Lowe’s throughout the United States. And it’s not only cash-in-hand labor. As with David Obi’s deal to bring generators from China to Nigeria, System D is multinational, moving all sorts of products — machinery, mobile phones, computers, and more — around the globe and creating international industries that help billions of people find jobs and services. In many countries — particularly in the developing world — System D is growing faster than any other part of the economy, and it is an increasing force in world trade. But even in developed countries, after the financial crisis of 2008-09, System D was revealed to be an important financial coping mechanism. A 2009 study by Deutsche Bank, the huge German commercial lender, suggested that people in the European countries with the largest portions of their economies that were unlicensed and unregulated — in other words, citizens of the countries with the most robust System D — fared better in the economic meltdown of 2008 than folks living in centrally planned and tightly regulated nations. Studies of countries throughout Latin America have shown that desperate people turned to System D to survive during the most recent financial crisis. This spontaneous system, ruled by the spirit of organized improvisation, will be crucial for the development of cities in the 21st century. The 20th-century norm — the factory worker who nests at the same firm for his or her entire productive life — has become an endangered species. In China, the world’s current industrial behemoth, workers in the massive factories have low salaries and little job security. Even in Japan, where major corporations have long guaranteed lifetime employment to full-time workers, a consensus is emerging that this system is no longer sustainable in an increasingly mobile and entrepreneurial world.

So what kind of jobs will predominate? Part-time work, a variety of self-employment schemes, consulting, moonlighting, income patching. By 2020, the OECD projects, two-thirds of the workers of the world will be employed in System D. There’s no multinational, no Daddy Warbucks or Bill Gates, no government that can rival that level of job creation. Given its size, it makes no sense to talk of development, growth, sustainability, or globalization without reckoning with System D. The growth of System D presents a series of challenges to the norms of economics, business, and governance — for it has traditionally existed outside the framework of trade agreements, labor laws, copyright protections, product safety regulations, antipollution legislation, and a host of other political, social, and environmental policies. Yet there’s plenty that’s positive, too. In Africa, many cities — Lagos, Nigeria, is a good example — have been propelled into the modern era through System D, because legal businesses don’t find enough profit in bringing cutting- edge products to the third world. China has, in part, become the world’s manufacturing and trading center because it has been willing to engage System D trade. Paraguay, small, landlocked, and long dominated by larger and more prosperous neighbors, has engineered a decent balance of trade through judicious smuggling. The digital divide may be a concern, but System D is spreading technology around the world at prices even poor people can afford. Squatter communities may be growing, but the informal economy is bringing commerce and opportunity to these neighborhoods that are off the governmental grid. It distributes products more equitably and cheaply than any big company can. And, even as governments around the world are looking to privatize agencies and get out of the business of providing for people, System D is running public services — trash pickup, recycling, transportation, and even utilities.

Just how big is System D? Friedrich Schneider, chair of the economics department at Johannes Kepler University in Linz, Austria, has spent decades calculating the dollar value of what he calls the shadow economies of the world. He admits his projections are imprecise, in part because, like privately held businesses everywhere, businesspeople who engage in trade off the books don’t want to open their books (most successful System D merchants are obsessive about profit and loss and keep detailed accounts of their revenues and expenses in old-fashioned ledger books) to anyone who will write anything in a book. And there’s a definitional problem as well, because the border between the shadow and the legal economies is blurry. Does buying some of your supplies from an unlicensed dealer put you in the shadows, even if you report your profit and pay your taxes? How about hiding just $1 in income from the government, though the rest of your business is on the up-and-up? And how about selling through System D even if your business is in every other way in compliance with the law? Finding a firm dividing line is not easy, as Keith Hart, who was among the first academics toacknowledge the importance of street markets to the economies of the developing world, warned me in a recent conversation: “It’s very difficult to separate the nice African ladies selling oranges on the street and jiggling their babies on their backs from the Indian gangsters who control the fruit trade and who they have to pay rent to.” Schneider suggests, however, that, in making his estimates, he has this covered. He screens out all money made through “illegal actions that fit the characteristics of classical crimes like burglary, robbery, drug dealing, etc.” This means that the big-time criminals are likely out of his statistics, though those gangsters who control the fruit market are likely in, as long as they’re not involved in anything more nefarious than running a price-fixing cartel. Also, he says, his statistics do not count “the informal household economy.” This means that if you’re putting buckles on belts in your home for a bit of extra cash from a company owned by your cousin, you’re in, but if you’re babysitting your cousin’s kids while she’s off putting buckles on belts at her factory, you’re out.

Schneider presents his numbers as a percentage of the total market value of goods and services made in each country that same year — each nation’s gross domestic product. His data show that System D is on the rise. In the developing world, it’s been increasing every year since the 1990s, and in many countries it’s growing faster than the officially recognized gross domestic product (GDP). If you apply his percentages (Schneider’s most recent report, published in 2006, uses economic data from 2003) to the World Bank’s GDP estimates, it’s possible to make a back-of-the-envelope calculation of the approximate value of the billions of underground transactions around the world. And it comes to this: The total value of System D as a global phenomenon is close to $10 trillion. Which makes for another astonishing revelation. If System D were an independent nation, united in a single political structure — call it the United Street Sellers Republic (USSR) or, perhaps, Bazaaristan — it would be an economic superpower, the second-largest economy in the world (the United States, with a GDP of $14 trillion, is numero uno). The gap is narrowing, though, and if the United States doesn’t snap out of its current funk, the USSR/Bazaaristan could conceivably catch it sometime this century. In other words, System D looks a lot like the future of the global economy. All over the world — from San Francisco to São Paulo, from New York City to Lagos — people engaged in street selling and other forms of unlicensed trade told me that they could never have established their businesses in the legal economy. “I’m totally off the grid,” one unlicensed jewelry designer told me. “It was never an option to do it any other way. It never even crossed my mind. It was financially absolutely impossible.” The growth of System D opens the market to those who have traditionally been shut out.

This alternative economic system also offers the opportunity for large numbers of people to find work. No job-cutting or outsourcing is going on here. Rather, a street market boasts dozens of entrepreneurs selling similar products and scores of laborers doing essentially the same work. An economist would likely deride all this duplicated work as inefficient. But the level of competition on the street keeps huge numbers of people employed. It liberates their entrepreneurial energy. And it offers them the opportunity to move up in the world. In São Paulo, Édison Ramos Dattora, a migrant from the rural midlands, has succeeded in the nation’s commercial capital by working as a camelô — an unlicensed street vendor. He started out selling candies and chocolates on the trains, and is now in a more lucrative branch of the street trade — retailing pirate DVDs of first-run movies to commuters around downtown. His underground trade — he has to watch out for the cops wherever he goes — has given his family a standard of living he never dreamed possible: a bank account, a credit card, an apartment in the center of town, and enough money to take a trip to Europe. Even in the most difficult and degraded situations, System D merchants are seeking to better their lives. For instance, the garbage dump would be the last place you would expect to be a locus of hope and entrepreneurship. But Lagos scavenger Andrew Saboru has pulled himself out of the trash heap and established himself as a dealer in recycled materials. On his own, with no help from the government or any NGOs or any bank (Andrew has a bank account, but his bank will never loan him money — because his enterprise is unlicensed and unregistered and depends on the unpredictable labor of culling recyclable material from the megacity’s massive garbage pile), he has climbed the career ladder. “Lagos is a city for hustling,” he told me. “If you have an idea and you are serious and willing to work, you can make money here. I believe the future is bright.” It took Andrew 16 years to make his move, but he succeeded, and he’s proud of the business he has created. We should be too. As Joanne Saltzberg, who heads Women Entrepreneurs of Baltimore — a business development group — told me, we need to change our attitude and to salute the achievements of those who are engaged in this alternate economy. “We only revere success,” she said. “I don’t think we honor the struggle. People who have no access to business development resources. People who have to work two and three jobs just to survive. When you are struggling in this economy and still you commit yourself to having a better life, that’s really something to honor.”

RADIO ZETA
http://www.washingtonpost.com/world/americas/equipment-seizures-show-mexican-drug-cartels-operate-sophisticated-communications-networks/2011/12/26/gIQAjJu6IP_print.html
http://www.wired.com/dangerroom/2011/12/cartel-radio-mexico/
How Mexico’s Drug Cartels Stay Networked
by Spencer Ackerman / December 27, 2011

Arranging drug sales on a cellphone, cryptic email or even a pager? That’s strictly for the small-time dealer. If you’re a Mexican drug cartel, you have your own radio network. Since 2006, the cartels have maintained an encrypted DIY radio network that stretches across nearly all 31 Mexican states, even down south into Guatemala. The communications infrastructure of the narco-gangs that have turned Mexico into a gangster’s paradise consists of “professional-grade” radio antennas, signal relays and simple handheld radios that cost “millions of dollars” — and which the Mexican authorities haven’t been able to shut down. If it sounds like a military-grade communications apparatus, it should. The notorious Zetas, formerly the enforcers for the Gulf Cartel and now its chief rival, were born out of Mexican Special Forces. But the Zetas aren’t stupid enough to make big deals over a radio frequency, even an encrypted one. According to a picture of what you might call Radio Zeta that’s emerged after three raids by the Mexican authorities, the bosses only communicate through the Internet. The radio network is for lookouts and lower-level players.

Here’s how it works, according to a fascinating Associated Press piece. The cartels divide up territory into “plazas.” The plaza boss has the responsibility for establishing nodes on the network — getting the antennas in place, concealing them as necessary, making sure the signal-boosting repeaters extend the network’s reach, equipping cartel personnel with handheld radios, and replacing what the security forces destroy. The cartels have even gone green, with solar panels powering the radio towers. The network is primarily an early warning reconnaissance system. “Halcons,” or “hawks,” holler on the handhelds when the federal police or soldiers roll through cartel territory. But it’s also an occasional offensive tool to intimidate the security forces. The cartels have been known to hijack military radio networks to broadcast threats. That’s keeping in line with the Zetas’ alarming tactic of slaughtering people for allegedly talking openly about cartel activity over the Internet.

Since September, three large raids conducted by Mexico’s beleaguered security forces have attempted to disrupt the radio network by snatching up its hardware. But much of the infrastructure — the towers, the receivers — is cheap enough to be easily replaced. The network is “low-cost, highly extendable and maintainable,” a security consultant told the AP. But there’s an alternative for taking down the cartel broadcasts. Since the U.S. already provides intelligence and security assistance to Mexico’s drug war, maybe it’s time to think about providing somemilitary-grade jammers as well. Mexico doesn’t seem to have a better idea for taking Radio Zeta off the air.

{Freenet means controversial information does not need to be stored in physical data havens such as this one, Sealand. Photograph: Kim Gilmour/Alamy}

FREENET
http://www.guardian.co.uk/technology/2009/nov/26/dark-side-internet-freenet
The dark side of the internet
by Andy Beckett / 25 November 2009

Fourteen years ago, a pasty Irish teenager with a flair for inventions arrived at Edinburgh University to study artificial intelligence and computer science. For his thesis project, Ian Clarke created “a Distributed, Decentralised Information Storage and Retrieval System”, or, as a less precise person might put it, a revolutionary new way for people to use theinternet without detection. By downloading Clarke’s software, which he intended to distribute for free, anyone could chat online, or read or set up a website, or share files, with almost complete anonymity. “It seemed so obvious that that was what the net was supposed to be about – freedom to communicate,” Clarke says now. “But [back then] in the late 90s that simply wasn’t the case. The internet could be monitored more quickly, more comprehensively, more cheaply than more old-fashioned communications systems like the mail.” His pioneering software was intended to change that. His tutors were not bowled over. “I would say the response was a bit lukewarm. They gave me a B. They thought the project was a bit wacky … they said, ‘You didn’t cite enough prior work.'” Undaunted, in 2000 Clarke publicly released his software, now more appealingly called Freenet. Nine years on, he has lost count of how many people are using it: “At least 2m copies have been downloaded from the website, primarily in Europe and the US. The website is blocked in [authoritarian] countries like China so there, people tend to get Freenet from friends.” Last year Clarke produced an improved version: it hides not only the identities of Freenet users but also, in any online environment, the fact that someone is using Freenet at all.

Installing the software takes barely a couple of minutes and requires minimal computer skills. You find the Freenet website, read a few terse instructions, and answer a few questions (“How much security do you need?” … “NORMAL: I live in a relatively free country” or “MAXIMUM: I intend to access information that could get me arrested, imprisoned, or worse”). Then you enter a previously hidden online world. In utilitarian type and bald capsule descriptions, an official Freenet index lists the hundreds of “freesites” available: “Iran News”, “Horny Kate”, “The Terrorist’s Handbook: A practical guide to explosives and other things of interests to terrorists”, “How To Spot A Pedophile [sic]”, “Freenet Warez Portal: The source for pirate copies of books, games, movies, music, software, TV series and more”, “Arson Around With Auntie: A how-to guide on arson attacks for animal rights activists”. There is material written in Russian, Spanish, Dutch, Polish and Italian. There is English-language material from America and Thailand, from Argentina and Japan. There are disconcerting blogs (“Welcome to my first Freenet site. I’m not here because of kiddie porn … [but] I might post some images of naked women”) and legally dubious political revelations. There is all the teeming life of the everyday internet, but rendered a little stranger and more intense. One of the Freenet bloggers sums up the difference: “If you’re reading this now, then you’re on the darkweb.” The modern internet is often thought of as a miracle of openness – its global reach, its outflanking of censors, its seemingly all-seeing search engines. “Many many users think that when they search on Google they’re getting all the web pages,” says Anand Rajaraman, co-founder of Kosmix, one of a new generation of post-Google search engine companies. But Rajaraman knows different. “I think it’s a very small fraction of the deep web which search engines are bringing to the surface. I don’t know, to be honest, what fraction. No one has a really good estimate of how big the deep web is. Five hundred times as big as the surface web is the only estimate I know.”

Unfathomable and mysterious
“The darkweb”; “the deep web”; beneath “the surface web” – the metaphors alone make the internet feel suddenly more unfathomable and mysterious. Other terms circulate among those in the know: “darknet”, “invisible web”, “dark address space”, “murky address space”, “dirty address space”. Not all these phrases mean the same thing. While a “darknet” is an online network such as Freenet that is concealed from non-users, with all the potential for transgressive behaviour that implies, much of “the deep web”, spooky as it sounds, consists of unremarkable consumer and research data that is beyond the reach of search engines. “Dark address space” often refers to internet addresses that, for purely technical reasons, have simply stopped working. And yet, in a sense, they are all part of the same picture: beyond the confines of most people’s online lives, there is a vast other internet out there, used by millions but largely ignored by the media and properly understood by only a few computer scientists. How was it created? What exactly happens in it? And does it represent the future of life online or the past? Michael K Bergman, an American academic and entrepreneur, is one of the foremost authorities on this other internet. In the late 90s he undertook research to try to gauge its scale. “I remember saying to my staff, ‘It’s probably two or three times bigger than the regular web,”‘ he remembers. “But the vastness of the deep web . . . completely took my breath away. We kept turning over rocks and discovering things.” In 2001 he published a paper on the deep web that is still regularly cited today. “The deep web is currently 400 to 550 times larger than the commonly defined world wide web,” he wrote. “The deep web is the fastest growing category of new information on the internet … The value of deep web content is immeasurable … internet searches are searching only 0.03% … of the [total web] pages available.” In the eight years since, use of the internet has been utterly transformed in many ways, but improvements in search technology by Google, Kosmix and others have only begun to plumb the deep web. “A hidden web [search] engine that’s going to have everything – that’s not quite practical,” says Professor Juliana Freire of the University of Utah, who is leading a deep web search project called Deep Peep. “It’s not actually feasible to index the whole deep web. There’s just too much data.”

But sheer scale is not the only problem. “When we’ve crawled [searched] several sites, we’ve gotten blocked,” says Freire. “You can actually come up with ways that make it impossible for anyone [searching] to grab all your data.” Sometimes the motivation is commercial – “people have spent a lot of time and money building, say, a database of used cars for sale, and don’t want you to be able to copy their site”; and sometimes privacy is sought for other reasons. “There’s a well-known crime syndicate called the Russian Business Network (RBN),” says Craig Labovitz, chief scientist at Arbor Networks, a leading online security firm, “and they’re always jumping around the internet, grabbing bits of [disused] address space, sending out millions of spam emails from there, and then quickly disconnecting.” The RBN also rents temporary websites to other criminals for online identity theft, child pornography and releasing computer viruses. The internet has been infamous for such activities for decades; what has been less understood until recently was how the increasingly complex geography of the internet has aided them. “In 2000 dark and murky address space was a bit of a novelty,” says Labovitz. “This is now an entrenched part of the daily life of the internet.” Defunct online companies; technical errors and failures; disputes between internet service providers; abandoned addresses once used by the US military in the earliest days of the internet – all these have left the online landscape scattered with derelict or forgotten properties, perfect for illicit exploitation, sometimes for only a few seconds before they are returned to disuse. How easy is it to take over a dark address? “I don’t think my mother could do it,” says Labovitz. “But it just takes a PC and a connection. The internet has been largely built on trust.”

Open or closed?
In fact, the internet has always been driven as much by a desire for secrecy as a desire for transparency. The network was the joint creation of the US defence department and the American counterculture – the WELL, one of the first and most influential online communities, was a spinoff from hippy bible the Whole Earth Catalog – and both groups had reasons to build hidden or semi-hidden online environments as well as open ones. “Strong encryption [code language=”-writing”][/code] developed in parallel with the internet,” says Danny O’Brien, an activist with the Electronic Frontier Foundation, a long-established pressure group for online privacy. There are still secretive parts of the internet where this unlikely alliance between hairy libertarians and the cloak-and-dagger military endures. The Onion Router, or Tor, is an American volunteer-run project that offers free software to those seeking anonymous online communication, like a more respectable version of Freenet. Tor’s users, according to its website, include US secret service “field agents” and “law enforcement officers . . . Tor allows officials to surf questionable websites and services without leaving tell-tale tracks,” but also “activists and whistleblowers”, for example “environmental groups [who] are increasingly falling under surveillance in the US under laws meant to protect against terrorism”. Tor, in short, is used both by the American state and by some of its fiercest opponents. On the hidden internet, political life can be as labyrinthine as in a novel by Thomas Pynchon.

The hollow legs of Sealand
The often furtive, anarchic quality of life online struck some observers decades ago. In 1975, only half a dozen years after the internet was created, the science-fiction author John Brunner wrote of “so many worms and counter-worms loose in the data-net” in his influential novel The Shockwave Rider. By the 80s “data havens”, at first physical then online locations where sensitive computerised information could be concealed, were established in discreet jurisdictions such as Caribbean tax havens. In 2000 an American internet startup called HavenCo set up a much more provocative data haven, in a former second world war sea fort just outside British territorial waters off the Suffolk coast, which since the 60s had housed an eccentric independent “principality” called Sealand. HavenCo announced that it would store any data unless it concerned terrorism or child pornography, on servers built into the hollow legs of Sealand as they extended beneath the waves. A better metaphor for the hidden depths of the internet was hard to imagine. In 2007 the highly successful Swedish filesharing website The Pirate Bay – the downloading of music and films for free being another booming darknet enterprise – announced its intention to buy Sealand. The plan has come to nothing so far, and last year it was reported that HavenCo had ceased operation, but in truth the need for physical data havens is probably diminishing. Services such as Tor and Freenet perform the same function electronically; and in a sense, even the “open” internet, as online privacy-seekers sometimes slightly contemptuously refer to it, has increasingly become a place for concealment: people posting and blogging under pseudonyms, people walling off their online lives from prying eyes on social networking websites. “The more people do everything online, the more there’s going to be bits of your life that you don’t want to be part of your public online persona,” says O’Brien. A spokesman for the Police Central e-crime Unit [PCeU] at the Metropolitan Police points out that many internet secrets hide in plain sight: “A lot of internet criminal activity is on online forums that are not hidden, you just have to know where to find them. Like paedophile websites: people who use them might go to an innocent-looking website with a picture of flowers, click on the 18th flower, arrive on another innocent-looking website, click something there, and so on.” The paedophile ring convicted this autumn and currently awaiting sentence for offences involving Little Ted’s nursery in Plymouth met on Facebook. Such secret criminal networks are not purely a product of the digital age: codes and slang and pathways known only to initiates were granting access to illicit worlds long before the internet. To libertarians such as O’Brien and Clarke the hidden internet, however you define it, is constantly under threat from restrictive governments and corporations. Its freedoms, they say, must be defended absolutely. “Child pornography does exist on Freenet,” says Clarke. “But it exists all over the web, in the post . . . At Freenet we could establish a virus to destroy any child pornography on Freenet – we could implement that technically. But then whoever has the key [to that filtering software] becomes a target. Suddenly we’d start getting served copyright notices; anything suspect on Freenet, we’d get pressure to shut it down. To modify Freenet would be the end of Freenet.”

Always recorded
According to the police, for criminal users of services such as Freenet, the end is coming anyway. The PCeU spokesman says, “The anonymity things, there are ways to get round them, and we do get round them. When you use the internet, something’s always recorded somewhere. It’s a question of identifying who is holding that information.” Don’t the police find their investigations obstructed by the libertarian culture of so much life online? “No, people tend to be co-operative.” The internet, for all its anarchy, is becoming steadily more commercialised; as internet service providers, for example, become larger and more profit-driven, the spokesman suggests, it is increasingly in their interests to accept a degree of policing. “There has been an increasing centralisation,” Ian Clarke acknowledges regretfully. Meanwhile the search engine companies are restlessly looking for paths into the deep web and the other sections of the internet currently denied to them. “There’s a deep implication for privacy,” says Anand Rajaraman of Kosmix. “Tonnes and tonnes of stuff out there on the deep web has what I call security through obscurity. But security through obscurity is actually a false security. You [the average internet user] can’t find something, but the bad guys can find it if they try hard enough.” As Kosmix and other search engines improve, he says, they will make the internet truly transparent: “You will be on the same level playing field as the bad guys.” The internet as a sort of electronic panopticon, everything on it unforgivingly visible and retrievable – suddenly its current murky depths seem in some ways preferable. Ten years ago Tim Berners-Lee, the British computer scientist credited with inventing the web, wrote: “I have a dream for the web in which computers become capable of analysing all the data on the web – the content, links, and transactions between people … A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.” Yet this “semantic web” remains the stuff of knotty computer science papers rather than a reality. “It’s really been the holy grail for 30 years,” says Bergman. One obstacle, he continues, is that the internet continues to expand in unpredictable and messy surges. “The boundaries of what the web is have become much more blurred. Is Twitter part of the web or part of something else? Now the web, in a sense, is just everything. In 1998, the NEC laboratory at Princeton published a paper on the size of the internet. Who could get something like that published now? You can’t talk about how big the internet is. Because what is the metric?”

Gold Rush
It seems likely that the internet will remain in its Gold Rush phase for some time yet. And in the crevices and corners of its slightly thrown-together structures, darknets and other private online environments will continue to flourish. They can be inspiring places to spend time in, full of dissidents and eccentrics and the internet’s original freewheeling spirit. But a darknet is not always somewhere for the squeamish. On Freenet, there is a currently a “freesite” which makes allegations against supposed paedophiles, complete with names, photographs, extensive details of their lives online, and partial home addresses. In much smaller type underneath runs the disclaimer: “The material contained in this freesite is hearsay . . . It is not admissable in court proceedings and would certainly not reach the burden of proof requirement of a criminal trial.” For the time being, when I’m wandering around online, I may stick to Google.