Citizen-archivist uses tech savvy to liberate government information

If you browse the press clips that tech-activist Carl Malamud has accumulated over the past fifteen years, the word “gadfly” crops up a lot. Reporters presumably mean to suggest that he is a small, ineffectual irritant.

Not true. Malamud may be a one-person operation, Public.Resource.org, a small nonprofit based in Sebastopol, California. But his guerrilla activism – pursued in collaboration with trusted board members and ad hoc partners – has been singularly effective in prodding government bodies to open up their vast digital warehouses of information to the public. Malamud’s self-declared mission is to assure that taxpayer-financed information and culture are placed in the commons, where it belongs.

A government official who receives one of Malamud’s letters may be tempted to wonder, “Who is this guy?” and ignore him. Big mistake.

That’s essentially what the Smithsonian Institution did in 2007 when Malamud pointed out that the nation’s premier cultural institution was making broad copyright claims for photographs that actually are in the public domain. Malamud also accused the Smithsonian of discouraging public use of the photos by requiring a complicated navigation to find images that could only be downloaded in low-resolution, watermarked formats.

After verifying that 6,288 images were indeed in the public domain, Malamud took unilateral action to “harvest” them and upload them to Flickr, the photo-sharing website. That got the Smithsonian’s attention. Suddenly the Smithsonian’s questionable stewardship of public-domain photos was a hot public issue. The pressure grew for the Smithsonian to join the Library of Congress’ initiative to put public-domain images on the Flickr Commons.

Malamud used a similar tactic to prod the National Technical Information Service, or NTIS, to put government-produced films online. The NTIS is the distributor of a vast inventory of U.S. Government films and videos that cover a wide variety of practical and historical topics. There is the Army film on “managing chemical and biological casualties”; a five-part series on “victims’ rights” produced by the Justice Department; and historical gems such as the scary-absurd Duck and Cover civil defense film from the 1950s.

Because NTIS was not digitizing its films or making them available online, Malamud invited ordinary citizens who had bought an NTIS film to donate it to Public.Resource.org. Then Malamud would digitize it and post it for free on the Internet.

The spectacle of citizens in effect paying a ransom in order to liberate films that belong to them in the first place, was a bit embarrassing for NTIS. So was the fact that Malamud’s website of NTIS films was soon out-ranking the NTIS website on Google search engine rankings. The agency, in its defense, insisted that it must recover its costs, and that digitizing thousands of films and videotapes would end up costing millions of dollars that it didn’t have.

In the end, Malamud concocted FedFlix, a project jointly run with the NTIS to distribute online versions of the U.S. Government’s best films. Public.resource.org digitizes 10-20 films a month, and posts them at the Internet Archive (run by another renegade archivist, Brewster Kahle) and YouTube. The FedFlix site boasts, “All of these fine flix are available for re-use without any restrictions whatsoever.” About 400 government films and videos are now online. “Watch FedFlix,” says a button on Public.Resource.org. “No late charges!”

Many of the information-liberation projects that Malamud undertakes are technically straight-forward and not that costly. And most if not all have no partisan bias; they are about democracy and good-government. So what’s the big deal?

Malamud admitted, “I don’t know why public access and transparency are controversial.” He points out, however, that many government agencies are technically clueless about how to make their work and deliberations more open. And many politicians and agency administrators are wary about putting information online lest it invite closer scrutiny of their work.

Which, to Malamud, is the point. The American people are financing all this information, and any functioning democracy or efficient marketplace needs the free flow of reliable information.

Although he is a serious and sophisticated policy player, and is capable of playing the political heavy, Malamud is not averse to collaborating with agency officials when it serves to bring about change. “Sometimes that takes a hammer,” he said, “ and sometimes you just have to show them how to do it.”

A “Rogue Archivist” is Born

Malamud has been working at the intersection of tech innovation and government information policy since the early 1990s, when he founded the Internet Multicasting Service and the first Internet radio station. He has a deep technical knowledge of computers and the Internet – he has written eight books on the subject – and so can grasp possibilities that elude many information-technology bureaucrats within federal agencies.

Malamud is effective, also, because he tends to be indifferent to the institutional or ideological concerns that often inhibit “respectable” advocacy groups. For example, conventional public-interest organizations might be shy about confronting friendly Senators or Representatives lest it complicate their long-term political agenda.

Not Malamud. He’s willing to swoop into a situation on his own terms, as a kind of caped crusader for information freedom. His free-wheeling, puckish style is conveyed by Public.Resource.org’s daffy logo – a smiling seal encircled by the words “The Great Seal of the Seal of Approval.”

Any Dada artist would be proud to see the Great Seal next to the official seal of a government agency on the same letterhead stationery or web page. My favorite example is a web page that pairs the Great Seal with the official seal of the State of Oregon on a webpage challenging Oregon’s claims to own copyrights in its revised statutes. Or take a look at the Public.Resource.org homepage:. If you move your cursor over the Great Seal, he suddenly tilts at a 45-degree angle, as if giving you a trickster’s wink.

p(photo-image). !/sites/default/files/imagecache/large/foil.png!

It is this penchant for acting as a trickster that has made Malamud such an effective change-agent. He works with the rakish panache of a Robin Hood accented by a dash of the absurd.

In 1994, working in loose collaboration with Nader tech activist Jamie Love – another pioneer in the field – Malamud pulled off an activist coup of path-breaking significance. He engineered a scheme to put the Securities and Exchange Commission’s database of corporate financial information online.

At the time, the newly born World Wide Web was making it easy to put government information within reach of everyone. Yet the SEC, following the historical custom for government agencies, wanted to give the EDGAR database to a private vendor, who would then re-sell access to the data for large sums of money.

Wall Street brokerages would still be able to acquire the data, at a price, but ordinary investors and citizens would be excluded from basic corporate financial disclosures. In addition, the stock market – which, like any marketplace, requires plentiful reliable information in order to function efficiently – would suffer from a serious lack of reliable information. Yet the SEC insisted that it would cost too much to put the data online.

Undeterred, Malamud finagled a donated computer and software, acquired the SEC data, and put the EDGAR database online himself. Jamie Love hammered away with rulmaking petitions to the SEC. “Two years later, with a little bureaucratic maneuvering and some loud bombast, the SEC was happily running the service themselves!” says Malamud.

The episode was a critical and timely precedent. It visibly demonstrated that government agencies could indeed afford to put their documents and data on the Internet. There was no need to give away government information to private vendors. As a result of the SEC case, other government agencies began to take charge of their own data and publish it on their own websites.

For the rest of the 1990s and early 2000s, Malamud had a variety of tech and activist affiliations. He was a visiting professor at MIT Media Lab and the organizer of an event he called the Internet World Exposition in 1996. He has been a board member of the Mozilla Foundation (which sponsors the open-source Firefox browser and Thunderbird email client), and was a fellow at the Center for American Progress in Washington, D.C.

In 2007 he decided to consolidate his information-advocacy into a new nonprofit, Public.Resource.org, which he founded with Marshall Rose, a computer protocol expert. He also moved from Washington, D.C., to Sebastopol, California, an hour’s drive north of San Francisco.

p(photo-image). !/sites/default/files/imagecache/large/opengovtgangsign.jpg!
“OG” (Open Government) gang sign. Photo by Joe Hall, joebeone, via Flickr, licensed under a CC BY license. http://www.flickr.com/photos/joebeone/2098021665.

Kick-starting the “Free Law” Movement

Since getting his advocacy re-organized under the auspices of Public.Resource.org, Malamud has revisited a battle that was started in the 1990s but has never been adequately resolved: liberating court decisions and other legal information from large corporate publishers.

For decades, West Publishing and later, LexisNexis, have had a stranglehold on the publication of court decisions and other legal information. The companies had the most advanced technical systems, which prevented newcomers from competing, and they had secured the goodwill of the legal establishment by giving preferential service and junkets to judges, members of Congress and even Supreme Court justices. To defend its quasi-monopoly, West even claimed a copyright in the page numbers of court rulings, which are the standard basis for legal citations in lawsuits.

Even though a court in 1998 overruled West’s claim to own the page numbers of court decisions, West has continued to dominate the $5 billion legal publishing market. The monopoly over access to the law is particularly offensive now that technology can make legal information easily available. Malamud has said that the law resembles the “operating system” of the nation. “The system only works if we can all openly read the primary sources. It is crucial that the public domain data be available for anybody to build upon,” he said.

In October 2007, Malamud decided it was time to convene the key players in the emerging “free law” movement and attempt a new campaign to make various bodies of legal information available. He is currently placing unpaginated appellate court cases from the 1880s and later decades on the Public.Resource.org website. But he also has ambitions to liberate other bodies of legal information – congressional hearings, the U.S. Code, the Federal Register, bankruptcy cases, and much else.

“Eventually,” said Malamud, “the judicial branch needs to get itself together.” Until it does, Malamud is exploring ways to overcome the resistance of the stodgy legal culture. He is trying to get the University of Pennsylvania to release the only surviving copy of a Justice Department database known as JURIS, which contains over two million pages of federal case law. Malamud also suggests that law schools could start their own “reporter societies” to make decisions made by specific courts available online.

Meanwhile, Malamud is pursuing another big information-liberation project, bulk.resource.org, under the auspices of Public.Resource.org. Bulk.resource.org is a website on which Malamud dumps large collections of raw data from government agencies – patent data, trademark data, copyright data, and more. Rather than try to build user-friendly interfaces or tailor the data for specialized purposes, Malamud said, “Our job is liberating the data in bulk” – leaving to others, both for-profit businesses and nonprofit organizations, the task of developing innovative ways to repackage the “wholesale data” for specific purposes.

In a sense, it is amazing that a tech crusader like Carl Malamud is needed in these times. The technology is available and relatively inexpensive. The information has already been paid for by the American taxpayer. So what’s so controversial? Can’t the government do this stuff?

Malamud’s work points to the hard realities of bureaucratic inertia and the aversion among the powerful to transparency and accountability. It also points to the vital importance of a resourceful, persistent citizen-crusader who can prod government to do the right thing.