Testing this plugin seems like a good excuse to promote a few of my favorite tweeters, just in case you’re looking for friends. I’ve grouped them roughly by category.

Me

@sarahdavies

Infomaniacs (those insanely obsessed with information and how it effects humans)

@emahlee

@WebEcology

@laurenpressley

@kathy_live and @kegill

@sarterus

Nonprofits

@ACLU and @ACLU_WA

@NTENorg

@vittana

@WashingtonBus

@seafreeschool

@equalrightswa

@civicactions

The Strange yet Awesome

@unicornbooty

@awesomefound

Note: If you’re reading this through rss or Facebook, this won’t work for you. You have to be on the site. Since Facebook is amazingly good at hiding what site this came from, here’s a link for all you Facebookers. http://sarahdavies.cc/2010/04/23/we-have-been-assimilated-by-the-twitter

Second note: Damn, Twitter, way to rip off Obama’s graphic design. You guys had a great cute opaque curvy thing going. You don’t need to steal other people’s style, we like you just the way you are!

Third note: Since my twitter is in the sidebar of my blog, will my @replies on twitter come back to my blog and get hovercards linking them back to twitter in an endless recursive nightmare of microbloggy madness?

	Serkan Piantino Facebook Inc
	Alan Schaaf Imgur LLC
	Kevin Weil Twitter
	Christopher Slowe Reddit
	Jason Kincaid TechCrunch

Imgur was released a year ago on Reddit. It was on a shared hosting plan. It lasted two days before the site was terminated for generating too much traffic. The site went down. Imgur moved to Mediatemple. That lasted three weeks, so they moved again, and again. Imgur moved four or five times in four months, scaling up to a better server with more bandwidth. Imgur went to foxhole.net, a content delivery network, because they have servers all over the world. That allowed the devs to concentrate on making the site faster rather than keeping the servers up.

Reddit is running on EC2 using about 50 machines. They have 20 app servers. They got a big speed boost by going single-threaded. They use Postgress and memcache.

Twitter started as a rails application tied to a single MySQL database. They have an open source queuing system, so they can do asynchronous processing.

A lot of the core architecture behind Facebook is still LAMP. They run newsfeed, ads, and search all on separate servers. On Facebook, you’re typically friends with 0-5000 people, whereas on Twitter you can follow millions of people, so they can render everything on the fly where Twitter can’t.

What is Reddit using for indexing?
They roll their own indeces using memcachedb. They are getting a .02% failure rate with that.

At what point are LAMP stacks not enough?
Knowing when a machine needs to be replaced is tough. Facebook has a monitoring system set up with alerts and teams dedicated to figuring out where they will have scaling problems before they have them. Monitor. Monitor. Monitor.

How do you scale search?
Search is really hard. The metric you are measured against is Google, which is a ridiculous standard. Reddit does about two queries per second. Getting quality results is really hard to tweak. It’s very qualitative in terms of what is “good” search.

What was the first thing that blew up?
Imgur had apache blow up first. “It was like trying to hammer a nail with a sledgehammer.” Twitter originally put the whole social graph in a MySQL database, but it was getting into the billions of rows. They had to build their own social graph store. They are in the process of open sourcing it.

What modules is Facebook using to convert PHP to C++?
They built a project called hiphop which compiles all their php down to binary C++. There are whitepapers about it, and it’s open source.

How do you deal with deployment?
Facebook and Twitter use BitTorrent to deploy builds to all their servers, cutting deployment from 12 minutes down to 30 seconds. Reddit cobbled something together in perl.

Why haven’t any of you used proprietary databases?
We prefer to work with open source. As you deal with scaling problems, you have to peak under the hood and see what you can tweak. Calling a vendor is a pain. Oracle is expensive. We like to be nimble and play well with the community.

Sarah: What does it take to motivate a community that spends 12 hours a day in front of a screen to meet in-person on a regular basis? How do you build that sort of community?
Mary Jane: I think this question hits at the heart of a lot of important issues. Technology is a wonderful tool for facilitating social interaction, which we all need. Like any tool, though, it can be misused. Multiple studies show how vital in-person communication is for maintaining the close relationships that are necessary for health and happiness. While virtual communication can definitely enhance relationships, it can’t ever totally replace the experience of being with other people in person. With so many demands on our time, though, it’s very tempting to try to replace face-to-face meetings with quick IMs or status updates. We need face time, though, and there’s really no replacement for it.

I think the key to building a successful networking community is to provide that in-person interaction in a way that is sensitive to busy schedules. Flexible, casual meet-ups work well, especially if there’s an incentive to attend, like an interesting topic, a cool venue, or, of course, free food! Timing is just as key, since it’s easier to cancel and go home than to rush through traffic to get to a meeting right after work.

Sarah: Do women have a unique role to play in the digital world, or should we have the same expectations for women that we do for men?
Mary Jane: Women absolutely have a vital and unique role to play in the tech industry. In addition to the hard tech skills required for our projects, women can also be excellent at fostering team cohesion and propagating a shared vision, and I think that most women do this very naturally. So often on a tech team, because we get engrossed in the details of our particular tasks, we forget that solutions are still created by people. That oversight can put a project at risk because even the best idea can fail without the right team to make it happen. I believe that women have a natural aptitude for bringing teams together above and beyond the explicit shared work items, and until we have machines to design, make, and repair our technology for us, the human factor will continue to be vital to the future of technological innovation.

Sarah: What perks can organizations provide to motivate young people, particularly women, to work there? Do you think most young people would take a pay cut for some of those perks?
Mary Jane: Flexibility and work-life balance are very important to young people, especially those who have family and volunteer commitments. Creative work arrangements appeal to bright, involved employees who have a lot going on outside of work, and there are some great models of how value increases when employees have more freedom and input about their work environment. For most tech jobs, flextime and working from home are easy to arrange with the right tech solution. It’s different for each organization, of course, but I think that in a lot of cases, especially for highly skilled, self-motivated employee bases, the added performance, decreased overturn, and increased project morale gained by keeping employees happy would probably more than offset the overhead. Implemented correctly, there’s no need for pay cuts, since the company would be getting a return on the investment.

Sarah: What do you think the next revolution will be for online dating?
Mary Jane: Online dating is a great way to meet potential friends and dates, when it’s used the right way. It’s most effective as an introduction tool, when communication moves from virtual to real life as early as possible. People are wired to respond to in-person communication, especially when it comes to dating, and the risk of building up unrealistic expectations increases the longer the communication stays strictly virtual. Of course, people want to have an idea of what they’re getting into first and there are real safety concerns, so some communication is important before the first meeting.

We’ve seen a lot of improvements in online dating since it first started out. I think that a service-oriented matchmaking site would be an interesting development. Dating services can offer more than simply providing a forum for user-generated content, some personality tests, and a chat client. I’d be interested to see some branching out into profile editing/advice, date scheduling, better screening, and maybe personalized relationship coaching.

Sarah: Is there a good way to help upper management folks understand digital culture, or do they just have to trust the people who are immersed in the internet everyday to provide the answers?
Mary Jane: I think the best way for management to better understand digital culture is to get more involved. It’s so simple to generate content that there is practically no barrier to entry. Setting up a blog or Twitter account that employees could read would be a great way to improve personal tech skills, get informal feedback on decisions, disseminate non-sensitive information, and improve team/company cohesion. Personally, with the low resource cost and high potential gains, I don’t know why more executives don’t participate in some form of active social networking.

The purpose of these interviews (in addition to just being fascinating) is to promote my panel proposals at this year’s sxsw, but the panel picker is now closed, so this one’s just a bonus!

Sarah: The internet has had a revolutionary effect on societies worldwide, yet academia seems to have utterly failed at documenting and studying it. Why is this such a difficult field to study and how can we get solid research on the ecology of the web?
Tim: The classic response to this is one you really hear a lot: stodgy old ivory-tower fogies don’t think the internet (and internet culture in particular) is important enough to study. While I think there is some truth in that stereotype, I have to say that I think there’s more to the story than just that — after all, there’s plenty of progressive, “with-it” folks in academia that see the value in exploring this space. In spite of this, the fact that universities remain slow to pick up on digital stuff I think suggests a deeper problem in the organizational element of the whole picture. The hierarchies of authority, the standards around publication, and the flexibility of creating and halting projects, all conspire to make it difficult for academia to keep up with the changing ecosystem of the web. Moreover, academic institutions are locked in a system of grants that often tie their hand with regards to what they can spend money on and invest in, which makes them inflexible and slow. We’ve been trying to experiment with new organizing models with The Web Ecology Project, and have been really excited about how things have been going.

Sarah: #iranelection seems to be the first meme that went globally mainstream. It worries me that the meme was almost entirely dependent on Twitter. They could have been DDoS’d or hacked, or paid to shutdown for a week. Are memes vulnerable to their platforms, or would the meme have carried on elsewhere if the platform went down?
Tim: Luckily, memes often aren’t completely platform dependent, so that the shutting down of any particular online space where cultural phenomena is happening won’t necessarily kill it completely. That being said, it’s true that certain platforms make particular activities /easier/ and that the amount of influence or attention that a space commands (or a given user commands in that space) is significant in powering the spread of a practice or an idea online. The “leakage” of memes depends to some extent on the the ease of users to adopt new platforms or their existing membership across platforms. So, there’s a bunch of variables — all told, it looks like from our research that it depends alot on the particulars of a situation. For the Iran Election, I think odds are it would have appeared elsewhere (though potentially in less visible spaces), given the media attention and the activist activity surrounding the event.

Sarah: Where do you see the future of nonprofits moving? It seems like we have to be increasingly agile to move at the speed of the web, to the point that restraints such as narrow mission statements or hierarchical management structures simply can’t compete. You’re involved in the Awesome Foundation, which has just about the broadest mission statement I’ve seen and zero management structure. Is that the future? Can it get even more agile than that?
Tim: The Awesome Foundation tries to keep it real. How many times have people applied for grants, only to try to cloak their real intentions of “hey wow wouldn’t it be great if…” from the granting organization? We’ve tried to eliminate that, make it easy for people to be honest about what they want support to do. There’s an advantage in that, particularly as we’ve tried to pursue lightweight structures that make it dead simple to apply and get money (we actually give the money directly, in cash). Think there’s two possibilities going into the future. One is to be exceedingly lightweight and broad, essentially what we’ve done with the Awesome Foundation. The other is to go entirely the other way — to craft incredibly narrow, incredibly curated groups. However, both of these disperse anti-foundation foundation models have only experimented with relatively small groups and small stakes so far. I think a big question going forwards is — can this scale? How much larger can these models get in terms of people and dollar amounts before they break down? Or is the future just an enormous, disperse framework of highly nimble granting groups? One thing seems clear: old non-profits seem increasingly slow to jump on supporting emerging efforts at the earliest stage.

Sarah: How do we solve copyright?
Tim: Sure, there’s GPL, Creative Commons, BSD, and a whole host of thought, projects (and arguments of the most vicious kind) that have gone into trying to figure out how to repair the structure of intellectual property more generally. Beyond quibbling about the details or whether so-and-so solution is better than that-or-this proposal, I think what all of them have in common is so key is to view the law as a space to be innovated on and experimented with. There’s an inherent risk-averseness to the law and lawyers, and a norm that limits the extent to which people feel they can craft new entities within the law. Though really, there’s no reason for that — even such established entities as “the Corporation” were the creations of legal innovation (really, legal hacking) at some point. This is what’s kept copyright behind as the entire environment has shifted around it — I think what’s necessary is for a shift in thinking about the law as open-ended to possibility and active manipulation in the same way Creative Commons constructed a new interface with the law, rather than something that’s a closed and static.

Sarah: What is it about the internet that makes some people incredibly vitriolic? What is the driving motivation behind youtube comments and death threats to prominent bloggers? Can we change the motivation structure somehow to make people more humane without sacrificing anonymous speech?
Tim: There’s two parts to this. On one hand, there’s some sense that the vitriol and assholery that typifies YouTube comments are actually just part and parcel of the aesthetics of communication on the web. So, there’s a part of me that says to not feed the trolls, take it in stride, move on, etc etc. On the other of course, this flavor of social interaction on the web is occasionally at odds with getting things done, and as you mentioning can be bordering on real danger in the form of death threats or otherwise. Not to make light of this, but I’ve always been fond of Randall Munroe’s proposal of having YouTube comments read back to you before they are posted. It points at the need to leverage design features in online spaces to adjust and shape human interaction. We’ve been looking into this at the Berkman Center with Yochai Benkler’s work — the general idea is to examine a broad range of cases in a quantitative way, and figure out the relationship between these structural features and how people collaborate (or don’t) together.

The purpose of these interviews (in addition to just being fascinating) is to promote my panel proposals at this year’s sxsw. In Generation Y and the Future of Nonprofit Communications, I’ll be talking about how to connect with folks like Willow, who care deeply about their communities, but also have very strong preferences over communication style. In Recruiting and Retaining Generation Y: Cheap But Not Easy , I’ll explain why you need people like Willow on your upper management team in order to keep up with an exponentially accelerating technology market. Please vote for those panels if you feel they would benefit the sxsw community. Today is the last day to vote!

Some of the resources mentioned on the panel:

Ways to post to multiple sites at once: Ping.fm and Hellotxt.com
Short explanatory videos about technology and social media: Common Craft
Demographic information about social networks: danah boyd
Alternative copyright licensing options: Creative Commons

Twitscoop scans twitter in real time for words that are suddenly becoming popular. This screenshot was taken 90 minutes after the ruling. I think it tells its own story, but I would like to add my appreciation that the first three readable words in the cloud are “ban bigots bullshit”. A rallying cry if I’ve ever heard one!