Moving the Economist to Drupal

Day 1 of Do It with Drupal Conference - #DIWD

I haven't blogged in ages but I figured this is a good enough way to get started - taking notes at a Drupal conference.

Panel: Rob Purdie (Economist); Moshe Weitzman (Cyrve)

RP: Moving the Economist incrementally and iteratively and working on improvements as they go. Currently these aspects are on Drupal: user comments and recs, user comment history is Drupal, articles and syncing all data to Drupal every 5 minutes. Article pages will be served in Drupal very soon.
Will be launching "channel" pages - currently there are "topic" pages that just list the articles incrementally, the channel pages will have the articles but will integrate additional content onto the pages.
"We benefit from Drupal sooner by taking this approach"
Vision: "Build the foremost destination online for analyzing and debating the global agenda, drawing on the intelligence of journalists, readers and guests."
Want to: increase publishing volume with user-generated content -- This makes me think of MercyCorps and what I call "Shortening the Content Supply Chain"
"Perfect is the enemy of better"

Selling Scrum/Agile Development
• Needed to sell scrum within the org
• Needed to sell scrum to management - emphasized that risk wold be decreased vs. a traditional management approach.
• "We chose comments/recommendations as the most important sub-system to deliver/iterate on first" - worked with the key people to determine what the most valuable areas were for the clients.
• Trained the Management and development teams in Scrum, then in Drupal.
• Developed a Sprint system - every two weeks set out specific deliverables with the intent to deliver (put live?)
Incremental Architecture
Integrating the thumbs up/down from the old platform
• Since comments have to be attached to nodes, they had to create the nodes on the fly for every request that comes up since the articles are not in the same system. Same thing is happening with the Voting API.
Migrating Content - better to strart doing this this sooner rather than faster. Cyrve's methodology for migrating legacy data:
2 Modules:
• Table Wizard - Looks at a mySQL table and will map out selected parts of the legacy node. You can make legacy columns in and also rename old terms. This allow you to map the old data to the new and keep them linked for reference.
• Migrate - build on the above and allows you to build views based on what is created in the tables.
? How many servers did it take for 20k page views a month? Maybe around 12 right now.

? SCRUM Process - How do you manage "emergency" scrum requests - Product Owner is the shield for the team. Requests go to that person and they approach the team and work together. They Split the site up into several product teams (articles, SN, etc.) - the Protected the teams.

Interesting Features
• Using "Pressflow" instead of Drupal 6
• Base theme is a 960 grid - allows you to place everything into a specific column - more uniform presentation.
• Continuous integration using "Hudson" -
• Apache SOLR hosted by Acquia - thousands of articles, current search not working well; emphasis on how fast it is to get quality search results (1 work day); can be self-hosted but there are benefits to the Acquia service.
? What Managing Tools are being used for SCRUM? You don't have to worry about resource management as much because the team approach is laid out that each member has 100% dedication to that project. Trying to move away from specialists toward 'generalized specialists' and to have everyone collectively own the product and so that you're not dependent on one or another person. Favor really low fi tools; influenced by: http://agilemanifesto.org/
• Use "Unfuddle" as a ticketing system, SCRUM "stories" are taken down into tasks
• Use Google docs a lot
• The "Focus should be people, not the tools"
Impediments
• Bad Practice: Command and Control - Telling people what to do and when by inhibits self-organizing teams.
• Black Box Development - One of the pillars is transparency; must be abel to adapt processes as you go. Don't let all the development happen in a black box.
• Development Silos - e.g. EX person only in one project, not integrating with others.
• Traditional Line Management - Building teams of peers, not traditional management structure.
Engineering Practices (specifically quality) - Ken Schwaber books talk about SCRUM - it is "a wrapper for your traditional practices" - SCRUM assumes that your engineering practices are great (or will be). Old habits die hard; they want to ship potentially implementable code with each iteration, in order to do this they have build an identical development environment with just fewer servers in order to ensure that new code will go through.

? Suggestions on working on live content; pre-readying content – MW: Actually, I usually practice on live servers and just be careful to keep it from the public.

On using Acquia SOLR – Self-host vs. not – if you can, it’s not a bad idea but you need to have a lot of mySQL and PHP skills and ability to maintain, Acquia takes care of this. Acquia’s facets are not available on the core search without additional modules. Apache SOLR is used for Netflix, CNET, etc.

With SCRUM, what you would ideally like is to have a product owner sitting in the room in all meetings, even sitting in the room. You are not allowed to change requirments mid-sprint. You have a “working agreements’ that lays out how you agree to work together. You can’t say yes to everything, the project will fail if you do.

SCRUM is not a prescription, you can pick and choose what parts you need. There are simple rooms, make rules; inspect your **, and **.