Conference Report: 2008 LISA
Conference Report: 2008 LISAThe following document is intended as the general trip report for me at the 22nd Systems Administration Conference (LISA 2008) in San Diego, CA from November 9-14, 2008. It is going to a variety of audiences, so feel free to skip the parts that don't concern or interest you.
I woke up before the alarm and managed to get out the door on schedule to head off to the Detroit airport. Got there, checked my bag, and cleared Security with no troubles. Security had no line, and two idle agents fought (good-naturedly) over who got to relieve their boredom by doing the CPAP bag check. We were delayed about 10-15 minutes in departure for fourteen passengers who were trapped in the terminal thanks to a fire alarm, but still managed to get into San Diego on time. After an interminably long wait at the unmoving baggage carousel, I reclaimed my luggage and caught a shuttle to the hotel, where one of the passengers insisted he had no luggage despite his suitcase being in the back fo the van. Anyhow, I checked in and got to my room quickly enough. A quick unpack and I headed off to lunch.
Since I hadn't had anything substantial for breakfast (Quaker granola bars don't count), I decided to just eat on campus. While in line at Sunshine Café (the breakfast buffet place in the middle of campus), I ran into AEleen Frisch and Mike C, so we dined together (and were joined later by Kyrre). After lunch, headed over to the Atlas Foyer, chatting with Mike C and the staff doing setup before registration opened at 4pm. Grabbed my badge and hung out in the area and went online (since paying for in-room Internet at $10/day is absurd when the conference space has free wireless) and chatted with folks as they arrived (including but not limited to AEleen, Elizabeth, Kyrre, Lee, Lois, Mark R, Marybeth, Matt D, Mike C, Moose, Paul, and Rudi) until a bit after 7pm when a subset of us (Lois, Marybeth, Mike C, Mark R, and I) went to dinner at the on-property steakhouse, Kelly's. I had a 12-oz prime rib dinner, which came with a salad (garden, blue cheese) and two sides (loaded baked potato, no sour cream — and their chives were really chives, not green onions — and barbecue baked beans). After dinner, which wrapped up around 9pm (or midnight biological-time), since Bob and Ted were zonked enough to not want to do a nightcap, I just headed back to the lobby to catch up on email before heading to the room to collapse for the night.
I slept moderately poorly (first night in a new bed syndrome) and was up by 7am local without the benefit of an alarm clock. My usual breakfast bunch was sleeping in today, so I did some trip report writing¹ before heading off to the conference area to start my day of meetings.
Caught up with Andrew, got hugs from Geoff before he had to run to set up the room, then did a late brunch with Amy, Bobert, Ted, and Trey. After that, met up with Dan, we hung out in the foyer chatting with folks between my meetings (skipping lunch thanks to the late breakfast). Around 3:30pm I headed, with Derek, Philip, and Trey, off to Greg and Pat's place for drinks (two different red wines), munchies (crackers, camembert, an Irish cheddar, and smoked gouda; veggie chips, tortilla chips, guacamole, and hummus; tomatoes stuffed with a mix of rice, bacon, mushrooms, and onions, topped with cheese), dinner (beef vegetable soup, roasted lamb, garlic bread, potatoes, carrots, sweet potatoes, salad, and corn), and dessert (ice cream sundaes, cream puffs, and Tim Tams, plus a local tawny port).
Headed back to the hotel campus, helped open up the LOPSA After Dark suite (1110), swung by Charlie's for drinks (I stuck with water), then headed off to bed before turning into a pumpkin.
Today started with breakfast (quel surprise). I had the buffet with Amy, Bob, Chris, Lee, and Lois.
Monday began my official sessions with the University Issues workshop.² In its fourth year, the University Issues workshop included 15 participants from a variety of higher education institutions, primarily in the US, with representation this year also from Finland, Australia, and Slovenia. Schools represented ranged in size from a few hundred students to tens of thousands, some with single campuses and others with over 20 campuses. The format of the workshop was a semi-structured round table discussion; one participant described it as Advanced Topics for people in education.
One of the topics that generated the most discussion was organizational structure. Many larger schools have divisions between central computing and various academic and adminstrative departments. These divisions lead to tensions and challenges, such as who owns the network equipment within a research cluster or how cost recovery is performed for shared services. Being able to compare methods of dealing with these was a highlight of the workshop. Participants stressed the importance of good communications channels within IT departments and with the rest of the institution and having SLAs to structure support agreements with other departments.
An ongoing area of discussion from previous years was the outsourcing of core software services to such places as Google Apps or other hosted providers. One particpant described her institution's project to provide the option of Google Apps for all students, noting that the challenges were only partially technical and that of greater importance is having a policy and governance structure for the outsourcing of a core IT service. Regarding such efforts, others brought up concerns about document retention policies, particularly for public institutions, and information classification and protection. A slightly different area of concern, particularly regarding email, was the fact that students are starting to prefer newer forms of instant communication over email and not seeing the value in an email account provided by the school; some places have considered providing different communications solutions for faculty and staff versus students or even not providing students with local email accounts unless requested.
Identity management systems were mentioned at several times during the day, as they have implications for many of the other topics of discussion. While some institutions are able to consolidate all authentication and identity management into one system such as Active Directory, others use a variety of solutions, including different LDAP implementations, Kerberos, and Shibboleth, most of which are tied together with local tools. Service authorization is still a problem area; traditional Unix groups, even in LDAP, have limits. One institution is using Shibboleth-style entitlements and LDAP ACLs to limit access to services, and others are using Cosign in conjuction with single sign on.
A number of other topics were discussed briefly, including virtualization systems, print management, feedback structures for users, and open source software. Several places are starting to use virtualization for productions systems, usually lightweight servers (web, DNS, DHCP), however its use in sometimes limited by the political need to tie a particular project to identifiable hardware. Many institutions are inclined to use open source software, however mangement still wants the kind of supportability and accountability that seems to come from commercial vendors.
The workshop closed with participants briefly describing some of the things that they've found that really work well, whether they're large systems or simple tools; the most popular suggestion in this discussion was the use of IPSca for free educational SSL certificates. In response to a final question, most participants said they expected to be working in academia a year from now.
After the workshop, I grabbed a group of people — Aaron, David, Ellen, Marybeth, Michael, Peter, and Travis — and we all headed off to Facist Valley Mall for the PF Chang's. The eight of us split six entrees (honey crispy chicken, kung pao chicken, lemon pepper shrimp, moo goo gai pan, orange peel chicken, and wok-seared lamb) and mini desserts (I had the banana spring rolls with coconut ice cream).
After getting back from dinner, visited the hot tub with Dan K, Ellen, Marcus, Mario, Michael, Peter, Tom L, Travis, and others until shortly before the 11pm close. Grabbed the laptop and headed off to the Atlas Foyer to catch up on email before heading off to bed around midnight when the naked cleaners showed up.³
Despite the late bedtime, for some reason my body insisted on waking up at 6:15am again. Did breakfast substantially later than that and headed off to my workshop.
Tuesday sessions started with the Advanced Topics Workshops. Once again, Adam Moskowitz was our host, moderator, and referree. [... The rest of the ATW writeup has been redacted; please check my LJ and my web site for details if you care ...]
After the ATW broke up, I dropped my stuff in my room and headed out to Todai for sushi with Chris, Nicole, Travis, and Xavier. Pigged out, headed back to the hotel, and was about 20 minutes late getting to the GBLT[UVWXYZ] BOF (also known as the motss.bof or the Alphabet Soup BOF). We had about 18 people come in over the evening, including at least 2 first-time conference attendees; we discussed the whole Proposition 8 issue and other politics, plus what the purpose(s) of the BOF should be and whether it's outlived its usefulness. The concensus was it's still useful to help ease newer attendees into groups instead of making them feel lost at a large conference like LISA, and it's a way to help people understand that this is a safe space. (Alas, as I was late I didn't have my bag with me, so the pink highlighter didn't make an appearance at the BOF this year.)
One bit of funny:
Moose enters. All: Hi, Moose! Moose: This isn't a 12-step program. Steven: It's a 4-step program. Step 1: Cut a hole in the box. Me: Step 4: Profit! After the BOF and armed with David's key, I headed to the LOPSA After Dark suite with Moose to set up for the party. I arranged the bar, poured out the chips and salsas and candies, and prepped the room; Moose cut the lemons, limes, and cheeses. I had a JD rocks and munchies and said Hello to lots of people (including David, Lois, Roger, Sheeri, Tom L, and a whole lot of people WANolJ or WLJIDK) before the noise got too great and I left around 10:45pm (leaving the sweatshirt behind, oops). Wrote up bits of the day in the trip report, trimmed the finger- and toenails, and bandaged the huge blister on my left big toe before actually crashing around midnight.
Once again, my body was awake too early (5:30am, finally out of bed by 6:10am). At least I'm getting 4 to 5 hours of quality sleep a night this year. (Ow.) Hotel breakfast buffet again (I could spend the same amount for a bagel with lox if all I wanted to drink was water), which is already growing old.
The conference technical (as opposed to tutorial) sessions began this morning. My day began with the keynote session, which started with the usual statistics and announcements. This was the 22nd annual Large Installed System Administration (LISA) conference, with about 1,000 attendees (on par with last year).4 This was followed by thanks to the usual suspects: program committee members, external readers, chairs for IT and Guru tracks, USENIX staff and board, speakers, attendees, sponsors, exhibitors, and vendors. Program Chair Mario Obejas reminded us of the Birds of a Feather (BOF) sessions in the evenings, and the poster sessions on Wednesday and Thursday evenings. This year we received 41 refereed paper submissions and accepted 18 and published 17 papers, though they did not announce the numbers at the conference.
This year's awards for Best Paper were:
- Best Paper — "ENAVis: Enterprise Network Activities Visualization," by Qi Liao, Andrew Blaich, Aaron Striegel, and Douglas Thain, University of Notre Dame
- Best Student Paper — "Automatic Software Fault Diagnosis by Exploiting Application Signatures," by Xiaoning Ding, The Ohio State University; Hai Huang, Yaoping Ruan, and Anees Shaikh, IBM T.J. Watson Research Center; and Xiaodong Zhang, The Ohio State University
The annual Chuck Yerkes Award for Mentoring, Participation, & Professionalism went to Dustin Pruyear for his consistent, helpful, and astute assistance on the member forums. Finally, the SAGE Outstanding Achievement award was given to the Samba team as represented by Gerry Carter.
Sean Dennehy, Chief of Intellipedia Development, Directorate of Intelligence, U.S. Central Intelligence Agency was our keynote speaker. He spoke about Intellipedia, the wikipedia for the intelligence community. Traditionally everything, not just top secret or otherwise-classified information, was compartmentalized, but Intellipedia allows for information sharing at three different levels: top secret (but uncompartmentalized), secret, and controlled-unclassified. One amusing note was that someone complained that information on the unclassified page should be more secure — and it was copied in its entirety from information on the public Internet. Some of the lessons learned include starting small (the acronym list was a huge winner), keeping things simple, and using grassroots efforts to spread it.
My first technical session was the invited talk, "Integrating Linux, Unix, and Macs into Microsoft Active Directory" by Mike Patnode of Centrify. It was a high level overview of some of Active Directory's features and mentioned Microsoft's Services For Unix, but in general was at too high-level to be useful. It seemed as if the speaker was perhaps too careful to avoid mentioning any commercial products (like his own company's) that it became a content-free talk. I was disappointed there wasn't more technical information here.
For lunch, headed out to the Mediterannean place at the Facist Valley Maul's food court for gyros and rice. After lunch, I did a quick swing through the vendor exhibition. Didn't see any great schwag; Google/YouTube had Rubik's cubes with the logos of several of their products and services instead of solid colors, but I didn't grab one before they vanished. One booth had a real booth bunny; I admit to feeling sorry for the poor woman wearing that rabbit suit. (And very sorry for hearing all of the subsequent "furry" jokes.)
After the break, I went to the invited talk "How to Proceed When 1000 Call Agents Tell You, 'My Computer Is Slow:' Creating a User Experience Monitoring System," by Tobias Oetiker. He'd won the SAGE Outstanding Achievement award in 2006, a Swiss magazine printed a congratulatory article about it and stuck him on the cover, which two managers at Swisscom saw and since they had problems with their call center they contacted Tobi to get it fixed. This talk was about what they did and some of the lessons learned.
The problem was that the computers weren't fast enough for the call center staff and was having a horrible impact on productivity. Despite having the IT staff physically watching what the call center people were doing, there was no obvious problem. IT ran some debugging tools and found nothing wrong. They needed to monitor how the computer was working during a call: what happens in which window, how long database queries take, and so on. But it needed to be passive (from user's point of view), let users give their input, be simple to set up and update, and have a central data store.
They implemented three tools for this:
- Client Performance Viewer (CPV) Monitor, to observe the system
- CPV Reporter, to let user report problems
- CPV Explorer, to view the results
Both storage and analysis of the collected data were problematic. As an example, one day (October 27, 2008) 1459 devices sent 2,417,807 samples. 40 days of data is about 100 million samples. The index doesn't fit in RAM on the 4-core 32-bit 4GB RAM machine and it's too much data to process. Postgres allows for a function-based index, so they used a 2-byte encoded number of hours since 2007, metric ID, and workstation ID as well as 2 SQL WHERE conditions to help manage the index.
To reduce the data to analyze, they assign a 2-byte random value to each sample, so to sample n% of the data, "select all samples with rand < maxrand n/100" will get those. However, data analysis is beyond the scope of the project.
The biggest lesson learned is that just monitoring the problem and providing a "this sucks now" button made the users happy. They managed to fix most of the problems and the side effects CPV introduced, though they still have some challenges with future work (including dll injection, webapps, and dealing with the amount of data).
The fourth session of the day didn't actually hold any interest for me today. Instead I did some catching up on email and hallway track conversations before exchanging the utilikilt for jeans and heading out to the 0xdeadbeef dinner. We went to Fleming's. The table split two of the wicked cajun barbecue shrimp appetizers (nice burn, great flavor). I went with the house salad of seasonal greens, candied walnuts, dried cranberries, tomatoes, and croutons, in a balsamic vinaigrette, and followed that with a 22-ounce bone-in ribeye steak medium rare with roasted garlic mashed potatoes and grilled asparagus. As a table we shared two bottles of cabernet sauvignon, though I didn't write down which.
Got back to the hotel campus in time to set up and prep the LOPSA After Dark suite (made more entertaining by not knowing what the power-shoppers brought back). Stayed for the party until it got too loud, then went back to the bedroom for a backrub from Frank before collapsing in bed.
Today I had breakfast with Amy, Bob, and Steven, with Dave, Elizabeth, Kalida, Mark, Moose, Opal, Paul, and Travis nearby. The conference day started with an experiment in asynchronous human search engines: People would say their name and ask a question, for "us" to answer by finding the questioners after the talk. Ran for 30 seconds per person, for about 8 minutes. It was fairly successful.
After that, the plenary session by Bruce Schneier, "Reconceptualizing Security," began. He talked about security, but not from the technical side; instead, he focused more on how we think about security, how the brain processes security fear and risk, and how our perceptions affection how we think about it. Security is both a feeling and a reality, and there are questions as to how we model it. This is two different concepts mapped onto the same word, and there's some value in distinguishing which of those we mean when. Part of the problem is that there're no good words for some of the concepts involved.
At a base level, security is a tradeoff: You give up something (time, money, or convenience) and get security. The question isn't "Is this any good," but more "Was this worth it?" People tend to have natural intuitions about the tradeoffs, but while you'd think we as people would be good at it, why do we suck at it in reality? We make the tradeoffs based on the feeling but not on the reality. This works well when the feeling and reality are the same, but when they're not it doesn't. Our brains take cognitive shortcuts all the time; we respond to stories more than facts.
Since this is hard, security is often designed to make you feel better but not have any real effect on reality. The resultant "security theatre" is what we see in, for example, airport security procedures.
My next session was the invited talk by Jordan Hubbard, Director of Apple's Unix Technology Group, titled "Mac OS X: From the Server Room to Your Pocket." He started with a quick history of OS X, from 2000 to 2009, with the release cycles (9- to 24-months) and the new functionality with each release. He also provided an unscientific metric, the Linux World Mac Laptop Adoption Factor (LWMLAF), which is the percentage of Mac laptops he'd seen participants using at Linux World. It ranged from none when 10.0 came out to well over 70% by the time 10.4 came out. He next detailed the new features in 10.5, organized into security features and Unix geek features.
He touched on some of the scary things coming up in the future, including the rise of the GPU, and the increasing number of cores (with more than 32 cores per CPU in 2010). We're out of head room in clock speed so we have to go lateral in design, but programming for multi-threaded cores is hard, and only the kernel can determine the right mix of cores and power states at any given time.
Ubiquity is happening with things like the iPhone. Lessons learned include requiring code signing and that innovation runs both ways (from desktop to phone and vice versa). It's all about power, which is about resource consumption, so we need to consider assumptions better. A questioner argued is it's more that efficiency is important.
In summary, a decent-enough talk, but it was more heavy on the history than on current integration or future plans, so I was a little disappointed. On the other hand, it was still interesting information and well-presented.
(Side note: Jesus Fracking Christ on a crutch, why is everyone asking questions from their seats during the talk? (And if you're one of those people and I see you, run away or I will stab you in the face with an unpowered rusty hand-cranked chainsaw.) The session chairs should be actually reminding people that questions in invited talks belong at the end and via the microphones.)
After another Facist Valley Maul Food Court lunch (this time pizza and ziti from Sbarro with Ellen and Paul), I went to Tom Limoncelli's guru session on time management. Like all Guru sessions, it was more q-and-a than a formal presentation. Tom started with his five tips:
- Use a mutual interruption shield. Coordinate with a coworker so one of you covers the phones, email, triage, or whatever for the morning so the other can get project work done, and then switch for the afternoon.
- Get into a routine. With order in your life you can reduce chaos and get into a routine.
- Document all requests. Don't rely on your brain; use paper, text files, a ticketing system, PDA (or PAA), whatever.
- Keep a daily to do list. Control your time. Schedule your work, prioritize what you do, and control the hours you work. Mail your manager monthly with status brain dump.
- Document the procedures you hate to do: checklists, procedures, the error-prone or do-infrequently tasks. Consider audio and transcriptions as well. Video may be possible.
Some of the topics we touched on were managing within a team, distributing tasks within a group, tips for working from home, documenting everything, and how to say No. Finally, we talked about what managers can do to help, including defining the scopes of control and responsibility, defining what constitutes an emergency in advance, and specifying how users get support.
My last session of the day seems to have been the most contentious as to whether folks liked it. Janice Gelb from Sun's invited talk, "WTFM: Documentation and the System Administrator," was a high-level overview about writing. After providing reasons to write and examples of what to write, she focused mainly on some tips and tricks for content like adjusting tone, including reasons not just the results, using titles instead of names, expanding acronyms on first use, providing a glossary, and avoiding passive voice (most of the time). She provided some examples of and formatting ideas for procedures, lists, tables, FAQs, and illustrations. On the question of how to improve docs for products we use, the best way is to complain to the vendor: file bug reports, use any feedback form, tell your sales engineer or account representative. Also you should contribute (especially in open source projects), write articles, and if it's an internal document you can offer to work with your documentation team.
The talk was (as I expected) pretty basic, but I still found it a useful refresher. Some colleagues (also sysadmins who write) thought it was too basic, that some of the tips were more for dedicated writers and would scare sysadmins away from writing.
For dinner this evening, four of us — Aaron, Chris, and Mark — went out to Rei Do Gado Churrascaria in the Gaslamp district. The all-you-can-eat meats were filet mignon wrapped in bacon, leg of lamb, pork loin, pork ribs, pork sausage, sirloin with garlic, skirt steak, top round with garlic, top sirloin, and tri-tip, as well as the all-you-can-eat salad bar. We were too stuffed for dessert so cabbed back to the hotel.
Thursday night was the night of the thousand parties. Well, okay, fine, just three. The scotch BOF was upstairs in the presedential suite on the tenth floor, and was mostly dead, but had good scotches; the LOPSA After Dark suite was open for its last night, and again was both crowded and loud; and the semiprivate birthday celebration that Cat threw was quiet and laid back, with good scotch and munchies. I managed to visit all three, though most of my time was spent at Cat's. I did find out that my tuxedo no longer fits, unfortunately, so I need to do something about that before the next time I actually need black tie.
I started the day with yet another hotel buffet breakfast with Amy, Bob, Geoff, and Lois.
The sessions began with another brief round of Human Search Engine before the final plenary, "The State of Electronic Voting, 2008," by David Wagner of the University of California at Berkeley. He started with some history: the 2000 election problems in Florida, which led to Congress passing the Help America Vote Act in 2002, which led to municipalities spending money on electronic voting machines. However, the machines (and their software) tend not to be secure.
One problem area is user interface design. In one Sarasota (Florida) election, the margin of victory was 369 votes (0.15% of votes cast), but in that race there were over 18000 (14%) votes not recorded; other machines show an undervote of 2-3% on average. In this particular race, the entire section and both candidates were at the top of the page as if it were a web page banner. The hypothesis is that many voters just didn't see it to vote, since we're effectively training ourselves to ignore that area of web pages ("banner blindness"). Another problem area is programming. Some machines overflowed their counters and either threw away or overwrote votes, disenfranchising voters.
He next talked about security. One vendor had their entire source code repository (in CVS) on their FTP server, and inspection showed a lot of bad coding practice. Many vote tabulator servers use no authentication at all, trusting anyone who connects.
In 2007, California Secretary of State Bowen required an investigation into all three companies. Vendors had to comply with the investigation or lose California certification; one chose not to, was decertified, and subsequently chose to participate. Some of the findings were that Sequoia developed their own password encryption algorithm (to append three specific characters to the unencrypted password), Diebold used a hardcoded password ("diebold"), and Hart used no cryptography at all. He showed several bad coding examples; all three vendors' systems allow malicious code to propagate virally, affecting in-precinct machines and the county vote tabulator server. Secretary of State Bowen decertified all three vendors and in the 2008 elections, Californian voters used mostly computer-scanned paper ballots with mandatory auditing to compare the paper and the computer vote counts.
In 2008, we saw no major fiascos, but we also saw some equipment failures, a lot of long lines, and some registration-matching issues. There's a video on YouTube which shows how voting in one race can affect the voter's already-entered vote in another race.
In short and in summary, elections are hard to get right, and while electronic machines can make things easier or faster, auditing is required. If you need the paper trail for auditing you're not getting any cost savings from electronic voting. Electronic voting does solve the problems of voting for disabled or impaired voters, having multiple languages available, and it makes some aspects of chain-of-custody easier. Eleven states (AR, GA, IN, KS, KY, LA, NJ, PA, SC, TX, and VA) don't have any paper trail or auditability, so there's no possibility of a recount.
Andrew Hume gave his invited talk, "Deterministic System Administration: The Battle Against Pixies and Entropy." He talked about the tools he's created in Ruby to track systems, racks, and wiring. The first tool, fettle, which needs to be rewritten, uses a text-based language to define the connectivity between boxes and generates VRML diagrams as output. It can also generate cable orders and analyze power utilization. Completeness of inventory is now theoretically trivial, since everything can be tracked in fettle. Completeness of the data center needs to include fibrechannel and ethernet patch panels. (See the 2007 LISA WIPs for more.) He also has 2 new tools: limn, to define clusters, and treetop for node specifications (though it's very slow). Enhancements include adding networking and possibly using graphics engines instead of VRML.
My last visit to the Facist Valley Maul Food Court was to what is now "Muscle Beach" but is still considered "Hot Dog on a Stick." Headed back to the hotel campus for the DreamWorks invited talk about the technology they use to make their computer-generated animated films. In addition to the technical bits, they showed us clips from several of their movies to illustrate their points.
A CG animated film takes over 2 years in preproduction, 1.5 years of a production schedule, requires over 200 artists, and all produces a 90-minute movie made up of 129600 frames. Each of those frames requires computer processing. The technology required is complex, since calculations include fluid dynamics and particle simulations. The computing needs high throughput with hundreds of thousands of individual jobs per day, and it must be reliable since some simulations literally take days to run.
They run Linux on commodity x86 hardware. They have over 2600 render farm servers, 350 back-end infrastructure servers (75% are Linux, the remaining 25% are Windows and all for their Exchange system), and over 2800 desktops (over half of which run Linux). A movie takes over 20 million rendering hours and can produce over 5 TB of data just for production (ignoring backups, temporary files, and version control).
The lessons they've learned are that building render farms has both technical (specifications, performance, and support) and logistic (ordering, taking delivery, and installing) aspects; that power, space, and cooling is important; and that they still need to be able to handle the unexpected (such as power or cooling failures and working around bugs).
The final session of the conference was the return of a LISA Quiz Show. Instead of pitting three or four individuals against each other, there were two teams of three (the Geeks and the Nerds), each competing mostly as a team. There was a new judge (Mario Obejas) joining Adam Moskowitz at the judges' table with game-driver Dan Klein. It's new (browser-based) software and with a new host, Jeremy Allison of Google. He arrived on stage wearing a neon pink jacket, silver sparkly bow tie, and a 1950s-ish white tuxedo shirt and bell-bottomed trousers (with gold lame flare). His costume began falling apart during the show, he mispronounced two of the three judges' names, and was unable to read the contestant biographies. (To be fair, some of the contestants had lousy handwriting.) I admit to bias here; I missed Rob Kolstad's hosting skills and I'm a little miffed that I wasn't even asked to participate in test, setup, judging, or production. I found Jeremy's patter during the event less than ideal. His "LOPSA, whatever that is" throwaway during the contestant biographies immediately red-flagged him; his patter felt forced; he seemed more insulting and humiliating than Rob Kolstad ever did. And his having to obviously step on the reset button for the hardware between questions (and his regularly having to be reminded by the audience to do so) was annoying.
The quiz show was in three rounds. In round 1, each question was worth 500 points and had a 250-point bonus question; in round 2, each question was worth 1000 points and there was no bonus question. The last question in round 2, which ended with a 6750-6750 tie, had to do with trebuchets. So round 3 was using trebuchets: Each team had to shoot candy from their trebuchet into a specific target. Getting the candy in the laundry basket was worth 1000 points, hitting a member of the opposing team was worth 5000 points, and destroying the enemy trebuchet was worth 10000 points. (Nobody answered how many points it was worth to hit the host.) The Geeks managed to get 5 candies into the correct basket (and 1 into the wrong one), and the Nerds only managed to get 1 into the correct basket, so the Geeks beat the Nerds 11750 to 8750.
After the conference formally ended, I wound up swinging by the LOPSA After Dark suite to collect folks for dinner. Ten of us — Aaron, Alan, Beth Lynn, Lois, Matthew, Steve, Trey, and two others whose names I didn't write down — went to the Pizzaria Uno's at Facist Valley Maul. We were later joined by Cat and Tom. I had a caesar salad and an individual pepperoni pizza and was comfortably stuffed.
Headed back to the hotel to pack up (except for the last-minute items, like CPAP, toiletries, and phone chargers) before the Dead Dog in the combined LOPSA After Dark suite and the neighboring suite where Dan Klein was staying. Some munchies, some drinks (a small glass of Balvenie, I think), and a game of 1000 Blank White Cards where I got to help create the starter deck. Gave up around 12:30am and headed off to bed.
Today was the travel day to return home. I managed to sleep in until 8:30am; I finished packing and headed down to the office to check out and schmooze with other departing folks (including Adam, David, Gabe, Geoff, Lois, Michael, and Travis). Wound up sharing a shuttle ride with Lois, and after checking our baggage we went through Security and hung out in the food court (I had my only actual meal of the day, a chili-cheese dog and fries) before it was time to head to our gates for boarding our respective airlines' flights.
The flight itself was uneventful. My row (6 portside) was me, a retired Navy man (in his mid 60s) wearing a Top Gun hat, and an active-duty Marine recently back from Iraq (in his early 20s). Our flight attendant had been doing this for 33 years (and is ready to retire), but was great at sassing us; when the Marine hadn't turned off his iPod and stowed his tray during landing, she actually (and obviously jokingly) told him to get his shit together.
Maybe Northwest heard how annoyed I was that it took for-fucking-ever to get my bags when I'd landed in SAN (it shouldn't take 20 minutes to get anything from the gate to the baggage carousels in an airport that small, especially when it took me, walking slowly instead of at-speed, so little time to get there), as my bag was the first one up the conveyor at DTW. Got to my shuttle driver, loaded up, and was home by 9:11pm. Managed to unpack, sort through the postal mail (though I didn't process any), get the laptop back on the home network, did the expense report, and collapsed in bed around 11pm.
| ¹ | How else do these get done within the first couple of days back in the office? | |
| ² | Much of the workshop writeup written by Rowan Littell. | |
| ³ | The security person let us know around 11:15pm that the cleaning crew would be by at midnight and needed the area clear so they could get everything, well, cleaned. Our comment — and no, it wasn't mine — was that they obviously were going to be cleaning naked and didn't want the audience. | |
| 4 | "Registered attendees" is misleading, as it includes people who are registered for one or more tutorials, one or more days of technical sessions, or an exhibition floor booth, and also includes comped registrations in addition to the paid registrations. |
Back to my conference reports page
Back to my professional organizations page
Back to my work page
Back to my home page