Do you believe that there is only one "right way" to do things? Do you ignore evidence that other solutions may also work? Do you dismiss the possibility that there are reasonable trade-offs to be made between the "right way" and some other way? (do you leave angry comments on my blog?)
Do you think I'm kidding? Read these comments about GET vs POST. There's no question that POST is the "right" way and generally safer, but sometimes it's annoying. Even though GET isn't "supposed" to work, it often can be made to. Don't believe me? Google does billions of dollars a year in GET based transactions in the form of CPC ad clicks (which can cost over $50/click). A lot of other things work this way too, including most non-ajax webmail (for changing read/unread state, at the very least).
If you're writing missile guidance software or something then, please, please, please do things the right way -- it's better safe than sorry, right?
However, if you're writing little web apps, then there's a good chance that you can bend a lot of the rules and produce a better product sooner. I'd rather have something imperfect but useful and popular than something "perfect" but unfinished and unused. Sometimes, it's better sorry than safe.
When is it safe to bend the rules? Just ask yourself, "What's the worst that could happen?" If the worst case isn't too bad, then you're probably ok. Minor bad things will happen no matter what, so it's often better to put your energy into general problem recovery, rather than imagining that you can simply avoid all problems. (this is especially true on the web)
Wednesday, June 13, 2007
Quick: Vibe
Seth Godin on "the vibe":
This is a very real and under-appreciated part of work. It was one of the first things I noticed when I started at Google after working at Intel -- the place just felt so much more alive. Now it's something that I watch for when visiting startups -- my guess is that upbeat and energetic companies will outperform the ones that feel oppressive and hopeless.
Have you ever been at a banquet or in a boutique or at a concert or a meeting or a company where the vibe was incredibly positive?
...
If vibe is so important, why does it sound flaky to worry about it? Who's in charge of the vibe at your place? Could it be better? A lot better?
This is a very real and under-appreciated part of work. It was one of the first things I noticed when I started at Google after working at Intel -- the place just felt so much more alive. Now it's something that I watch for when visiting startups -- my guess is that upbeat and energetic companies will outperform the ones that feel oppressive and hopeless.
Monday, June 11, 2007
Quick: Wolves
A great story from Steve Yegge. It sounds like Steve recently realized that there are wolves at Google. Can you see the wolves in your organization? How are you protecting yourself? (or are you being eaten alive?)
"Have you ever noticed that all our folklore about wolves has certain themes and patterns to it? I've never met a wolf, but I give them fair credit for being wily creatures. I'm a big man, not one to fear a big dog's bite, but I would give a wolf a wide berth, because they're reputed to be so crafty. And crafty, friends, I am not. A craftsman, yes, but I realize now that I am more sheep than wolf.
"The landowner in my tale is a good, affable fellow; I could see it within minutes of meeting him, and I stand by him today still. He's a man with a clear vision and a good heart, and he hired crew members who, for all our talents, know nothing of wolves or wizards.
"The problem with setting up an untended, untrained flock of sheep is that there are wolves aplenty, and as sheep are not adequately prepared to recognize or deal with wolves, they will all eventually be eaten. This seems obvious to us. Nearly as obvious is that this idea applies equally to government, with the minor change that the wolves are savvy politicians and the sheep are incompetent ones. Eventually the sheep are all eaten, so our assumption is generally that the government is filled entirely with wolves — an assumption that has proven to be essentially axiomatic, across time and nation.
"What's far less obvious is that the idea applies outside flocks and governments, with the same consequences. If you set up any organization of saintly do-gooders, with no politics anywhere in sight, then the presence of a single master politician can go unrecognized for long enough for every single sheep to be eaten and replaced with a wolf. At some point the shepherd looks down and perceives that he now has a flock of wolves, and he wonders how it happened. But a shepherd rarely witnesses the process while it's in motion, because the wolves are so sneaky.
"I was being eaten alive on my mansion project, and yet I was utterly oblivious to anything but the need to survive, which due to various odd circumstances required working without sleep and eating marshmallows without respite, apparently until either I was dead or forever happened, whichever came first.
Saturday, June 9, 2007
Email blackmail is unnecessary
From Slashdot:
It's tempting to think that such schemes, although distasteful, are necessary to fight spam. They are not.
A good reputation system can be used to provide reliable delivery to high-volume, non-spamming senders, and it can be done without punishing non-profits, mailing lists, startups, or others who can't afford to pay extra delivery fees.
How does that work? Brad Taylor of Google published a paper about the Gmail reputation system, which is quite effective (pdf or html).
Unfortunately Google does not currently publish this reputation information, though I'm hopeful that they will eventually. (but I know that the team there is already hard at work on a number of other important projects)
Under the guise of fighting spam, five of the largest Internet service providers in the U.S. plan to start charging businesses for guaranteed delivery of their e-mails. In other words, with regular service we may or may not deliver your email. If you want it delivered, you will have to pay deluxe.
It's tempting to think that such schemes, although distasteful, are necessary to fight spam. They are not.
A good reputation system can be used to provide reliable delivery to high-volume, non-spamming senders, and it can be done without punishing non-profits, mailing lists, startups, or others who can't afford to pay extra delivery fees.
How does that work? Brad Taylor of Google published a paper about the Gmail reputation system, which is quite effective (pdf or html).
... spammy domains get nicely clustered around a reputation of zero. We know these domains are spammy because if our users disagreed with us, they would unmark spam on them and the reputation would no longer be zero. Nonspam domains are more loosely clustered in the 90-100 range. There ends up being a smattering of domains in be- tween. Many of the messages in the in-between category are from legitimate bulk senders, and the lower than 90 percent reputation is a reflection of less-than-ideal sending practices. But not all bulk senders are the same. Some have very high SPF and DomainKey reputations. For example, eBay's DomainKey reputation is 98.2, enough to be whitelisted by Gmail. This is the "holy grail" of bulk sending, and it is all done without requiring any payment or extra effort on the part of the sender other than just having good mail hygiene.
Unfortunately Google does not currently publish this reputation information, though I'm hopeful that they will eventually. (but I know that the team there is already hard at work on a number of other important projects)
Friday, June 8, 2007
Optimizing everything: some details matter a lot, most don't
Optimization is something that all engineers and product designers (all people, really) should understand. However, their words and behavior suggest that most either don't understand it, or haven't really internalized the lessons of optimization. (and by the way, I'm talking about CS-style "make it better" optimization, not "make it optimal" optimization)
Like everything, optimization is a complex topic, so I'm going to simplify it into three easy steps:
Most programmers probably think of optimization primarily in terms of CPU profiling, but it really applies to everything. (and in fact, this post is about the general process of optimizing, especially for things other than code -- for specific cases of hardcore game engine optimization or whatever it might be totally wrong)
For example, people who continually "pinch pennies" (perhaps driving around town to save a few cents on gas) but then waste thousands of dollars on something stupid like a fancy car or credit card interest are probably not optimizing their spending very well. They are putting most of their effort into low reward activities (saving a few cents on gas) while losing much more money in other places. Big, dumb, Dilbertesque companies are also notorious for this kind of thing, cutting back on pencils while running super-bowl commercials, or something.
This is also the kind of activity that I was talking about in my post titled "Wasting time on things that really don't matter". It's not that there is zero difference between two slightly different ways of testing for the empty string, it's just that your product probably has 100,000 more serious problems, and so time spent worrying about the empty string style is time NOT spent fixing those more important problems. Put another way, fix all of the other problems first, then debate the empty string.
Of course you may enjoy debating the empty string (and in fact there are some interesting arguments), and that's fine. The zeroth step in optimization is to decide what you are optimizing for. If you're looking to maximize the amount of time spent on endless debates, or perhaps code "beauty", then maybe the empty string debate really is the highest benefit/cost activity (because the benefit is so high). If, on the other hand, you are trying to maximize the success of your product, then the empty string debate probably has negative value and should be strongly avoided.
The empty string comparison is obviously a small detail, and it's tempting to think that we should not bother with such small details, but that too is a huge mistake. Another important lesson of optimization is that some small fraction of the details will determine the majority of the outcome. This is commonly referred to as the 80/20 or 90/10 rule, stating for example that "90% of the time is spent in 10% of the code", but it could just as easily be 99/1 or something like that (and for code that has already been well optimized, this 80/20 rule may not hold -- it's just a heuristic). In code it may be just a few lines in the "inner loop", or perhaps it's the interest rate on your mortgage ("what's a few percent?"). I once profiled a server at Google and discovered that something like 90% of the time was being spent on a single line of code: a call to sscanf (repeated millions of times) that someone had snuck into the scoring code (that's the 99999/1 rule).
Determining which details matter a lot (and which don't) is also the key to designing successful products. In the classic slashdot iPod post from 2001, they summarized the newly released iPod thusly: "No wireless. Less space than a nomad. Lame."
History suggests that those weren't the important details, since it's 2007 and there still isn't an iPod with wireless (not until the 29th, at least), and most iPods have less than the original 5GB of storage. Apple did, however, get a few very important details right: it's easy to get your music onto an iPod, it holds "enough" music (unlike the early flash players), and it fits easily into your pocket (unlike the early disk-based players).
So how do we determine which details are important? Observation. (and some judgment, but I think that's mostly formed via observation) If you're optimizing an application, observe which resources are being exhausted or causing the delay (it's not always CPU -- it's often disk seeks), and then observe where they are being consumed. If you're trying to optimize money, first observe where most of the money goes to and comes from. To optimize a product design, observe which things are most "useful", and which are most painful or annoying (waiting or thinking, usually).
If you get all of the important details right, you'll be so far ahead of everyone else that the other details probably won't matter. But remember, your time is finite, so time spent on unimportant details is time not spent on important details! Most engineers are very detail oriented people, and this is good because they can really focus on the important details, and it is bad because they can really focus on the unimportant details! I suspect that learning to tell the difference will be worth your time.
By the way, to get even more details right, decrease the cost side of the benefit/cost equation so that you can do more things in less time (by finding easier solutions, avoiding endless debates, etc).
Like everything, optimization is a complex topic, so I'm going to simplify it into three easy steps:
- Find the problems ("profiling")
- Sort the problems according to benefit-of-fixing/cost-of-fixing ("ROI")
- Fix them, in the order of highest benefit/cost to lowest
Most programmers probably think of optimization primarily in terms of CPU profiling, but it really applies to everything. (and in fact, this post is about the general process of optimizing, especially for things other than code -- for specific cases of hardcore game engine optimization or whatever it might be totally wrong)
For example, people who continually "pinch pennies" (perhaps driving around town to save a few cents on gas) but then waste thousands of dollars on something stupid like a fancy car or credit card interest are probably not optimizing their spending very well. They are putting most of their effort into low reward activities (saving a few cents on gas) while losing much more money in other places. Big, dumb, Dilbertesque companies are also notorious for this kind of thing, cutting back on pencils while running super-bowl commercials, or something.
This is also the kind of activity that I was talking about in my post titled "Wasting time on things that really don't matter". It's not that there is zero difference between two slightly different ways of testing for the empty string, it's just that your product probably has 100,000 more serious problems, and so time spent worrying about the empty string style is time NOT spent fixing those more important problems. Put another way, fix all of the other problems first, then debate the empty string.
Of course you may enjoy debating the empty string (and in fact there are some interesting arguments), and that's fine. The zeroth step in optimization is to decide what you are optimizing for. If you're looking to maximize the amount of time spent on endless debates, or perhaps code "beauty", then maybe the empty string debate really is the highest benefit/cost activity (because the benefit is so high). If, on the other hand, you are trying to maximize the success of your product, then the empty string debate probably has negative value and should be strongly avoided.
The empty string comparison is obviously a small detail, and it's tempting to think that we should not bother with such small details, but that too is a huge mistake. Another important lesson of optimization is that some small fraction of the details will determine the majority of the outcome. This is commonly referred to as the 80/20 or 90/10 rule, stating for example that "90% of the time is spent in 10% of the code", but it could just as easily be 99/1 or something like that (and for code that has already been well optimized, this 80/20 rule may not hold -- it's just a heuristic). In code it may be just a few lines in the "inner loop", or perhaps it's the interest rate on your mortgage ("what's a few percent?"). I once profiled a server at Google and discovered that something like 90% of the time was being spent on a single line of code: a call to sscanf (repeated millions of times) that someone had snuck into the scoring code (that's the 99999/1 rule).
Determining which details matter a lot (and which don't) is also the key to designing successful products. In the classic slashdot iPod post from 2001, they summarized the newly released iPod thusly: "No wireless. Less space than a nomad. Lame."
History suggests that those weren't the important details, since it's 2007 and there still isn't an iPod with wireless (not until the 29th, at least), and most iPods have less than the original 5GB of storage. Apple did, however, get a few very important details right: it's easy to get your music onto an iPod, it holds "enough" music (unlike the early flash players), and it fits easily into your pocket (unlike the early disk-based players).
So how do we determine which details are important? Observation. (and some judgment, but I think that's mostly formed via observation) If you're optimizing an application, observe which resources are being exhausted or causing the delay (it's not always CPU -- it's often disk seeks), and then observe where they are being consumed. If you're trying to optimize money, first observe where most of the money goes to and comes from. To optimize a product design, observe which things are most "useful", and which are most painful or annoying (waiting or thinking, usually).
If you get all of the important details right, you'll be so far ahead of everyone else that the other details probably won't matter. But remember, your time is finite, so time spent on unimportant details is time not spent on important details! Most engineers are very detail oriented people, and this is good because they can really focus on the important details, and it is bad because they can really focus on the unimportant details! I suspect that learning to tell the difference will be worth your time.
By the way, to get even more details right, decrease the cost side of the benefit/cost equation so that you can do more things in less time (by finding easier solutions, avoiding endless debates, etc).
Sunday, June 3, 2007
Java running faster than C
Note: A lot of people seem to be taking this post to be the "Ultimate C vs Java shootout". It's not. Performance is a very complex topic. My only real point is this: Java (which used to be slow) has reached the class of "fast languages". For the majority of applications, speed is no longer a valid excuse for using C++ instead of Java.
I just saw this page comparing the performance of several languages on a simple Mandelbrot set generator. His numbers show Java being over twice as slow as C, but then I noticed that he's using an older version of java and only running the test once, which doesn't really give the JVM a chance to show off.
I quickly hacked up the code to run 100 iterations 3 times and then used my standard "go fast" flags (there may be better flags, but I'm lazy). Here are my results:
C still wins on the first iteration, but Java is actually slightly faster on subsequent iterations!
Obviously the results will be different with different code and different machines, but it's clear that the JVM is getting quite fast.
This test was run with Java 1.6.0-b105 and gcc 4.1.2 under Linux 2.6.17 under Parallels on my 2.33GHz Core 2 Duo MacBook Pro. Here is the hacked up code: Java and C.
For extra fun, I also tried running the JS test using the Rhino compiler:
Compiled JS is about 9x slower than C on this test. If CPU speed doubles every 18 months, then JS in 2007 performs like C in 2002.
Update: A few more cpp flags have been suggested. -march=pentium4 helps a little, but it's still slower than Java.
Adding -ffast-math puts C in the lead, but I'm not sure what the downside is. The gcc man page says, "This option should never be turned on by any -O option since it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules/specifications for math functions." That sounds like an optimization that Java might not use.
Update: Several people have claimed that the performance difference is due to fputs (including the top rated comment on reddit, aumusingly). That is not correct, at least not on my computer. I tried replacing the print calls with a trivial function (but with side-effects), and it actually helped the Java more than the C:
Many people have pointed out that '-O8' is more than enough 'O' levels. I know, and I don't care -- it's just as good as '-O3' or whatever.
I just saw this page comparing the performance of several languages on a simple Mandelbrot set generator. His numbers show Java being over twice as slow as C, but then I noticed that he's using an older version of java and only running the test once, which doesn't really give the JVM a chance to show off.
I quickly hacked up the code to run 100 iterations 3 times and then used my standard "go fast" flags (there may be better flags, but I'm lazy). Here are my results:
$ java -server -XX:CompileThreshold=1 Mandelbrot 2>/dev/null
Java Elapsed 2.994
Java Elapsed 1.926
Java Elapsed 1.955
$ gcc -O8 mandelbrot.c
$ ./a.out 2>/dev/null
C Elapsed 2.03
C Elapsed 2.04
C Elapsed 2.05
C still wins on the first iteration, but Java is actually slightly faster on subsequent iterations!
Obviously the results will be different with different code and different machines, but it's clear that the JVM is getting quite fast.
This test was run with Java 1.6.0-b105 and gcc 4.1.2 under Linux 2.6.17 under Parallels on my 2.33GHz Core 2 Duo MacBook Pro. Here is the hacked up code: Java and C.
For extra fun, I also tried running the JS test using the Rhino compiler:
$ java -cp rhino1_6R5/js.jar -server -XX:CompileThreshold=1 org.mozilla.javascript.tools.shell.Main -O 9 mandelbrot.js 2>/dev/null
JavaScript Elapsed 21.95
JavaScript Elapsed 17.039
JavaScript Elapsed 17.466
JavaScript Elapsed 17.147
Compiled JS is about 9x slower than C on this test. If CPU speed doubles every 18 months, then JS in 2007 performs like C in 2002.
Update: A few more cpp flags have been suggested. -march=pentium4 helps a little, but it's still slower than Java.
$ gcc -O9 -march=pentium4 mandelbrot2.c
$ ./a.out 2>/dev/null
C Elapsed 1.99
C Elapsed 1.99
C Elapsed 1.99
Adding -ffast-math puts C in the lead, but I'm not sure what the downside is. The gcc man page says, "This option should never be turned on by any -O option since it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules/specifications for math functions." That sounds like an optimization that Java might not use.
$ gcc -ffast-math -O9 -march=pentium4 mandelbrot2.c
$ ./a.out 2>/dev/null
C Elapsed 1.66
C Elapsed 1.67
C Elapsed 1.67
Update: Several people have claimed that the performance difference is due to fputs (including the top rated comment on reddit, aumusingly). That is not correct, at least not on my computer. I tried replacing the print calls with a trivial function (but with side-effects), and it actually helped the Java more than the C:
C Elapsed 1.88
Java Elapsed 1.554
Many people have pointed out that '-O8' is more than enough 'O' levels. I know, and I don't care -- it's just as good as '-O3' or whatever.
Friday, June 1, 2007
Wasting time on things that really don't matter
It's easy to waste a lot of time debating things that aren't very important. Maybe that's fun if you don't have anything better to do, but when you're actually trying to accomplish something it can be deadly.
This post on "coding horror" is a great example of this phenomenon, or more specifically, the comments are a great example. At least half of the comments are from people who are debating which of these two lines of code are better:
Here's my answer: It's not important!
It's easy to get dragged into these debates because we all have opinions (and the one form really is better!), and of course the more time is spent talking about the issue the more important it seems.
Here's my trick for killing these stupid debates: Let's list all the issues and problems that we are facing right now. Now, where does this issue rank on that list? Is it in the top 10? The top 100? The top 1000? Are we spending more time talking about this one minor issue than we are spending working on any of our top ten issues? (You don't necessarily have to make the list -- just suggest that it exists and then guess where on the list the issue appears)
Sometimes that works. Sometimes it brings a little perspective.
And yes, this misallocation of time (aka "worrying about dumb stuff") is probably a top 10 productivity killer.
Update: Here's a good link (via news.yc) which offers one explanation for why people often spend the most time on the least important issues:
Update Two: Wasting time is fine, and even inane discussions of the best way to compare empty strings can be interesting. My real point is this: if we are actually trying to get something done, then we need to kill this stuff, or it will kill productivity. If, on the other hand, we're ok being in time-wasting-mode, then silly debates are probably harmless enough. I picked on the blog comments above not because there is anything wrong with them per-se (what are blogs for, if not inane debate?), but because it reminded me a lot of the silly debates that I've seen on work mailing lists (where they do cause harm).
This post on "coding horror" is a great example of this phenomenon, or more specifically, the comments are a great example. At least half of the comments are from people who are debating which of these two lines of code are better:
if (s == String.Empty)
if (s == "")
Here's my answer: It's not important!
It's easy to get dragged into these debates because we all have opinions (and the one form really is better!), and of course the more time is spent talking about the issue the more important it seems.
Here's my trick for killing these stupid debates: Let's list all the issues and problems that we are facing right now. Now, where does this issue rank on that list? Is it in the top 10? The top 100? The top 1000? Are we spending more time talking about this one minor issue than we are spending working on any of our top ten issues? (You don't necessarily have to make the list -- just suggest that it exists and then guess where on the list the issue appears)
Sometimes that works. Sometimes it brings a little perspective.
And yes, this misallocation of time (aka "worrying about dumb stuff") is probably a top 10 productivity killer.
Update: Here's a good link (via news.yc) which offers one explanation for why people often spend the most time on the least important issues:
Parkinson shows how you can go in to the board of directors and get approval for building a multi-million or even billion dollar atomic power plant, but if you want to build a bike shed you will be tangled up in endless discussions.
Parkinson explains that this is because an atomic plant is so vast, so expensive and so complicated that people cannot grasp it, and rather than try, they fall back on the assumption that somebody else checked all the details before it got this far. Richard P. Feynmann gives a couple of interesting, and very much to the point, examples relating to Los Alamos in his books.
A bike shed on the other hand. Anyone can build one of those over a weekend, and still have time to watch the game on TV. So no matter how well prepared, no matter how reasonable you are with your proposal, somebody will seize the chance to show that he is doing his job, that he is paying attention, that he is *here*.
Update Two: Wasting time is fine, and even inane discussions of the best way to compare empty strings can be interesting. My real point is this: if we are actually trying to get something done, then we need to kill this stuff, or it will kill productivity. If, on the other hand, we're ok being in time-wasting-mode, then silly debates are probably harmless enough. I picked on the blog comments above not because there is anything wrong with them per-se (what are blogs for, if not inane debate?), but because it reminded me a lot of the silly debates that I've seen on work mailing lists (where they do cause harm).
Subscribe to:
Posts (Atom)