Peter Norvig on Google's approach to hiring: “You can see how hire-above-the-min leads to a precipitous drop in skill level; one we've been able to avoid.”
I don't think I've ever seen a quantitative comparison of hiring strategies before. Of course it's a lot easier if it's possible to quantify a candidate's quality.
(Gavin: “With all the stats available, it would be cool if google could actually point to the person/hire who brought the overall intelligence down.”)
I like to interview programmer candidates. Maybe it's like having a standard geeky conversation except that the context demands that both interviewer and interviewee have extra special focus on the conversation and try harder than usual to be smart (except that there's also pressure not to be an arrogant asshole—take that, nerd!). Ever since Will gave me a neat programming exercise as part of the hiring process at Intell/Agent, I've been a big fan of using take-home programming assignments as the best way to get a feel for the person's grasp of programming. The exercise I use is actually an exact copy of the one Will gave me in 1995 or so, including the Lisp data files. For the past few years I've mostly been involved in hiring C++ programmers, so I throw in some C++ code to read the data.
The exercise takes a good programmer somewhere around 5 or 6 hours, which I think is substantial enough to get a good feel for the candidate's design skills as well as their raw coding ability. The instructions for the assignment and the data include some minor inconsistencies and areas of vagueness which were not originally intended, but which have explicitly not been fixed, so I get to see how people deal with that too (emailing me with questions is a great response, as just one example). HR departments have, at times, not wanted to ask candidates to spend so much effort and time on the exercise, which I think is pretty misguided. It's too hard to judge someone's coding ability without seeing significant chunks of code, and it's not common for people to have a significant chunk of code that isn't legally encumbered in some way that prevents them from showing it to other people (working on an open source project might be one way to solve this problem, but I wonder if there are other good solutions, too).
I used to ask candidates what they don't like about their favorite programming languages, but too often people didn't have any good answers. Which kind of blows my mind, and depressed me enough to create a pain barrier to even asking. Don't forget, every language sucks, man, and if you don't realize that you better start wondering why not. As Steve Yegge says,
The very best Ruby programmers, as with any language, are the ones who can think outside Ruby; i.e., they know its limitations as well as its strengths. It’s extremely uncommon for average programmers or language novices to be able to speak intelligently about their favorite language’s weaknesses, because the language books and tutorials rarely focus on the weaknesses.
One of these days I'll finish my “20 True Reasons Lisp Sucks” post.
Posted by jjwiseman at April 03, 2006 12:35 PMHere's a couple of reasons: inconsistent design (fmakunbound vs. unintern) and defining all the exports for a package by hand. The first is completely unintuitive and took me a while to hunt down fmakunbound. It's also tricky with methods, and it could use an 'e' too. The second is just a PITA to keep up to date and is rather far away from the actual definitions.
And Ruby could use some strong typing to dispatch methods/messages. Something like "def method(int n)" would be better than what it has where you have to test the type with an IF-ELSIF-ELSE block. And there's another issue, "elsif". WTF?
I am at the point where I'm starting to have delusions of an ideal programming language...
Posted by: Nolan Eakins on April 3, 2006 03:52 PMFun graph but business decisions are never one-dimensional. Doesn't take a Google employee to figure that out...
Posted by: geoff on April 3, 2006 07:00 PMRe: comment above
Strong typing sucks. It sucks less in interpreted languages (which benefit from not having to figure out the type at runtime) but seems really silly in a compiled language where the computer has all the time in the world to figure out the difference between an int and a string. Languages seem to get this backwards I think.
Chris:
That's one of my delusions at the moment: making the strong typing of C++ optional. I think one byproduct of that would be the elimination of templates, or at least they'd be useless in most cases. I wonder why the committee didn't think about that one?
Personally I think Common Lisp has the right mix of strong and dynamic (weak?) typing. Though I still fail to understand why every function is not also a method. It would be great if was CL specified so I could redefine #'+ to work with my own types. I suppose that could be done though.
Posted by: Nolan Eakins on April 3, 2006 11:03 PMSo what was this programming exercise that Will gave you? I checked his site but didn't find anything obvious.
Posted by: Jason Cornez on April 4, 2006 12:59 AMWe did a variation of the takehome assignment at Vyatta. Following our public launch, we had a bunch of people asking us about joining the company. To help screen the serious from the not-so-serious, our standard response (here: http://www.vyatta.com/careers/) is to ask people to fix a bug. Being open source is great in this respect. Note that we don't specify how big of a bug you have to fix, just that you should fix one.
There are a number of benefits to this approach, IMO:
1. It's actually beyond a programming exercise. It's a total system exercise so we can see if they are capable of navigating the bug database, checking out code, fixing something, and then generating a patch, etc.
2. It's an exercise on the actual codebase they will be maintaining, so the candidate gets some familiarity with what they will be working on (good and bad). We get to see if they understand the domain (if you want to work with us, show us that you know your way around routing code).
3. Either way (hire or no hire), the project gets a bug fixed or more people involved with the codebase, which is a good thing for an open source project.
We have had a couple people come to our attention this way, and it really cuts down on the fluff resumes where people are just job-hopping to try to work on something "cool" for a year, then move on to something else with no actual contribution.
BTW, if anybody is wanting to write code for a network appliance/router, we're still looking for good engineers. I'd love to get some smart Lisp programmers in here, even if you end up having to work in C/C++.
Posted by: Dave Roberts on April 4, 2006 11:20 AMEleven things about Common Lisp that suck:
1. Pathnames.
2. Behaviors unspecified by the standard, particularly in safe code.
3. Inconsistent naming. Why is it UNION and NUNION, but REMOVE and DELETE? And why isn't SORT called NSORT?
4. Not enough name spaces (take that, Scheme)! Special variables should have their own.
5. The class precedence algorithm should be monotonic, as in Dylan.
6. The inflamed scars where they amputated the MOP.
7. More builtin types should be extensible (NUMBER, REAL, SEQUENCE, etc.), with associated builtin functions becoming user-extensible generic functions.
8. The flat package namespace.
9. Arrays specialized to type NIL.
10. (type-of 10) can't be FIXNUM.
11. Lack of operations on environments.
Nolan:
I agree with you on C++ - but, I think that C++ should be scrapped all together. It does way too much too poorly, and encourages other languages like C# to do the same.
Lisp is pretty much the perfect language for me. I can't say that I love everything about it, and I also can not say that it sucks.
Posted by: Chris Parker on April 4, 2006 01:19 PMSo how on earth does one get an actually decent candidate to walk in the door? We've been using a very simple programming exercise filter for a long time (if it takes as long as five minutes, you're in trouble), and yet we still end up doing many interviews in which people struggle and struggle with what should be extremely basic programming problems.
Of course, given the number of "do my intro CS homework for me" requests on rentacoder and other sites of that ilk, this may not be that surprising.
Posted by: Daniel Martin on April 4, 2006 02:05 PM@Nolan:
I'd say in that example it's not inconsistent per se, because it's consistent with `makunbound'. At most its a subtly of arguable utility.
@Paul:
> 4. Not enough name spaces (take that, Scheme)! Special variables should have their own.
Intrigued. Can you give an example of what you envision?
I like your 6 and 7. From a "modern" POV CL doesn't seem so large any more.
> 7. More builtin types should be extensible.
Yes.
For my own top personal pet peeve, I pick `loop'. I am far more comfortable with SERIES.
Posted by: Ben on April 5, 2006 12:59 AMPaul Dietz aside, I think the people posting their "pet peeve"s about Lisp are completely missing the point. (They're certainly missing my point; whether they're missing John's original one -- note "True" in "20 True reasons Lisp sucks" is another matter).
loop cannot possibly be a true reason lisp sucks, because no-one is forcing anyone else to use it. Sure, it exists in the language, but its existence doesn't stop people from writing loops in other ways; to the user, even the loop-hating user, there is pretty close to zero cost in loop's existence. To a lesser extent, I disagree with the people who decry the inconsistent design (fmakunbound vs unintern, sort/nsort/remove/delete and so on) because there is a small, fixed cost with having to learn the shared vocabulary.
Similarly, things which are trivial to implement (which have been mentioned in other places: e.g. DOVECTOR, DOSEQUENCE) have a small cost from not being in the language: the cost of their nonexistence is bounded by the cost of their implementation, plus a little bit for bookkeeping.
So, for me, anyway, true ways in which lisp sucks are either things that are in the language which are not easily workaroundable (Paul's 1, 5, and 9) and things missing from the language that are not easily implementable (2, 4, 6, 11). I'd add to those things such as poor support for partial evaluation and compiler hooks; it's quite hard to get the equivalent performance characteristics of specialized operations as one can in C++ using templates, for instance.
I don't understand why Paul dislikes TYPE-OF's behaviour so much; yes, it might be unintuitive for a brief moment that (type-of 10) isn't fixnum, but then again why should it be?
As for the generic operations and extensible built-in types, I am currently unconvinced. Yes, it is clear that this is something missing from the language that is not easily implemented or workaroundable; however, I haven't seen even a sketch of a (non-portable) implementation, or a terribly compelling use case: certainly none which addresses either the cost of implmementing this functionality or the potential cost to other pieces of code within the same image; if I define a new number type, does all arithmetic everywhere need an extra dispatch?
Posted by: Christophe Rhodes on April 5, 2006 01:44 AMIntrigued. Can you give an example of what you envision?
I view the presence of naming conventions (like the *...* convention for special variable names) as a symptom that another name space is needed.
So, I imagine something analogous to the FUNCTION special operator, say a new special operator SPECIAL. This would have some kind of reader macro abbreviation, perhaps #!. Or, one could just use (symbol-value '...), appropriately reader-macro-ized.
One way this would be awkward would be in forming bindings; you couldn't use ordinary LET (and the other variable binding forms). So, that would either be disallowed, or LET (and some of the others) would be extended to allow SETF-able forms.
I'd also like a separate name space for named constants (the +...+ naming convention is another symptom).
Christophe:
if I define a new number type, does all arithmetic everywhere need an extra dispatch?
Presumably, the code that implements generic arithmetic (or sequence operators, etc.) would have to do dispatch anyway. The user-provided types could be checked for only after all the standard types, so there would be a performance cost only for cases that would have been erroneous anyway.
Code that had type declarations to standardized types would not need dispatch in either case.
Posted by: Paul Dietz on April 5, 2006 02:38 PMPaul:
Code that had type declarations to standardized types would not need dispatch in either case.
This sounds like it will either go horribly wrong or leave some people dissatisfied. Judging by this, there remain some types which are unextensible: the types for which extra dispatch is not necessary.
To take a concrete example, let's say that in (defun foo (x) (declare (type list x)) (find 1 x)) there is no dispatch, because list is a standardized type. This implies that the user cannot define a new list subtype: if he can, then there must be dispatch.
But then along comes someone who says that his doubly-linked list should really be a subclass of LIST, and it's not fair. So maybe LIST should be subclassable?
A similar argument comes when someone decides that his #<EXCELLENT-CLASS Z32-POLYNOMIAL> is a subclass of (unsigned-byte 32) and that clearly not only should (unsigned-byte 32) be a class but it should also be subclassable. The howls of protest from those using Lisp for cryptography would be heard for miles.
So, where do you draw the line? And how will you justify that line to other users of the language?
Posted by: Christophe Rhodes on April 6, 2006 06:51 AMChristophe: I was envisioning allowing extension of the 'abstract' builtin classes, like NUMBER or SEQUENCE, not those that correspond to specific implementation types. Any case of your objection could be addressed by interpolating additional such superclasses into the hierarchy.
Posted by: Paul Dietz on April 6, 2006 08:09 AMYa lisp programmers need penis enlargement pills? Hurry! Top discounts coming up this summer! And please remeber nerds and google-wannabes will always look cooler with bigger dicks! :-)
Never, ever... ever click here:
http://www.penisenlargement.com
PS: If you wanna know I used to program LISP but now switched to C++ because my software is not half-assed interpreted very-late-binding crap, but something that runs fast! And nowwriting in C++ I can afford getting a bigger dick - and you should too!!!! :-)
Posted by: Stan Kubik on April 7, 2006 01:14 AMJudging by Peter Norvig's chart (http://tinyurl.com/mhwne) it surely can't be very difficult to be above average: no lables on axes, no units, no title ... ;-)
Posted by: Adam on April 7, 2006 04:32 AMI don't really care what this site is about, i just wanted to post that I had a phone interview with Google 2 weeks ago, then went on a 1-week vacation with my beautiful girlfriend in the Bahamas.
I got back and found and email and a voice mail that said (quoting):
Subject: I've Got Some Good News...
Hi -myname-,
When you have a moment, please give me a call. I can be reached at xxx.yyy.zzzz.
Thanks,
nameless google recruiter
The voicemail reinforced this "good news." So after calling back twice - once on monday, once on tuesday after I had heard the initial report of "good news" - I get a call from the recruiter with the old "this is the hardest part of my job..."
That's just f@*$ing rude. If you're not gonna hire me, just tell me. How is that "good news?"
Assfaces.
Posted by: WTF? on March 22, 2007 09:52 AM