Monday, March 24, 2014

What is #/products/toaster - The origin of the URL Hash

This post is historical, and exaggerated to emphasize points. It is not historically accurate, and it's tone is intentionally light, so that the subject matter is easier to absorb.

In the beginning, there was the internet

Every website was a single webpage.  They were horrifying geocities nightmares, like this.


The internet was a treacherous place, and the Back button had one purpose, to escape something you never wanted to see in the first place.  But the .com boom happened, and every company wanted a webpage.  It became immediately apparent that a single page wasn't enough. So...

Routing was born

Routing initially meant that you could link to any folder in the file tree that contained your website.  If you had a Tools folder inside of the folder with your website, then you could reach it on the web by adding /Tools to the URL.


The default page configuration was born.  When you go to a folder, the website would try to open the default page (usually index.html or default.aspx) even if you didn't specify the file name in the URL.  Of course, you could browse to specific files in the file system as well.  Developers used this to manage the location of javascript and css files and the world of geocities was no more.  It was replaced by the world of PHP.  Everyone with a computer could suddenly host a website, and it was...better than it was before.  At least, the Back button had a purpose.  Companies would allow you to browse their websites now, with multiple pages, and you could go back to different parts of the website and look at neat things.  The users got used to this feature and liked using the Back button to go back if they didn't want to be somewhere (like the buy now page).

Routing grew up

Routing was all well and good, but there was this concept of clean code in programming that this file structure based routing ignored.  So we finally got around to implementing a concept that some guy created in like the 70s.  Ruby on Rails showed us how to use MVC (if you're new to web development, Model View Controller is a pattern for organizing code).


Suddenly, routing wasn't just based on your file structure.  Developers had a routing engine they could play with.  They could directly point you to different files, or even different functions via Routing.  They began to generate webpages on the fly, and the web application was born--er, well made better.  Suddenly you found cool pages like amazon.com/products/1234 and websites had thousands of pages because each page was generated as you needed it.  And the internet was a happy place.  Users could now send links to their favorite stuff to their colleagues, directly.  The internet was easier to find your way around!  The routing engine was born--er, enhanced--er, made more clear & explicit.  Whatever, the point is web programmers started using Routing Engines in their code.

Microsoft fixed the internet

Somewhere along the way, it was decided that the internet was ugly, and slow.  People would try to upload a 9000 hour YouTube video and complain that they had to wait while it uploaded.  Waiting?  That's not the way things should work in 1995.  So Microsoft started trying new things.  In 1996 they created frames (god help us) and more important, in 1998 they created a Javascript feature that we now call AJAX.


AJAX created this world where you could click a button and the webpage didn't refresh.  Instead, the data was transmitted behind the scenes, and you were able to keep doing other stuff on the website.  AJAX truly changed the internet.  The internet was starting to look like the applications users ran on their desktops, and people were happy.

But that's not good enough!

Software developers are always trying to make things better.  In the early 21st century some super smart over-eager developers decided it was time to improve the internet.  Sure, you could implement some of your website in AJAX, but why not all of it?  What if your website never had to load a new webpage at all?  Was this a good idea?  WHO CARES!


The Single Page Application was born.  They would implement an entire website in AJAX, and the user would only see one page and click around it.  To the users, the difference was negligible.

Except it was broken

Broken you say?  Nonsense!  Very smart people created this.  It works flawlessly.  People explained that the simple features they'd come to expect from websites did not exist anymore.  When they clicked the Back button, it took them out of the app entirely, instead of going back to the previous section where they were working.  When they sent the link to their friends, it brought their friends to the home page, rather than the specific section where they were working, so it was confusing.  The internet was broken!  The users were confused by these new Single Page Applications.  Developers were forced to fix it.

The URL Hash was born

Well, actually, developers had known about it for years.  In the URL, you can directly navigate around the page using the # hash tag.  In fact, it's been used on FAQs quite a bit to link to the answer that's already on the page so that you don't have to scroll down.  What was neat about this feature, is by including it in the URL, you could send a user directly to that part of the page.  The website was able to read from this tag and do stuff. COOL!


*cough* I mean Hash tag Routing was born

So a bunch of guys got together and decided to take the URL Hash, and build a Routing Engine for it!  And (after some attempts) frameworks like Sammy.js were born.

The URL hash could be used to send users to sections of the Single Page Applications, just like they were used to the rest of the URL sending them to different parts of the Website.  The Back button would take them to the previous hash, so they would be moving around the app like they expected, and not completely dropped out of it.  And they could link their friends to the neat stuff they'd seen.  And the web was happy again.

But the internet didn't end there!

OK, yes the web didn't end at URL Hash Routing Engines.  Some guys improved upon it at Google by making AngularJS and lots of other fancy stuff is continuing to make the web a better (albeit more complex) place.  This is not the end, but just know, that as long as there's capable developers out there, there will always be more stuff being built.  The SPA features have matured, and we've learned to use SPAs sparingly and for some very specific purposes (like posting on this blog).  SPAs also showed that a website should be made up of numerous web applications, instead of being just one, and so the web is still absorbing this new feature and I expect speed bumps to continue.

Friday, March 21, 2014

How to write good tests (in any language)

In the last few years, I've read many many articles on how important testing is.  In fact, I'd say that it's possibly the single most blogged about thing in programming.  All of this advice is great, but what if you don't know how to write tests in the first place.  You can be following all of the advice, and writing bad tests, and getting no results for your efforts.  In fact, I suspect that there are PMs out there who have convinced their team to give up on testing because of this.  So, let's talk about writing good tests.

Scaffolding

When it comes to testing, I personally follow the Steve Sanderson methodology: Arrange, Act Assert.  But this technical terminology just makes the comments hard to read, and in practice, they actually become Given, When, Then, so I skip the middle man.  Here's a sample of how I write test code.

//Given: Some intialization logic


//When: We call some method/function


//Then: We expect these results


//CLEANUP (optionally, unchange the things you've changed in your test)

This scaffolding guides me in building my test to ensure that I write good tests.  However, scaffolding is no good if you don't undertsand the purpose of each line, so let's go over their purpose.  For consistency, I'm going to use a single example throughout this blog post, and give you the complete code at the end.  The example is simple.  We have an array of an object called Employee.  We want to add a new employee to the array through a function we created called AddEmployee on a class we'll call EmployeeManager.

Given - Initialization logic

In the Given clause, you prepare your environment.  You set up the variables and values throughout your code to get your code to exactly the point you want to test.  This is not where you trigger the events that cause the test.  If you're testing some code that should trigger at midnight, this is not where you set the time to midnight.  This is where you create the events that you will trigger  If you're testing code that should trigger at midnight, this is where you turn on the program and set your logic to get ready to control the time of day.  In our example, this is where we initalize the EmployeeManager class and create the employee that we will add.

//Given: an EmployeeManager and a new Employee
var employeeManager = new EmployeeManager();
var employee = new Employee{ FirstName = "John", LastName = "Doe" };

As you can see, once the Given section is complete, we have everything we need to run our test.  The only thing left is to call the method that we're testing.

When - Call the method you're testing

In the When clause, you execute the method you're testing.  If it returns some value, you'd save it to a results variable here.  If you were testing some code that should trigger at midnight, this is where you set the clock to midnight.  In our case, we'll assume that our AddEmployee() function will return an EmployeeID, a number to identify our new employee in the future.  So we'll save that value to a results variable.

//When: we call AddEmployee
var result = employeeManager.AddEmployee(employee);

The when clause is always the shortest.  Good testing tests a single method at a time.  If you wish to test what happens when one method is called after another, then you run the first method in your Given section.  Only the method you're testing goes in the When clause, and it should always be just one method.  This may sound a little like I'm advocating "Unit Testing" here and while I'm a big fan of Unit Testing, that's not what this is about.  A single method can test your entire stack of code.  A single method can be an integration test, a load test, a unit test, or even an end-to-end test.  Good testing implies that you're only testing one thing at a time.  Otherwise, how will you know what's broken?

Then - Figure out what happened

So now we have to find out what happened.  All this other stuff was just executing your code.  The Then clause is where we really find out if everything is working.  Steve Sanderson calls it "Assert" for good reason.  This is where we call that famous testing function "assert" over and over again until we're sure we know what happened.  Smaller tests, like unit tests, should have very few assertions.  Larger tests, like end-to-end tests should have a much larger number of assertions.  To be honest, I've seen the Then clause scare off more developers from testing than I care to admit.  I find that the best way to approach a scary test where you're not sure what will happen is just to debug it.  Check the values, and build your then clause based on those values.  Remember, testing isn't about getting the right answer.  It's about figuring out what answer the program is really giving you.  You might find out that a massive piece of logic you thought was controlling everything was just for show and that the real code turned out to be a sneaky "return 12" for font size.  Here's where you learn the truth.  If you're testing code that should be called at midnight, in the Then clause you need to verify that your function was actually called.  In our simple employee creation case, we need to test two things.  First, we need to see if our result was an EmployeeID like we expect.  Then we need to check that our employee was actually added to the array.  Here's what it might look like.

//Then: we should see our employee is added and get an EmployeeID back
Assert.AreNotEqual(null, result);    //verify the result is not null
Assert.AreEqual(true, result > -1);  //verify the result is a non-negative number
Assert.AreEqual(true, employeeManager.Employees.Contains(employee));  //verify the employees array contains our new employee

So this example is pretty standard in my opinion.  I'm not showing you the logic of our EmployeeManager, because you don't need to know it in order to test it.  You only need to know what it outputs.  If you write good tests, then you will not only know that your code works as expected, but you'll know when something changes, even without seeing what changed.  You can do things like safely refactor thousands of lines of code in no time, or safely let other developers play in your code base sure that your tests will make sure they don't break anything.  Testing may be challenging, but spending the time doing it is worth the effort.

CLEANUP - Undo the damage you've caused

Our final step, CLEANUP, is only needed if our test does something that needs to be undone.  For our code that would need to run at midnight, it might alter some settings in the computer that we need to change back, or start some job that we may need to stop.  For completeness, let's assume that our EmployeeManager also writes to a database.  The test will not work if we run it a second time since our employee is already in the database.  We need to remove him.  Since the EmployeeManager's AddEmployee method added the employee to the database, we'll pretend that a RemoveEmployee method would do the opposite.  If such a situation exists for you in your code, I would highly recommend testing the RemoveEmployee method as well, just to make sure that this cleanup procedure works.  In a more security concious situation, I might run some more direct SQL logic to remove it from the database manually, rather than relying on another method, but this is a smiple example and I want to keep it language agnostic.

//CLEANUP
employeeManager.RemoveEmployee(employee);

As you can see, by following these simple steps we can keep our test code clean, organized, self-explanatory with thorough commenting, and even self-contained by running cleanup on our logic.  Some testing suites offer you initialization and teardown logic for handling this, but I like to keep my code together in one place.  However you do it, it's important that you test often, and test well, so let's see the whole code in one place.

//Given: an EmployeeManager and a new Employee
var employeeManager = new EmployeeManager();
var employee = new Employee{ FirstName = "John", LastName = "Doe" };
//When: we call AddEmployee
var result = employeeManager.AddEmployee(employee);
//Then: we should see our employee is added and get an EmployeeID back
Assert.AreNotEqual(null, result);    //verify the result is not null
Assert.AreEqual(true, result > -1);  //verify the result is a non-negative number
Assert.AreEqual(true, employeeManager.Employees.Contains(employee));  //verify the employees array contains our new employee
//CLEANUP
employeeManager.RemoveEmployee(employee);

With simple tests like this, we can clearly see what's being tested, and we have a scaffolding so that we don't forget what's being tested.  If you think this test looks small, you're mistaken.  I've written shorter tests than this and been happy with the result.  That said, an integration test or an end-to-end test may have a much larger quantity of assertions in the Then clause, and possibly a larger section for cleanup.

All of this is a just a guide though.  It's important that you remember the value of testing, and that you test often.  But none of that matters if you don't test well.  So do this for each and every method in your code, and you'll have a good start.  Code coverage tools will tell you that's not enough though.  You need to test each code path.  That's just a fancy word for testing your if statements and your loops.  If you have a switch statement, you need to have another test for each of the potential inputs to that switch.  The same is true for any code path like if and else.  If you can achieve 100% or nearly 100% code coverage, using a testing style like this, I guarantee you'll get phone calls like the one I got from one of my previous employers three months after I left.

"We finally deployed that program you wrote to production."
"Really? Cool."
"Yeah, it works great, thank you very much."
Production code that doesn't get rolled back 5 or 6 times?  Man, that was a good feeling.  You try it.

Sunday, March 9, 2014

How to identify a skilled programmer during an interview

How does one identify a skilled programmer?  No company that has interviewed me could tell the difference between myself and other programmers they'd interview.  The interview process is truly a game of luck in this industry--on both sides.  Both the programmer and the company are basing their actions entirely on luck.

Companies have come up with numerous methods to attempt to discern a good programmer from a bad one.  The best tricks they have include a series of math problems, algorithms, problem solving technique tests, and even obscure programming questions, some without definitive answers.  As an example: Is there an authoritative source of information on the core principles that define object oriented programming?  I've heard everywhere from 3 to 7.  In a field of research about a synthetic concept, an authoritative answer is almost impossible to obtain.

Programmers were then forced to study to the interview.  Careercup is one of my favorite sites for this.  This almost certainly detracted from them growing in knowledge about their field, but also resulted in an interesting outcome.  The people selected as the best programmers were the best interviewers with the ability to memorize large amounts of obscure bits of information.  While this skill may benefit a programmer, it certainly isn't the ideal way to determine the quality of that programmer.

Of course, when companies saw their attempts not working, they started getting more creative, which really just means they got more obscure in their questioning.  It may not be news to you, but it is unbelievably silly to me.  There are programmers who deliberately train in order to write code that compiles on a whiteboard!

Seriously, there is never a case where writing code on a whiteboard has benefited me.  Whiteboards are certainly important for explanation and diagramming code concepts, but never for actual code.  Pseudo-code at best.  And even then, an IM (at the very least) would be a faster and easier form of communication.

If you're with me this far, you must agree that the interview process for programmers is silly.  So how do we fix it?  For one thing, we need to ask questions relevant to programming.  Some companies are doing just this.  The interview process is long and tedious, but effective.  They send programming problems that are custom created to their interviewees.  Of course, there are flaws here too, some of this is preempted by (can you believe it) over the phone coding samples.  Additionally, the companies who create these interview questions are more concerned with the result than the process, which is contradictory to the way programming works.

I can provide such an example from my own experience.  Once I interviewed for a position and they told me that they wanted me to be tested on a new technology which I had never worked with before.  When I explained that it was new to me they said "Great, then we'll really get to see how you do" which is precisely the right answer.  Unfortunately, it was all talk.  The test they gave me was ultimately broken.  They hadn't even attempted to compile their own code.  In the extremely limited time frame (2 hours) I was only able to debug it enough to get it to compile correctly.  That's right, in 2 hours I figured out a brand new technology and fixed broken code.  But that's not the answer they wanted.  They wanted a complete program built around this technology.  I was surprised, but I shouldn't have been.  Good ideas, failed in the end.  They said they wanted the tech to be new to me (it was not in their job requirements at all).  This may have been true, but then they were hiring based on the resulting code wrapped in that technology.  Of course, this turned the interview game back into a game of luck.

So companies are getting the right ideas but they're not executing them well.  I mean, Amazon.com still asks questions like "estimate the number of jelly beans in this jar".  So the followup question that anyone who's reading this probably really cares about is "what are the right questions?".  I am going to list them, but I'm going to ask for patience here, because the questions don't make sense unless you understand where I'm coming from when I write them.

First, the interviewer needs to understand that programming is not like factory work.  This is a brain oriented business, not a wrench-turning mentality.  Therefore, being able to recite techniques off the top of his or her head is less beneficial than basic problem solving and research skills.  This would lead programming interviews to be more like academic or scientific interviews.

Programming isn't like academic or scientific work either.  In fact, in a world where the technology to learn is so readily available as it is in programming, and the only barrier to that knowledge is the person.  Self-taught programmers benefit from a drive that most scholarly programmers don't have.  In fact, in programming, those with degrees tend to be less skilled than those without, all else being equal.  Discovering that, most hiring managers tend to assume that they should then base a programmer's interview on their own.

Programming is most certainly not like management.  Programmers benefit from a lack of social interaction, which is of course the opposite of what most managers benefit from.  I've literally had companies turn me down as a candidate for the position when I said "I'm not great with people".  While a level of courtesy is demanded by anyone working with others, to be excellent at dealing with other people certainly does not fit my job description (and I don't want to work for a company who thinks it does).

Further, anyone who's done any research on interview techniques knows that it is a bad idea to ask simple questions.  This is more important in a field as diverse and complex as programming.  Therefore, any silly trick questions, or questions with one sentence answers are a bad idea.  Every question I've written is designed to provoke a discussion, so expect that to happen during the interview.

Programmer's are a unique type of people, and they therefore demand a unique sort of interview.  So here are a few simple questions I've come up with as a starting ground for the way that you might interview a programmer.  If this post becomes popular, you certainly will want to deviate from these questions.  But the core ideas are important, and without it, you will gain no real benefit from this post.

  1. If another developer is overworked and one of his projects is assigned to you, what do you do when you have difficulty understanding his code?
    1. This questions is a qualifier.  A junior developer will answer that he will ask the other programmer for information or if he's overzealous preclude that he would figure out all the code on his own.  These answers are distinct and do not mix in junior developers.  Make no mistake, both of these can blossom into great developers, but if they answer your question this way, they are a junior developer in mentality.
    2. A mid-to-senior developer will go along with the idea of asking for help from the previous programmer only after he has exhausted all other avenues of research (including Google), and this period of research is sure to come up in the discussion.  In fact, if you spend enough time, this question alone might let you define the difference between mid and senior as well, but this is all about a discussion, not a test.
    3. A non-developer would say something like ask the Project Manager or read through the documentation.  These sorts of things aren't even discussed by developers as they're assumed in the question and by the time the programmer has begun answering, he has likely assumed that these are useless.  Again, remember that these questions are meant to provoke a discussion.  A skilled programmer might mention briefly about documentation, but the core of the discussion would revolve around 1 or 2.  If you struggle to get him to discuss, you might want to throw out examples of where they got stuck in the code.  If you are not knowledgeable enough to do so, bring a programmer in on the interview.
  2. You are taking over a project filled with legacy code.  Where do you begin?
    1. This question is relevant to your organization.  If you have legacy code and you absolutely do not want to upgrade, then the last thing you want him to answer with is "A complete rewrite" (but many programmers will laughingly suggest this first even if they don't mean it).  Alternatively, if you're trying to learn to be a more agile organization, you might want him to consider things like re-architecting the code.
    2. Regardless of his answer here, which should help you determine how he'll fit in your organization, there are important key ideas to listen to in the discussion.  You want him to talk about refactoring, even if he doesn't know the exact word to define the concept.  You want him to talk to you about testing the code to verify it works the same as it did before (if he goes into detail about testing, he's more senior developer level than junior).  You want to hear the programmer expound his favorite software concepts.  Keep the discussion going as long as you can and you can really grasp stuff here.  It is doubtful that a programmer would run out of things to say here, so if your interviewee does, then there's reason to suspect he may not be a real programmer.
  3. Describe a large scale project you worked on by yourself.
    1. Not all programmers have worked on large scale projects on their own, but the good ones have.  I've got at least half a dozen websites, from e-commerce to D&D character sheet web-apps that I've built on my own, and that's just in my free time.  Don't set an explicit standard (and definitely don't base it on me), but the more large scale projects they've built on their own, the more capable they are as a programmer.
    2. While any good programmer is going to be very proud of his or her work on his own, this will begin the process of seeing how they work with other programmers.  Sometimes it's purely a matter of the jobs they've had that define whether they've worked with other programmers enough to have had a good team.  But you can start to sense whether they have an intimate dislike of working with others from this discussion.  This is the first question on this list that really will show you what type of programmer they are, instead of simply if they're a programmer at all.
    3. Tell the interviewee to use a whiteboard.  From his use of it you can also learn about how he explains things.  Bring another developer in for this question.  See if the other developer can understand the concepts after the discussion.  A senior developer should be able to effectively articulate the architecture to the other developer, and it should be interesting enough to get that developer involved and asking questions.  A junior developer will struggle more here, and he may ultimately be describing a mess of spaghetti code, but it's all his, and the developer in the room should be able to help you define the developer's skill level.
    4. This question does not imply it's opposite.  It is generally a bad idea to ask about a large scale project that programmers have worked with a team on because their role is dependent largely on the organization, and not themselves.  They may not have been involved in the architectural process, or even be permitted to know what their team members were doing (security and all).  And of course, they may not be permitted to talk about that project at all (NDA).
  4. When storing user information in a database, programmers encrypt or hash passwords.  Why?
    1. This is another good qualifier.  It's not about knowing a particular concept.  If you have to explain to the interviewee about databases and encryption, you might have an issue, but most of the concepts, hell even hashsets can be completely alien to a highly skilled developer.  This question instead asks the more broad question of  "Why"
    2. The answer of course is security, but don't let the discussion end there.  After all, the data is stored on your servers, how could there be concerns about the data being accessed from the outside.  Concepts like SQL Injection and even SSL & Session storage can easily mix into this discussion.  We're not eliminating programmers based on a specific set of arbitrary terms here, but based on concepts and large-scale understanding about security over the internet.
  5. What language are you most confident in?
    1. I am honestly surprised to tell you I've never once been asked this question.  Not one company that has hired me cared if I was most confident in their preferred language.  And, if my company told me they were going to ask this question in the interview, I would advise them against it.  Because they aren't in the mindset that I've emphasized throughout this blog post.
    2. When you think about programmers like researchers, you can think about this question differently.  It's not about the answer, so much as how quickly they answer.  A programmer who has not explored a lot of languages thoroughly will answer immediately, and a non-programmer will answer immediately with the language listed on your job requirements.  But a programmer with a breadth of skill across multiple languages, someone who dedicates his life to the pursuit of programming knowledge will pause, and list a few of his favorites.
    3. The languages that are mentioned matter too.  If he is a web-developer he will list among his top 3 languages JavaScript and some back-end language like C# or PHP.  If he is a windows developer, he'd better mention .NET.  If he's a linux guy he'll likely mention Perl.
    4. This question should spark others, like "How did you get into that one?"  And "What was your first language?"  Steer this discussion correctly and you can get a programmer to describe his entire career.
Well, it's just a few, but it's a start.  I hope that you benefit from these questions and that the programming community as a whole can start better hiring practices, which create better code in organizations, which creates better programmers competing for those jobs and helps the community grow stronger.

One last word of warning.  I mention bringing programmers in on the discussion, but for many organizations this is a double edged sword.  Some organizations are acutely unaware that they have hired incompetent programmers.  What's worse, some programmer's have taken to protecting their jobs by ensuring the organization hires people less knowledgeable than themselves.  If you bring a programmer in on an interview, there are a few key things to look for to ensure that he is a participant, and not a barrier.  First, if he bogs your interviewee down in the details of code implementation, he's not being helpful.  If he starts trying to ask questions on his own that are outside of the existing discussion, it's important to ensure that he's doing so in good faith.  Finally, if your interviewee fits your qualifiers as a skilled senior developer and the developer you involved in the discussion suggests not hiring him, it may be a personality conflict, or it may be something more.  Put simply, more communication is better than less here.

Alright fine, one last one.  This is a gift.  Any modern day programmer should know the answer to this question.  If they don't, they're not a real programmer.  If you're reading this post in the distant future, this question may no longer be relevant, but at the time of posting it is extremely relevant.