Gmail import broke for about 2 hours today.
Today, our Gmail OAuth2 integration broke shortly after 2pm PDT. We deployed a fix by 4:10pm. Here’s what happened.
Firstly, here’s a short background. After a user grants consent to their contacts, all OAuth2 providers require us to send credentials to identify our server in order to obtain an access_token. The access_token is used for subsequent calls to the providers’ contacts APIs. The OAuth2 standard permits sending these credentials as parameters in the body of the request or as a standard Basic Auth header. It turns out that most providers support both ways and don’t care which way we send the authorization. But there are a few that are picky: Yahoo wants only Basic Authorization in the header but AOL and Facebook want the parameters sent with the body of the request.
We manage many OAuth2 providers through the same library of code and so from an engineering standpoint, the more general we can be with our treatment, the better. So we originally sent it both ways. And during our development and testing, we found that this worked quite nicely and produced the simplest code.
Well, that was working just fine until 2pm today when Gmail unexpectedly started to return this error.
OAuth 2 parameters can only have a single value: client_secret
Our monitoring systems alerted us to the issue. I was able to reproduce it locally and confirm my suspicion: suddenly Google’s OAuth2 service decided that it is OK to send the data as a Basic Auth header or as parameters in the body of the request, but not both ways. The fix was deployed shortly after.
This was an embarrassing bug, but we caught it quickly and updated our system immediately to resolve it. And I want to be open and clear about why it happened and what we did to resolve it.
