Today I Learned: BigQuery Scheduled Queries Don’t Fail on Missing Columns

I’m continuing to enjoy the relaxed abilities that BigQuery brings compared to other common databases. However, yesterday I was bitten by how relaxed it can be.

I had been refactoring an existing scheduled query & as part of it, I had been (manually) converting many of the reserved words in ALL CAPS. Well, I had used a search & replace at some point that accidentally changed the gender column to the gENDer column in the query’s text. I always make sure to run these queries manually to verify that they work…and no error had come back. I save the changes, updated the schedule, & went home.

When I got to work this morning, there was an email thread from someone who noticed that the gender column had changed its name & that all the values were NULL. My reaction: “Huh? How can that be?” I was disappointed that I’d overlooked the accidental change of the column name reference, but there hadn’t been any error! However, it was the all NULL values that really made my head swim…!

I’m not sure how/why BigQuery is so laissez-faire about this (yet still can be case-sensitive about the column name!) but I sure learned how important it is to be consistent with column names!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.