Debugging Hadoop 2.x on Amazon EMR

Hadoop 2.x upgrades the previous web UI with a detailed ResourceManager. Having previously browsed the simpler JobTracker UI of Hadoop 1.x using lynx on the master node, finding things on the new interface took a bit of experimentation.

Reverse Engineering Honeybadger Incoming Error Request

Honeybadger error summaryOur systems at Thinknear run on a 24/7 basis and monitoring them for errors is essential to prevent production issues that can have a negative impact on our business. One of the tools we use for error reporting is Honeybadger.

Testing @ Thinknear - Part One: Technology Choices

At Thinknear we believe automated tests are essential.

Rails Migration and Dependencies on User Defined Tables/Views

Redshift clusters need to accommodate tables and views created not only by our applications but by our operations and data science teams. It is quite common for user defined tables and views to rely on application defined tables and views, which makes migrations a challenge. In the following post, we present two SQL queries that are useful when trying to identify dependencies before running migrations.

Interviewing With Thinknear

We are hiring like crazy here at Thinknear. (Current openings on our careers page.) We're solving massive scale challenges in the hundreds of thousands of requests per second, pressing databases to the limit, and we have more data than we know what to do with. As a result, we're looking for engineers, data scientists, and managers.

Understanding the Decision to Move From AWS EMR/Hive to Redshift

At Thinknear we always want to make sure we are doing our best to use the right tool for the job. So when Redshift came out we decided to evaluate our current reporting and analytics pipeline and see if Redshift could help us improve. At the time we were using Hive/Hadoop on EMR for all our reporting and analytics purposes. We saw Redshift as a way to speed up our reporting infrastructure without completely rearchitecting and give our business team a much easier way to access the data. Given these goals we evaluated Redshift against our current Hive/Hadoop solution and found the following pros and cons.

Advanced Angular UI Router Part 1: Nested Views

At ThinkNear, we have an in-house administrative dashboard that our ad operations team uses to set up and manage ad campaigns. The dashboard is an AngularJS frontend with a Ruby on Rails backend, with the ui-router plugin for permalinks and navigation. While ngNewsletter's Diving deep into the AngularUI Router was a helpful primer, we found it didn't go deep enough.

Heroku Cost Optimization Using Generic Workers

During the early days of Thinknear, Resque was the most prevalent background job processor for our Rails applications. However, Resque was not multithread-friendly, and, as our applications grew, this put a toll on our Heroku monthly bill.

Contact us. Let's create magic together.

Our Newsletter is good. Sign up so we can deliver the goods. (Not bad, huh?)

Request a call