Gartner Blog Network


Wall Street is Hiring for Big Data

by Svetlana Sicular  |  April 28, 2014  |  1 Comment

Big data finally reached Wall Street. Not for small science experiments, but seriously, in grand style, with exorbitant salaries, for production and DevOps. A couple of years ago, Silicon Valley companies were bragging to each other about hiring Wall Street quants for data scientists. Now Wall Street is happy to grab  — whom? No, not data scientists (New York has plenty of its own quants) — Data Architects, the breed who can come up with new architecture to combine structured and unstructured data where architecture for one use case usually (still) does not apply to another.

Dice.com returned just 34 data scientist jobs vs. 645 data architects in the New York area tonight. Even my search of “data architect + Hadoop” returned twice as many data architecture jobs compared to data scientists. Sorry, data scientists: data architects are sexy!  My client inquiries shifted lately to no-nonsense big data architecture, management and real-time use cases. Big data vendors hinting left and right about “Wall Street customers in alpha,” e.g. newly signed contracts.  My friend, a Wall Street recruiter, had to cut his vacation short – Wall Street is in a hurry to get data architects right now.  And also, performance engineers, and developers, and administrators. And did I say, DevOps? Yes, for agility, my friends.

This hiring means that Wall Street is ready to use big data strategically.  The picture below shows typical stages of big data adoption described in my research note The Road Map for Successful Big Data Adoption.  The red dot on this picture — a stabilized infrastructure — is the most prominent milestone. After the infrastructure has been built, a capability to derive value from big data technologies leaps to a new level.  Nonbelievers turn into believers.

BigDataAdoption

At the red dot, big data becomes the new normal. It eventually gets related to other information sources. (When using the term “big data” analytics and information management professionals (across the Globe!) first say that they don’t like it.) At the red dot, companies substantially expand the number of nodes or totally rebuild the earlier small layouts. 

A widespread myth is that Hadoop is inexpensive to implement. Really? With Wall Street salaries? An initial implementation is usually more expensive than expected. It involves a lot of unanticipated technical and nontechnical difficulties. By the way, another myth is that big data infrastructure usually takes advantage of commodity hardware. Maybe, but not on Wall Street. Enterprises buy high-end hardware.

I gave my friend a Wall Street recruiter a t-shirt stating “My data is bigger than yours”— he wears it to work (on Fridays). I should make one for myself, with the text Data is what used to be big data. Wall Street is getting there.

 

Follow Svetlana on Twitter @Sve_Sic

Category: data-scientist  big-data  big-data-market  crossing-the-chasm  data  hadoop  market-analysis  

Tags: big-data  big-data-adoption  crossing-the-chasm  data  data-paprazzi  end-users  hadoop  hiring  market-analysis  

Svetlana Sicular
Research Director
3 years at Gartner
21 years IT industry

Svetlana Sicular has a uniquely combined experience of Fortune 500 IT and business leadership, product management at world-class software vendors, and Big Four consulting. She primarily handles inquiries in the areas of data management strategy, ...Read Full Bio


Thoughts on Wall Street is Hiring for Big Data


  1. HS says:

    Svetlana, very interesting. Then we should not be surprised that Architect jobs are also in abundance in Big Data freelance marketplaces like Experfy (http://www.experfy.com), which recently launched. Also, see this very interesting Mashable article on Experfy: http://mashable.com/2014/04/26/experfy/



Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.