Gartner Blog Network

Wall Street is Hiring for Big Data

by Svetlana Sicular  |  April 28, 2014  |  1 Comment

Big data finally reached Wall Street. Not for small science experiments, but seriously, in grand style, with exorbitant salaries, for production and DevOps. A couple of years ago, Silicon Valley companies were bragging to each other about hiring Wall Street quants for data scientists. Now Wall Street is happy to grab  — whom? No, not data scientists (New York has plenty of its own quants) — Data Architects, the breed who can come up with new architecture to combine structured and unstructured data where architecture for one use case usually (still) does not apply to another. returned just 34 data scientist jobs vs. 645 data architects in the New York area tonight. Even my search of “data architect + Hadoop” returned twice as many data architecture jobs compared to data scientists. Sorry, data scientists: data architects are sexy!  My client inquiries shifted lately to no-nonsense big data architecture, management and real-time use cases. Big data vendors hinting left and right about “Wall Street customers in alpha,” e.g. newly signed contracts.  My friend, a Wall Street recruiter, had to cut his vacation short – Wall Street is in a hurry to get data architects right now.  And also, performance engineers, and developers, and administrators. And did I say, DevOps? Yes, for agility, my friends.

This hiring means that Wall Street is ready to use big data strategically.  The picture below shows typical stages of big data adoption described in my research note The Road Map for Successful Big Data Adoption.  The red dot on this picture — a stabilized infrastructure — is the most prominent milestone. After the infrastructure has been built, a capability to derive value from big data technologies leaps to a new level.  Nonbelievers turn into believers.


At the red dot, big data becomes the new normal. It eventually gets related to other information sources. (When using the term “big data” analytics and information management professionals (across the Globe!) first say that they don’t like it.) At the red dot, companies substantially expand the number of nodes or totally rebuild the earlier small layouts. 

A widespread myth is that Hadoop is inexpensive to implement. Really? With Wall Street salaries? An initial implementation is usually more expensive than expected. It involves a lot of unanticipated technical and nontechnical difficulties. By the way, another myth is that big data infrastructure usually takes advantage of commodity hardware. Maybe, but not on Wall Street. Enterprises buy high-end hardware.

I gave my friend a Wall Street recruiter a t-shirt stating “My data is bigger than yours”— he wears it to work (on Fridays). I should make one for myself, with the text Data is what used to be big data. Wall Street is getting there.


Follow Svetlana on Twitter @Sve_Sic

Additional Resources

Complete Your Data and Analytics Strategy With a Clear Value Proposition

As a data and analytics leader, one of the most important things to articulate in your strategy is the value proposition. Learn how to create a modern, actionable D&A strategy that creates common ground amount stakeholders.

Read Free Gartner Research

Category: data-scientist  big-data-market  crossing-the-chasm  data  data-and-analytics-strategies  hadoop  market-analysis  

Tags: big-data  big-data-adoption  crossing-the-chasm  data  data-paprazzi  end-users  hadoop  hiring  market-analysis  

Svetlana Sicular
Research VP
6 years at Gartner
23 years IT industry

Svetlana Sicular is passionate about bringing analytics to domain experts and helping organizations successfully compete by applying their business acumen in analytics and data science. She is convinced that domain expertise and high-value data are the greatest assets that companies should monetize in new analytics applications. Read Full Bio

Thoughts on Wall Street is Hiring for Big Data

  1. HS says:

    Svetlana, very interesting. Then we should not be surprised that Architect jobs are also in abundance in Big Data freelance marketplaces like Experfy (, which recently launched. Also, see this very interesting Mashable article on Experfy:

Comments are closed

Comments or opinions expressed on this blog are those of the individual contributors only, and do not necessarily represent the views of Gartner, Inc. or its management. Readers may copy and redistribute blog postings on other blogs, or otherwise for private, non-commercial or journalistic purposes, with attribution to Gartner. This content may not be used for any other purposes in any other formats or media. The content on this blog is provided on an "as-is" basis. Gartner shall not be liable for any damages whatsoever arising out of the content or use of this blog.