Senior Big Data Administrator

May 11, 2018
Pasadena, California

Senior Big Data Administrator
Long Term Contract
Pasadena, CA

Job Description

Client, a leading provider of digital and mobile advertising technology, seeks a Senior Data Systems Engineer responsible for the performance and uptime of various Client Big data and related database systems/services. Client serves more than 55 billion requests per day through thousands of servers across a worldwide datacenter footprint, backed by complex interconnected data systems. Experience at large scale is highly desirable. 

You will be responsible for maintaining and improving service uptime and scaling our systems for continued rapid growth. This role will handle planned and unplanned maintenance events, participate in on-call, and support a consistent software release process. The ideal candidate has experience with large-scale management of thousands of physical servers and the operation of distributed systems with hundreds of nodes. Excellent communication skills are required in order to successfully interact with the rest of Client Engineering. This is a “hands-on” role and competence is expected in Linux/UNIX fundamentals and command-line.

Developing and supporting our infrastructure presents many interesting technical challenges. We especially desire candidates with a passion for open-source software and an interest in the latest system architecture trends.

Responsibilities include:
  • Design, implement, and support a high-performance, highly-available infrastructure.
  • Improve the efficiency and flexibility of our datacenters.
  • Build and maintain models for growth and capacity planning.
  • Tune large-scale data clusters for optimal performance and efficiency.
  • 24/7/365 on-call rotation.
  • Own the day-to-day health, uptime, monitoring, and reliability of all data platforms and database systems.
  • Work closely with project management and engineering peers to develop innovative technical tools and solutions.
  • Identify tactical issues and react to emerging areas of concern.
  • Adhere to a DevOps philosophy by evangelizing communication, collaboration, and integration with software development teams.
  • Think long-term and be unsatisfied with band-aids.
  • Identify unnecessary complexity and remove it.

  • Overall 10+ years of technology experience with at least 3 to 4 years experience in the Big data area, specifically, Cloudera (or equivalent) Administration at very large scale.
  • Experience installing and managing one or more of the following: Big Data and related services, RDBMS platforms (e.g. MariaDB / MySQL, PostgreSQL, Vertica), distributed data systems (e.g. Riak, Druid, Kafka)
  • Solid knowledge of Linux/UNIX command-line tools.
  • Experience with installing and managing large scale Cloudera/Hadoop clusters on one of public cloud computing platforms (e.g EMR on AWS or equivalent)
  • Demonstrated experience in network and large scale UNIX system troubleshooting and maintenance practices.
  • Capability to script and automate solutions with strong competence in at least one programming language.
  • Firm grasp of storage protocols and filesystems.
  • Deep experience installing and managing Hadoop clusters and related services.
  • Ability to do upgrades, performance tuning, cluster tuning, capacity analysis on Hadoop clusters
  • Implementation and management of monitoring and metrics tools (e.g. Nagios, Graphite, Grafana, SumoLogic).
  • Excellent organizational skills and the ability to work in a fast-paced and hectic work environment.
  • Capable of technical deep-dives into code, networking, systems, and storage with SRE and software engineering.
  • Knowledge and interest in the latest system architecture trends.
  • Ability to learn and understand new systems.
  • Ability to communicate effectively and write accurate, clear documentation.
  • Humility and integrity.

Nice to have:
  • Experience with Cloudera.
  • Hadoop-based computational technology like YARN and Impala.
  • Other operational data technologies like HBase, Spark, Redis and RabbitMQ.
  • Running and troubleshooting Erlang, Java or Python applications.
  • Hardware configurations for data systems.
  • Analytical data platforms like Vertica and MicroStrategy.
  • Configuration management systems like Salt Stack and Chef.
  • Container technology like Docker and Kubernetes
  • Agile development practices.