Lead Data Engineer

CAS uses intuitive technology, unparalleled scientific content and unmatched human expertise to help companies create groundbreaking innovations that benefit the world. As the scientific information solutions division of the American Chemical Society, CAS manages the largest curated reservoir of scientific knowledge, and for 116 years, has helped innovators mine, assess and apply that information to keep businesses thriving. The CAS team is global, diverse, endlessly curious and strives to make scientific insights accessible to innovators worldwide.

CAS is currently seeking a Lead Data Engineer.  This position will be located in our headquarters in Columbus, Ohio.

Description

Data engineers are responsible for supporting ingestion and transformation pipelines that handle data for analytical or operational uses across broad business areas and enterprise data domains. The data engineer often works as a dedicated member of support teams, focused on providing production stability for data processing workflows that will be used by analytics groups and data scientists who are interrogating information for predictive analytics, machine learning and data mining purposes.

Duties

• Designs and develops complex and large scale data structures and pipelines for ingestion and transformation of data for analytical or operational uses across a wide variety of business needs and enterprise data domains.
• Ensures production stability for data processing workflows used by analytics groups and data scientists who are interrogating information for predictive analytics, machine learning and data mining purposes.
• Writes complex ETL (Extract / Transform / Load) processes, designs database systems and develops tools for real-time and offline analytic processing.
• Develops frameworks, standards and reference material for architecture and associated products.
• Collaborates with data science team to transform data and integrate algorithms and models into production systems.
• Uses in-depth knowledge of Hadoop architecture, HDFS commands and experience designing and optimizing queries to build scalable, modular and efficient data pipelines.
• Uses advanced programming skills in Python, Java or any of the major languages to build robust data pipelines and dynamic systems.
• Integrates data from a variety of sources, assuring that they adhere to data quality and accessibility standards.
• Experiments with available tools and advises on new tools in order to determine optimal solutions given the requirements dictated by the model/use case
• Behaves as mentor to less senior team members to provide technical advice.
• Applies knowledge of company systems and products to consult and advise on additional efforts across multiple domains spanning broader enterprise

Qualifications

• Master’s degree (preferred) or Bachelor’s degree in Computer Science or similar discipline with 8+ years of software engineering experience
• 5+ years’ experience in data integration, ETL, data warehouses, data profiling, data governance understanding
• 3 years of experience with DevOps, Continuous Integration and Continuous Delivery and knowledge of some of these various technologies (Maven, Jenkins, GitHub/GitLab, CodeRepository, CodePipeline)
• 3 years of experience working as MLOps Engineer
• Experience working with Data Science, Machine Learning, Spark (Pyspark), and Python
• Hands on experience with Big Data, Hive, cloud platform, microservices, and Hadoop
• Hands on experience in big data environments such as Cloudera or Hortonworks
• 3+ years’ experience with programming in Scala, Spark, Python, JavaScript, and Java, as well as strong Unix shell skills
• Minimum 2 years’ experience in Cloud providers like AWS (preferable)
• Experience building Data Ingestion on the cloud (using tools like Glue)
• Solid understanding of Big Data NoSQL databases/technologies (MarkLogic, Hbase, Hive, Spark, MongoDB)
• Strong written and verbal communication skills

Desired, but not required:

• Knowledge and experience in chemistry, drug discovery/development, or medical related industry

CAS offers a competitive salary and comprehensive benefits package, including a generous vacation plan, medical, dental, vision insurance plans, and employee savings and retirement plans.  Candidates for this position must be authorized to work in the United States and not require work authorization sponsorship by our company for this position now or in the future.  EEO/Minority/Female/Disabled/Veteran

The work schedule at CAS, consists of working onsite at least three days per week. While always welcome to work in the office, employees may work the other two days of the week from their home. New employees are immediately eligible for this work arrangement. CAS offers relocation assistance, if applicable.