Data Engineer II
JOB SUMMARY The Data Engineer II will independently support about our data operations functions, pipelines and architectures. Through their understanding our data content and information architecture, they will design new and make changes to existing data workflows. They will design ETL, machine learning and artificial intelligence algorithms, data flows and monitor/tune our data environment for optimal performance. They will also participate in demonstrating value in our data assets by collaborating with analysts, architects and other stakeholders to develop business cases, plans and cost estimates needed to demonstrate value of our initiatives. QUALIFICATIONS To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. Education Level Requirements:
Bachelor's Degree Computer Science, Statistics, Applied Mathematics, Data Management, or Information Science or related quantitative subject OR- Master's Degree Computer Science, Statistics, Applied Mathematics, Data Management, or Information Science or related quantitative subject Work Experience Less than 2 years Data Engineer I experience OR - 3 - 5 years Business, information, or technology experience General Employee Knowledge, Skills, and Abilities o Ability to establish effective working relationships among team members and participate in solving problems and making decisions o Ability to present and express ideas and information clearly and concisely in a manner appropriate to the audience, whether oral or written o Ability to actively listen to what others are saying to achieve understanding, sharing information with others and facilitating the open exchange of ideas and information o Ability to establish courses of action for self to accomplish specific goals, develop and use tracking systems for monitoring own work progress, and effectively use resources such as time and information o Ability to make right decisions based on perceptive and analytical processes, practicing good judgment in gray areas o Ability to design, manage, and implement simple data flows and information architectures for financial institutions o Knowledge of analytics scripting tools using languages such as R, Python/PySpark, Java, and C++. o Skill with big data platforms like Hadoop and Azure Synapse o Ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management. The ability to work with both IT and business in integrating analytics and data science output into business processes and workflows o Skill with popular database programming languages including T-SQL for relational databases and certifications on upcoming NoSQL/Hadoop oriented databases like MongoDB, Cassandra, others for nonrelational databases o Ability to work with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional data integration technologies using APIs, SQL Server SSIS, etc. o Ability to work with SQL on Hadoop tools and technologies including HIVE, Impala, Presto, Hortonworks Data Flow (HDF), Dremio, Informatica, Talend and others o Ability to work with and optimizing existing ETL processes and data integration and data preparation flows and helping to move them in production o Ability to work with both message queuing technologies such as Kafka, JMS, Azure Service Bus, and stream data integration technologies such as Apache Nifi, Apache Beam, Apache Kafka Streams, Amazon Kinesis and stream analytics technologies such as Apache Kafka KSQL, Spark Streaming, Samza and others o Knowledge of popular data discovery, analytics and BI software tools like Microsoft PowerBI for semantic-layer-based data discovery o Ability to working with data science teams in refining and optimizing data science and machine learning models and algorithms o Knowledge of data science platforms such as Python, R o Knowledge of agile methodologies, DevOps and DataOps principles o Written and verbal communication and presentations skills o Knowledge of data discovery, analytics and BI software tools like Microsoft PowerBI. o Ability to continuously learn in the information management, architecture, technology, and business fields Essential Functions o Design, build and optimize managed data pipelines from operational data stores to points of consumption such as an enterprise data warehouse, data mart, or end user technologies o Use innovative tools, techniques and architectures to automate most common, repeatable tasks to maximize data quality and productivity o Collaborate across teams to work with stakeholders to ensure that data requirements are appropriately documented, refined and satisfied by the team o Apply and use modern data preparation, integration and AI-enabled metadata management tools and techniques o Track data consumption patterns to proactively identify new requirements or refinements, and report against preset baselines o Use, understand and gain experience with artificial intelligence (AI), machine learning (ML) and predictive analytics to help drive customer experiences o Perform optimization of data environments using techniques like intelligent sampling and caching o Manage logical and physical data models in all their forms, including conceptual models, relational database designs, message models and others o Implement data architectures that can identify, prioritize and execute the data and analytic initiatives focused on defined enterprise strategies and business outcomes o Follow data and analytics security requirements and solutions, and work with management to manage risks and ensure confidentiality, integrity and availability of enterprise data and analytics assets #LI-CN1
Salary Range:
$80K -- $100K
Minimum Qualification
Data Science & Machine Learning, Systems Architecture & EngineeringEstimated Salary: $20 to $28 per hour based on qualifications.
Bachelor's Degree Computer Science, Statistics, Applied Mathematics, Data Management, or Information Science or related quantitative subject OR- Master's Degree Computer Science, Statistics, Applied Mathematics, Data Management, or Information Science or related quantitative subject Work Experience Less than 2 years Data Engineer I experience OR - 3 - 5 years Business, information, or technology experience General Employee Knowledge, Skills, and Abilities o Ability to establish effective working relationships among team members and participate in solving problems and making decisions o Ability to present and express ideas and information clearly and concisely in a manner appropriate to the audience, whether oral or written o Ability to actively listen to what others are saying to achieve understanding, sharing information with others and facilitating the open exchange of ideas and information o Ability to establish courses of action for self to accomplish specific goals, develop and use tracking systems for monitoring own work progress, and effectively use resources such as time and information o Ability to make right decisions based on perceptive and analytical processes, practicing good judgment in gray areas o Ability to design, manage, and implement simple data flows and information architectures for financial institutions o Knowledge of analytics scripting tools using languages such as R, Python/PySpark, Java, and C++. o Skill with big data platforms like Hadoop and Azure Synapse o Ability to design, build and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management. The ability to work with both IT and business in integrating analytics and data science output into business processes and workflows o Skill with popular database programming languages including T-SQL for relational databases and certifications on upcoming NoSQL/Hadoop oriented databases like MongoDB, Cassandra, others for nonrelational databases o Ability to work with large, heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using traditional data integration technologies using APIs, SQL Server SSIS, etc. o Ability to work with SQL on Hadoop tools and technologies including HIVE, Impala, Presto, Hortonworks Data Flow (HDF), Dremio, Informatica, Talend and others o Ability to work with and optimizing existing ETL processes and data integration and data preparation flows and helping to move them in production o Ability to work with both message queuing technologies such as Kafka, JMS, Azure Service Bus, and stream data integration technologies such as Apache Nifi, Apache Beam, Apache Kafka Streams, Amazon Kinesis and stream analytics technologies such as Apache Kafka KSQL, Spark Streaming, Samza and others o Knowledge of popular data discovery, analytics and BI software tools like Microsoft PowerBI for semantic-layer-based data discovery o Ability to working with data science teams in refining and optimizing data science and machine learning models and algorithms o Knowledge of data science platforms such as Python, R o Knowledge of agile methodologies, DevOps and DataOps principles o Written and verbal communication and presentations skills o Knowledge of data discovery, analytics and BI software tools like Microsoft PowerBI. o Ability to continuously learn in the information management, architecture, technology, and business fields Essential Functions o Design, build and optimize managed data pipelines from operational data stores to points of consumption such as an enterprise data warehouse, data mart, or end user technologies o Use innovative tools, techniques and architectures to automate most common, repeatable tasks to maximize data quality and productivity o Collaborate across teams to work with stakeholders to ensure that data requirements are appropriately documented, refined and satisfied by the team o Apply and use modern data preparation, integration and AI-enabled metadata management tools and techniques o Track data consumption patterns to proactively identify new requirements or refinements, and report against preset baselines o Use, understand and gain experience with artificial intelligence (AI), machine learning (ML) and predictive analytics to help drive customer experiences o Perform optimization of data environments using techniques like intelligent sampling and caching o Manage logical and physical data models in all their forms, including conceptual models, relational database designs, message models and others o Implement data architectures that can identify, prioritize and execute the data and analytic initiatives focused on defined enterprise strategies and business outcomes o Follow data and analytics security requirements and solutions, and work with management to manage risks and ensure confidentiality, integrity and availability of enterprise data and analytics assets #LI-CN1
Salary Range:
$80K -- $100K
Minimum Qualification
Data Science & Machine Learning, Systems Architecture & EngineeringEstimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
