Senior Data Engineer (AWS)
-
You are passionate about modernising the HR ecosystem through
strategic and secure management of cloud data products, pipelines and
databases
-
We are undergoing one of Australia’s largest digital
transformations and establishing an Engineering function within HR
-
Together we bring an engineering culture and passion for
DevSecOps and GitOps within the cloud
Do work that matters
As a Senior Data Engineer and database specialist you will be designing,
creating and managing the cloud databases and ETL Data pipelines that
underpin our decoupled cloud architecture and API first approach. You
have proven expertise in database design, data ingestion, transformation,
data writing, scheduling and query management within a cloud environment.
You will have proven experience and expertise in working with AWS Cloud
Infrastructure Engineers, Software/API Developers and Architects to
design, develop, deploy and operate data services and solutions that
underpin a cloud ecosystem. You will take ownership and accountability
of functional and non-functional design and work within a team of
Engineers to create innovative solutions that unlock value and modernise
database and data pipelines.
You will role model continuous improvement mindset in the team, and in
your project interactions, by taking technical ownership of key assets,
including roadmaps and technical direction of data services running on
our AWS environments.
See yourself in our team
The HR Technology function works in an Agile methodology with our HR
business to plan, prioritise and deliver on high value HR & technology
objectives with key results that can be experienced by all of our
employees.
You will work within the Middleware, ETL and Data Engineering Chapter
that is working on initiatives such as establishing decoupled HR
architecture through API’s and establishing ML Ops (Sagemaker). You will
do this through expert knowledge of database design that supports an
Operational Data Store and Caching, data engineering and data
transformation.
A typical day would work with data producing systems such as Workday
Engineers to define, extract, transform and load to PostgresDB and then
working with AWS Engineers and API developers to expose the data products
through API’s for secure consumption in various HR Applications and
solutions.
We’re interested in hearing from people who
-
Have design and implement databases and data models experience
-
Have experience in developing and maintain scripts or ETL (Extract,
Transform, Load) processes to ingest raw data from various sources,
including JSON and XML files
-
Can design and implement transformation logic to convert raw JSON and
XML data into structured formats such as CSV to write to databases
-
Can create and optimize scripts to write transformed data into
relational databases (e.g., PostgreSQL- Ensuring that data is
properly indexed, normalized, and stored efficiently
-
Can automate data ingestion and transformation processes using
scheduling tools
-
Can monitor and troubleshoot data pipelines to ensure reliability and
performance
-
Have experience working through performance and scaling through
horizontal scaling designs vs database tuning
-
Can design application logical database requirements and implement
physical solutions
-
Can develop back-end database code to support API requirements.
-
Can collaborate with business and technical teams in order to design
and build critical databases and data pipelines
-
Can performance tune applications from a database code and design
perspective
-
Can advise business owners on strategic database direction and
application solution design
Tech skills
We use a broad range of tools, languages, and frameworks. We don’t
expect you to know them all but having significant experience and
exposure with some of these (or equivalents) will set you up for success
in this team.
-
AWS Data products such as AWS Glue and AWS EMR
-
AWS Aurora RDS such as PostgreSQL
-
AWS S3 ingestion, transformation and writing to databases
-
Realtime streaming technologies, Kafka
-
Proficiency in programming languages like Python, Java, or Ruby for
developing data ingestion and transformation scripts
-
Strong knowledge of SQL for writing, optimizing, and debugging
queries
-
Familiarity with database design, indexing, and normalization
principles
-
Understanding of data formats (JSON, CSV, XML) and techniques for
converting between them. Ability to handle data validation, cleaning,
and transformation
-
Experience with all components of Ab Initio - GDE, Continuous
Graphs (desirable)
-
Proficiency in automation tools and scripting (e.g., bash scripting,
cron jobs) for scheduling and monitoring data processes
-
Experience in Agile frameworks
Working with us
Whether you’re passionate about customer service, driven by data, or
called by creativity, a career with CommBank is for you.
Here, you’ll thrive. You’ll be supported when faced with challenges and
empowered to tackle new opportunities. We really love working here, and
we think you will too.
If this sounds like the role for you then we would love to hear from you.
Apply today!
If you're already part of the Commonwealth Bank Group (including
Bankwest, x15ventures), you'll need to apply through
Sidekick to submit a
valid application. We’re keen to support you with the next step
in your career.
We're aware of some accessibility issues on this site,
particularly for screen reader users. We want to make finding
your dream job as easy as possible, so if you require
additional support please contact HR Direct on 1800 989 696.
