Data Engineer III at LexisNexis South Africa

About the Role 


At LexisNexis Intellectual Property (LNIP), our mission is to bring clarity to innovation by delivering better outcomes to the innovation community. We help innovators make more informed decisions, be more productive, and ultimately achieve superior results. By helping our customers achieve their goals, we support the development of new technologies and processes that ultimately advance humanity.
As a Data Engineer at LexisNexis Intellectual Property (LNIP), you will be designing, building, and maintaining complex data systems to support business needs. Utilizing your expertise, you'll contribute to maintaining a Data Platform that serves our crucial and critical product. Your role will be critical in ensuring the integrity, security, and accessibility of business-critical data. 


Responsibilities 


Be a hands-on Data Engineer working across our ecosystem of products and Data Platforms.
Interface with other technical personnel or team members to finalize requirements.
Developing and maintaining data infrastructure supporting real-time data processing in streaming architectures.
Implementing scalable data ingestion and ML pipelines, incorporating Data Lakehouse concepts for unified data management
Successfully implement development processes, coding best practices, and code reviews. 
Designing APIs for diverse business units, ensuring efficient data lineage tracking
Utilizing DataOps principles to enhance system performance and reliability.
Automating system lifecycle management, ensuring robustness and scalability
Integrating advanced data engineering techniques and tools to streamline processes.
Resolve technical issues as necessary. 
Keep abreast of new technology developments. 


Technical Skills   

Required


Experience in SQL Server, Data Lake (Azure/AWS).
Possess extensive modern Data Engineering experience.
Demonstrate in-depth knowledge of large-scale data platforms (Databricks, Snowflake) and cloud-native tools (Azure Synapse, RedShift).
Knowledge of software development methodologies including Scrum, Kanban, and Agile more broadly. 
Experience of analytics technologies (Spark, Hadoop, Kafka).
Experience with test-driven development. 


Nice to have


Understanding of Elasticsearch, Solr, PostgreSQL, Databricks, Delta Share & Delta Lake. 
Ability to work with complex Patent and Litigation data models. 
Ability to work well with internal and external technology data resources, including DocDB, ESpacenet & USPTO. 
Experience with Pandas & PySpark.