Data Engineer

Data Engineer

Whitehall Resources require a Data Engineer to work with a client based in Hampshire.

The client are at the beginning of a major step change in the role that Business Intelligence and Analytics plays within the company as it moves to a more data-driven organisation. They are looking to build a strong in-house capability to enhance the existing traditional descriptive business BI/Analytics but also to grow capability to support high end predictive/prescriptive analytics including areas such as Big data, Machine Learning and Artificial Intelligence.
As part of this change they are looking for a Data Platform Engineer that will provide the necessary skillsets to ensure that they have robust, managed platforms on which to deliver these new capabilities.
This Data Engineer position is a new role within the organisation to add responsibility for the management, design and on-going support of the various BI/Analytics platforms in use to the Information & Insight team. This includes the existing BI platforms of Microsoft SQL 2017 but also to work on establishing new Analytical platforms based on Splunk, Hadoop, Docker, Python etc.

* The role day to day will be to design, implement, manage, and administer the Analytic platforms and services. The goal is to deliver an open, secure, robust, scalable, reliable and high-performance solution.
* You will work closely with the data, network, application and analytic teams to make sure that all the big data applications are highly available and performing as expected. Experience with capacity planning and estimating the requirements for lowering or increasing the capacity of Hadoop based platforms will be helpful.

* Represent the Information & Insight team at meetings with internal infrastructure colleagues as well as external suppliers/vendors.
* Ensure that appropriate audit and security controls exist for data hosted and served from the platforms.
* Work with the Data Sourcing teams to ensure data is loaded, maintained and available from the data platforms to meet the required availability, security and performance requirements asked.
* Maintain an awareness of emerging tools, skills, techniques and technologies, and where appropriate make recommendations relating to their timely introduction for competitive advantage.


* Ideally 5 or more years’ experience in designing, running and troubleshooting Hadoop based analytic platforms, however for a development candidate then working knowledge of these platforms might be sufficient.
* Preferably 3 or more years’ experience in supporting Microsoft SQL 2016 or above Data & BI platforms.
* Experience in working on implementing and supporting data platforms within secure, regulated industries.
* Preferably 3 or more years of experience in information-intensive industries or digitally advanced enterprises.

* Excellent interpersonal skills; able to work across teams to influence and effect change to achieve common goals.
* Either detailed knowledge of (full role) or familiarity with (development role) some of the following data related platforms or technology:
* Expertise in at least one commercial distribution of Hadoop.
* Experience or knowledge in the Splunk toolset.
* Batch and streaming job frameworks (such as Spark, Storm).
* NoSQL databases (e.g HBase, Cassandra, MongoDB).
* Middlewares and messaging systems (e.g. Kafka, RabbitMQ, FTL, Ultra Messaging).
* Scripting (e.g. R, Python, Perl).
* Automation via the use of configuration management (such as Puppet or Chef) and orchestration tools (such as Ansible).

The following would be advantageous but not necessary:
* Service discovery (e.g. Zookeeper, Consul, etc).
* Data collection and querying (such as Flume, Sqoop, Hive).
* Time series databases (such as I

All of our opportunities require that applicants are eligible to work in the specified country/location, unless otherwise stated in the job description.