Data Engineer | UK/Europe | Remote | £50-70K

About SearchPilot

SearchPilot built the first SEO A/B testing platform to come to market. We are on a mission to prove the value of SEO for the world’s biggest websites by empowering them to make agile changes and test their impact, and we are looking to expand our small engineering team.

SEO is Search Engine Optimisation - helping our customers grow their visibility in Google and other search engines. We are making the process of SEO more data-driven and scientific. We are proud to bring real value to the teams and companies using our technology.

About the Role

We are looking for a Data Engineer who is based in the UK or Europe to help us as the amount of data that we ingest and process continues to increase. You would be someone who has a strong understanding and several years of experience with storing and processing large amounts of data in different formats.

We are looking for somebody who could take ownership of the existing data infrastructure and help architect solutions for the next generation of that infrastructure as we look to store more data, whilst optimising performance & resiliency. Knowledge of Python, MySQL and AWS are key in this role.

SearchPilot operates by sitting between a customer’s CDN and their origin server. As part of our service, we proxy traffic for many other websites, which requires a large, robust, and cutting-edge infrastructure and high performance.

Our current data stores:

• MySQL is used for storing customer analytics data.
• Multiple instances of Redis, distributed globally, for storing proxy rules.
• CloudWatch storing log data.
• AWS Managed Prometheus for storing metrics data from our infrastructure.
• Elasticache is used for storing log data, and search indices for the web app..
• Various S3 buckets

Our current data pipelines (primarily written in Python):

• Daily imports of analytics data via Email (for multiple websites).
• Daily API pulls from Google Analytics (for multiple websites).
• Kinesis is used for storing specific log data.
• Daily jobs running deep mathematical analysis on test data (large datasets).
• Infrastructure metrics via scraping (Prometheus)
• CloudWatch log data.

You should have experience with:

• Python - strong Python skills will be necessary as this is the language used for most aspects of data pipelining. Experience with Pandas & Numpy are beneficial.
• 3+ years of experience working with large datasets, with an understanding of the considerations around storage and pipelining.
• MySQL - ideally with experience around optimisation.
• AWS - knowledge of AWS services focused on data would be very useful (RDS, Elasticache, S3, Kinesis).

You will work alongside our existing engineers, and report directly to the CTO.

An overview of your responsibilities:

• Working with developers to spec out and build out solutions to cater to new features relating to storing or analysing data.
• Reviewing existing infrastructure around data, and architecting improved solutions to optimise the performance, resiliency and complexity.
• Maintaining and monitoring data stores.

Location Flexibility

We are looking for applications from UK or European-based candidates who wish to work primarily remotely, with occasional travel to London (1-2 a year).

We believe in the benefits of working remotely, but also value a sense of shared mission and personal community, which benefits from sometimes getting together. If your circumstances don’t perfectly align with this, we are flexible to discussion.

Application Process

We are more interested in your skills and potential than we are in where you went to school or whether you went to university, but SRE/DevOps experience and strong AWS knowledge are essential.

We anticipate the interview process will include a short qualification call followed by 2 remote video interviews with different members of the team. We may ask you to complete a short technical task between the interview rounds.

Preferred Experience / Skills

We’d love to meet anyone who ticks most of the following - note that you do not need to have deep experience in all of these areas to apply:

• 3+ years of experience working with large scale data
• Knowledge of MySQL and Python
• 12+ months AWS experience.
• Experienced and confident enough to architect robust solutions independently.
• A strong communicator who can explain technical concepts clearly
• Enjoys getting things right whilst realising that done is better than perfect
• Enjoys learning new things
• Self-directed and self-motivated
• Experience with Prometheus is useful.
• Experience with Kinesis is useful.
• Experience of time series data analysis is useful

Benefits

• Expected salary range £50-70k depending on experience
• 25 days annual leave plus 8 bank holidays
• Subsidised private healthcare and life insurance
• Enhanced family leave and pay entitlement - same for both parents - up to 39 weeks paid
• 5% pension matching
• Subsidised gym membership
• A Kindle with some books preloaded
• A £600 personal budget each year you can use to make you happier or more productive at work such as a new flashy monitor, a new office chair, or training
• A laptop and second monitor (naturally)

About SearchPilot

SearchPilot spun out of Distilled in early 2020, with the whole software team of 14 becoming part of the new company. At Distilled, the product was called DistilledODN.

At SearchPilot we are passionate about our team, and we want that to be more than just words. We want to continue to build a team of people who can thrive and succeed, as well as feel valued and welcomed, at all levels.

As part of that goal, we are trying hard to actively build a diverse team, as we believe diversity is a necessary component to building the strongest company possible. To this end, one of the steps we have taken is committing to an annual report assessing how we are doing on this front: 2021 report. / 2022 report.