When you're evaluating custom ETL services, how can you make sure to pick the right solution?
Every data team comes across the occasional long tail connector. That random webinar tool, the referral tool, or the HR tool that you need to connect to your data warehouse, but you can't find an off-the-shelf ELT tool that supports the system, and you really don't want to write code.
What are your options?
- You could write code yourself
- You can find a service provider to build the custom ETL pipeline for you
If you decide to engage a custom ETL service, you've inherently made a few decisions.
First, you don't want to manage infrastructure.
Second, you don't want to write a script to move your data.
Third, you trust the service provider to not only build the custom ETL for you, but to maintain the ongoing data flow, and the infrastructure forever.
For most data teams, you should NOT think about infrastructure, and you should definitely NOT spend time reading API documentation, writing scripts, managing rate limits, and troubleshooting errors.
Great decision to find a solution to your problem.
However, the third point above is risky.
When you engage a custom ETL service provider, you're paying for development of the custom ETL pipeline, and then you're paying an ongoing fee for maintenance and troubleshooting.
That makes sense at the highest level. But as time goes on, the consultant or service provider will have other opportunities.
They'll have big projects that pay lots of money.
Or they'll have new clients that will pay more for development of a new custom ETL pipeline than you're paying for maintenance of an existing custom ETL pipeline.
If they stop supporting your pipeline, you have a major problem.
You're back to the drawing board on maintenance. And you very well might be back to the drawing board on development of your custom ETL pipeline as well.
Are there other options?
You can find a product company, that focuses exclusively on development and maintenance of long tail custom ETL pipelines.
A company that won't get distracted by large consulting engagements or the next flashy custom development project.
At Portable, we focus our entire business on building and maintaining custom ETL pipelines for our clients.
You can think of us as a direct substitute for a custom ETL service (because we build net new connectors on demand for clients), but with the stability, reliability, and pricing of a SaaS product.
How is this possible?
We're experts in data integration, and we've invested years of R&D to develop the most efficient platform to build custom ETL pipelines.
We can turn around new pipelines from APIs to data warehouses in a matter of hours or days, so you can have a no-code product experience with the customization of a custom ETL service.
Enjoy our posts? Subscribe to our newsletter to receive content directly in your inbox.