You can gain significant leverage by selecting the right Extract, Load, Transform (ELT) tool for your business. In this guide, we outline the key considerations, questions to ask, and what to look for when evaluating ELT solutions
There are five main considerations
- Connectors - Which connectors do they support? Do they cover the core systems you need to stand up your data stack? Can they provide you with connectivity to the long-tail of your organization's applications?
- Roadmap - How fast can they build? What's their vision for growing with you as an organization? Are they responsive to requests?
- Pricing - How much does the solution cost? Does the pricing model align with your data profile?
- Support - How do you know your pipelines will be maintained? Who is 'on call' when things break? Do you have a direct line to someone that can solve your problem?
- Security - How does the company approach security, privacy, and compliance? Does it align with your company's particular considerations?
Let's dig into each concept in more detail
Consideration #1 - Connectors
- Support for your data warehouse or analytics environment. Most ELT solutions support all of the major analytics environments as destinations. However, you should confirm that the particular ELT solution you're evaluating supports the environment you use to process data today, but also any environments you may want to switch to over time
- Depth of mainstream connectors. When first standing up a data stack, you need support for your product database, your Customer Relationship Management (CRM) solution, and your other core business applications. For each system, look for robust schemas and a simple path to start creating value quickly. The goal of these initial connectors is to demonstrate a positive return on investment (ROI) for your analytics initiative as a whole
- Breadth of long tail connectors - Once you have the groundwork in place for analytics at your company, you'll need a breadth of data sources to dig into particular aspects of your business (customer churn, employee retention, strategic partnership opportunities). This means no-code connectors beyond your core business applications. Finding a partner flexible enough to support these long tail connectors is critical to unlock unique insight into your business operations
Consideration #2 - Roadmap
- Demonstrated ability to build connectors fast - How quickly does the vendor build new connectors? Ask the vendors you are evaluating to show you how many net new connectors they have delivered in the last few months. If they're not adding new connectors, they may still be able to help you stand up your initial data stack, but won't be the right partner when you need a breadth of long tail connectors for deep insight into your business operations
- Vendor focus on existing vs. future clients - For connectors, ask about the future roadmap and see if the vendor prioritizes connector development based on requests from existing clients, or do they rather prioritize connectors to unlock net new clients for them. You want to find a vendor that prioritizes the needs of their existing clients instead of simply getting you live on the platform, and moving on to the next client. Otherwise, you'll be looking for another ELT tool to fill in the gaps with long tail ELT connectors. If you find yourself trying to track down long tail ELT connectors, luckily Portable can help
- Alignment of vendor's strategic roadmap with your needs - Beyond ELT connectors, identify the features and capabilities the platform is building that help you and your team specifically. Will they offer other types of integrations you can use? Will they simplify data transformation? Can they help with orchestration? You should make sure you understand whether their roadmap revolves around your needs, or expanding their own client base
Consideration #3 - Pricing
- Key drivers of cost - First and foremost, you need to make sure you understand how their pricing structure works, the levers you can pull to reduce costs, and the drivers of higher expense. Are you paying per connector? Per record of data moved? Per GB of data processed? Or on a metric that is difficult to track and understand? Make sure you can create realistic assumptions to forecast and track costs
- Estimated costs today and in the future - Based on the pricing model, create low usage and high usage assumptions, and work with the vendor to estimate how much things will cost initially, and how they will increase as data volumes and connector usage expand
- Worst case scenario - It's always important to understand what happens if you somehow end up moving significantly more data than expected. Could your monthly cost double overnight? Will your data stop flowing? Are there alerts in place for these scenarios? Regardless of the outcome, the most important part is being prepared for what happens in these scenarios
- Hidden costs from open source and self-hosted software - If you're leveraging open source or self-hosted technology, it's important to not just consider managed service costs, but also to budget for any cloud processing bills your team will have to pay in order to sync the data
Consideration #4 - Support
- Maintenance of existing connectors - Make sure you understand who is 'on call' when a connector breaks, or a schema changes. Make sure you understand what issues are your responsibility to troubleshoot (i.e. invalid credentials, etc.) and what is the responsibility of the vendor (i.e. system wide outages, etc.)
- Support channels - Do you have a direct line to someone that can solve your problem? Will you be pushed to a community forum? What's the typical turnaround time for issues? Do you have a technical contact to troubleshoot data issues? Is monitoring available?
- Accountability for solutions backed by open source - For connectors backed by open source projects, you need to understand who is actually accountable when things break. Is it the vendor you're paying accountable? Or is a third party developer somehow accountable for your SLA? Is there a clear path to escalate if the third party doesn't help? Or are you left stranded with a broken data flow?
- Maintenance of infrastructure - For open source projects, make sure you understand the overhead of infrastructure deployment and management to keep your pipelines working. Do you need an entirely different layer of monitoring for the infrastructure you're now managing?
Consideration #5 - Security
- Understand your company's requirements - Security, privacy, and compliance are critical to running a data team within any organization. Make sure you understand the regulation in relevant geographies (CCPA, GDPR, etc.) and verticals (HIPAA, COPPA, etc.) that apply to your business. Make sure you understand the procurement considerations for vendors, and any requirements your company must adhere to as part of audit processes (SOC 2, etc.) or legal considerations (DPAs, etc.)
- Key security considerations - At a minimum, there are three questions you should always ask of a vendor that is moving data on your behalf: 1) How do you secure the access credentials we provide to our systems, 2) Do you retain any of my underlying data within your systems after a load job completes, and 3) Who inside your organization, or outside your organization, has direct access, or possible access to our credentials and our underlying data
- Ask for a summary of policies, procedures, controls - Companies typically implement Information Security Management Systems (ISMS) with documented policies, procedures, and controls. Most security frameworks (SOC 2, ISO 27001, etc.) are a formalized way of reviewing these policies, procedures, and controls. Ask for a copy of the company's information security materials, and make sure you understand the controls in place to protect your data
Selecting the right ELT tool for your business can provide unbelievable leverage to your team and organization, but the ecosystem is complex. We hope the recommendations above can help you on your journey
In many scenarios, it can make sense to find one tool to get started with ELT, and a second ELT partner flexible enough to grow with your needs and support the long tail connectors that unlock unique insight into your business operations
If you have any questions when evaluating vendors, or think Portable could be a good partner, don't hesitate to reach out
Want to learn more? Book time for a discussion or a demo directly on my calendar
Enjoy our posts? Subscribe to our newsletter to receive content directly in your inbox.