Five remarkable technologies for cloud data management
Technologies are no longer just a means to an end. Technologies are the foundation for the resilience and progressiveness of every company. The challenge lies in identifying the right technology stack, the cornerstones of which are anchored in a modern cloud architecture. In the future, no business strategy will work without a parallel technology strategy. By foregoing technology, every company would catapult itself into the digital middle ages.
So far, so clear. But the number of decisions that now need to be made is overwhelming for many. Strategic issues such as cloud implementation or data science offensives are still completely unresolved in some cases, and the selection of suitable technologies and end devices raises many questions. Furthermore, building a technology stack is no longer just an issue for the IT department, but affects all employees in the company. This is because they are a key success factor for the digital transformation.
Technologies and tools - the right stack is crucial
So which technology, which tool, which stack is the right one for a company? The answer to this question is, of course, a highly individual matter. However, some cloud technologies are currently clearly gaining ground, as they respond comprehensively and flexibly to a wide range of customer needs. From our perspective and practical experience, companies should keep an eye on these five technologies:
1. Microsoft Azure for the hybrid cloud
Together with AWS and Google, Azure is one of the top 3 providers in the field of cloud technologies - and is becoming increasingly popular. The expanded hybrid cloud offering, which makes it easier for companies to enter the cloud environment, is also likely to contribute to this. Applications and processes can easily be partially outsourced to the cloud while the company continues to use familiar on-premise solutions. A major advantage for Microsoft is also the use of established systems such as Microsoft 365 or Power BI. Companies that already rely on these or other solutions from the Microsoft universe see the decision to use Azure as a logical extension of their tech stack. However, it is not only the affinity with the provider that is convincing, but also the comprehensive and flexible solutions and services. Security is a top priority at Microsoft anyway: The Azure Security Center uses a variety of physical, infrastructural and operational procedures for the basic protection of Azure, even without additional intervention.
2. fivetran for cloud-based data integration
According to Bigdata Insider, the provider of automated data integration once again doubled its annual turnover and customer base last year. As a cloud-based data integration service, Fivetran manages its customers' data pipelines and provides instant access to more than 1.5 trillion rows of data per month. Thanks to the ELT (Extraction-Load-Transform) stack, data is made available faster. Fivetran makes data from a wide variety of source structures in a data warehouse (DWH) or other databases possible without any coding and with very little maintenance. This leaves more time for analysis. George Fraser, CEO of Fivetran, sees his vision as "making access to data as simple and reliable as electricity from the socket." Analysts can make better decisions with Fivetran. This is because the tool manages more than 700,000 schema changes every month. Such modifications usually lead to ETL pipelines being interrupted and only running again once they have been adjusted by data engineers. Up-to-date data is available at all times, even if the source systems change.
3. snowflake as a cloud data warehouse
"I love data. I love computer science."
"Me too."
With passion, creativity and outstanding technical expertise, the two Snowflake founders Benoit Dageville and Thierry Cruanes have created the biggest bromance(see Snowflake video!) and success story in data warehousing in the last ten years. They started from scratch and built a data platform that harnessed the immense power of the cloud. The system separates data computing and data storage, which was a revolution in data warehousing - and is still unique today. Almost every other database combines both, which means they have to carry a maximum system load and also the associated costs. The data can all be stored in a single location, while the calculations are carried out independently of each other. This positions Snowflake as a Data Warehouse-as-a-Service. Last year, Snowflake recorded triple-digit growth in product sales. "In conjunction with this rapid growth, we have been able to improve operational efficiency while expanding our presence globally," says Frank Slootman, CEO of Snowflake. The success story continues!
4. dbt for cloud-based data transformation
Integrating all data into a data warehouse can be more than challenging due to the complexity and heterogeneity of the data. In the field of analytics engineering, data transformation is the work step that stands between loading the data into the warehouse and the subsequent analysis. The SQL-based data building tool dbt makes this process considerably easier. Put simply, dbt is a transformation and modeling tool that can provide significant, cloud-based support for the work of software engineers during the data transformation process. The tool is now the standard for merging disparate data. Within the company's technology stack, ideally in combination with Snowflake and Fivetran, dbt can and should play a routine role.
5. tableau enables self-service analysis in the cloud
As a fully cloud-hosted analytics platform, Tableau facilitates the creation of interactive visual analysis of data. The cloud service is described by the provider itself as the most popular modern analytics platform in the world, helping people and organizations to work in a more data-driven way. The fact that this is no longer just a vision can be clearly seen in everyday data work: The extremely clear and easy-to-configure dashboards enable even non-technical analysts and end users to transform data into comprehensible graphics. Tableau can be connected to any data source. The analysis tool's strongest competitor at the moment is certainly Microsoft Power BI, but it differs in striking ways (read our blog post Power BI vs. Tableau!).
Selection of cloud services remains a Herculean task
It is certainly always a very individual and complex decision as to which services and tools ultimately become part of the technology stack. Numerous factors such as consideration of the IT strategy, business perspectives and historical service provider relationships influence the selection process down to the smallest detail. In our opinion, the above overview represents the most valuable, future-oriented and innovative tools currently available, which we also know very well from experience.
