Each year, we’re treated to a dizzying array of new services, capabilities and technologies at AWS’s biggest cloud event of the year, re:Invent. From leadership keynotes and breakout sessions to builder labs and bootcamps, there’s something for everyone!
But it’s not just about the cloud, this year, AI/ML took centre stage. So, we asked Software / Data Engineer and Optimisation Specialist at DiUS, William Infante, for his top four (because three just isn’t enough!) AI/ML announcements that might just revolutionise the way we use machine learning and leverage smart algorithms.
1. New geospatial data capabilities for ML
Today, the majority of data generated contains geospatial information, yet only a small fraction can be used to train machine learning models. So, I was pleased to learn that Amazon SageMaker now supports geospatial data, even if it’s only available in preview mode (for now).
Instead of using third-party services or integrations, Amazon SageMaker can now efficiently process or enrich large-scale geospatial datasets to accelerate model development. I can see huge value in this new geospatial data capability, especially when it comes to enriching visual machine learning models, such as interactive maps for food crop assessment, drone delivery and consumer demand prediction.
Read more about this announcement.
2. Automated, real-time ML model validation
The announcements continued, with not one, but eight new Amazon SageMaker capabilities. In addition to geospatial data, Amazon SageMaker now offers automated, real-time model validation for shadow testing. But why is this a big deal? Well, shadow testing routes real-time inference requests from the production model to the model under testing.
Previously, validating a model would mean using historical data, and if real-time inference requests were needed, a manual process would need to be built and the required testing infrastructure established.
Not only will this new machine learning model validation capability make it easier for engineers to perform shadow testing, it’s been built to scale and includes a dashboard that displays performance differences across key metrics. My hope is that this new capability will support testing and validating models closer to the standard (not optional) way of bringing models to production.
Read more about this announcement.
3. Ready-to-use quantum algorithms
Quantum computing can help resolve scaling issues for specific use cases such as scheduling, route optimisation and resource allocation. However, without ready-to-run code, it can be extremely difficult to get started when experimenting with quantum computers or building quantum algorithms.
While Amazon Braket takes out a lot of the guesswork, the launch of Braket Algorithm Library will help reduce the time it takes for engineers and data scientists to get started. Specifically, I like that this new capability provides common routines with Python implementations of well-known quantum algorithms. Additionally, you can run parts of the code using the Braket console (or GitHub), and the examples can help estimate the costs associated with running these services.
Read more about this announcement.
4. Automated sensitive data discovery
Due to recent data breaches, protecting sensitive data, such as personally identifiable information, has become the number one priority for many organisations. So, I was pleased to learn that Amazon Macie now offers automated sensitive data discovery capabilities.
While I haven’t yet had the opportunity to experiment with this new service, I understand that it intelligently inspects S3 Buckets using machine learning and pattern matching techniques to automatically identify sensitive data such as names, addresses and financial information. Additionally, the interactive data map shows where the sensitive data resides by using techniques such as resource clustering to strike the right balance between identifying sensitive data and keeping costs low, while enabling organisations to quickly respond to potential cyber attacks.
Read more about this announcement.
That’s a wrap (well, kind of)…
So, there you have it! These are just four of the many new services, capabilities and technologies announced at this year’s AWS re:Invent. If you (like us) love to stay across the latest and greatest, you can watch these announcements and more on the AWS re:Invent 2022 YouTube playlist. Happy watching!