Managed AI and machine learning cloud services — including AWS Bedrock, Google Vertex AI, and Microsoft Azure AI — have transformed enterprise software development. Developers now build intelligent applications by composing foundation model capabilities (language, vision, speech, embeddings, code) directly as black-box, pay-per-call APIs. This shift has created a significant and largely unaddressed challenge: the performance of these services is not stable, predictable, or transparent. The same API call can vary by an order of magnitude in response time across providers, regions, load levels, and times of day, yet no principled framework exists to help enterprise developers reason about this variability when selecting a service.
Aim
This project aims to systematically characterise, model, and predict performance variability in managed AI/ML cloud services across AWS, Azure, and Google Cloud Platform, and to translate those predictions into an evidence-based service selection framework that enables enterprise developers to choose the cloud AI service best suited to their end-to-end performance requirements.
Objectives
- Conduct a longitudinal, multi-provider measurement study of managed AI/ML APIs, capturing latency distributions, throughput, failure rates, and cost across varied load levels, input characteristics, regions, and time periods over 12 months.
- Develop a variability attribution framework that identifies and quantifies the root causes of observed performance variation — including multi-tenancy interference, cold-start dynamics, rate-limit interactions, and model routing — using only externally observable signals.
- Build and validate a calibrated probabilistic prediction model that forecasts latency distributions (p50/p95/p99), failure probabilities, and throughput ceilings for a given workload specification, with support for transfer learning to newly released APIs.
- Design and evaluate a variability-aware service selection framework that matches a developer’s performance and cost requirements to the most suitable managed AI service, supported by a user study with enterprise developers.
Significance
Managed AI cloud services are the dominant infrastructure layer for enterprise AI adoption globally, yet the lack of transparency in their performance behaviour imposes hidden costs, reliability risks, and poor user experiences on applications built on top of them. This project addresses a genuine gap in both research and practice: no systematic measurement dataset, variability model, or selection framework exists for this class of services. The open dataset produced in the first phase will serve as a lasting community resource for cloud systems researchers. The selection framework will provide immediate practical value to the thousands of enterprise development teams navigating multi-cloud AI service choices. The findings also directly support policy objectives around AI service transparency under the Australian AI Ethics Framework and emerging international standards.
Ideal Candidate
Applicants should hold an Honours or Masters degree in Computer Science, Software Engineering, Information Technology, or a related discipline, with a strong academic record. A background in one or more of the following is desirable: cloud computing, distributed systems, software performance, or data analysis. Basic programming experience, particularly in Python, is expected. Candidates with an interest in empirical research methods and a motivation to produce work with real-world impact are encouraged to apply. Additionally, the applicants should meet the eligibility criteria for entry into a PhD program at Curtin University.
This project is open to International and Domestic applicants.
Scholarship
If you are identified as the preferred candidate for this project, you may be considered for an RTP scholarship.
Enquires and How to Apply
For enquires about this opportunity contact Dr Sheik Fattah at Sheik.Fattah@curtin.edu.au
To formally apply submit an Expression of Interest to Dr Sheik Fattah during the Central Scholarship round (July 1st – July 31st 2026)