Network Security for PaaS Workloads
September 8, 2019
In the wake of the Capital One hack and breach, questions are again being raised on the overall security of public cloud infrastructures. Cloud Service Providers (CSP) have always maintained that security of the infrastructure is a shared responsibility between the customer and cloud provider. The number one priority for enterprises today as they migrate (or plan to migrate) to public clouds is to get a better understanding of the security model and evaluate and adopt the tools needed to improve their overall security posture in this new environment.
Enterprise Public Cloud infrastructures are fundamentally different from the ones in the on-prem data center. Applications being developed today in public clouds use a mix of IaaS and PaaS services. Enforcing security in this highly variable and dynamic application architecture is a complex exercise as I will describe later. In this blog we will focus our attention on PaaS services and what you as a customer can do to adopt solutions to protect against breaches and unauthorized access.
What is PaaS and Why is it Popular with Developers?
In addition to public cloud migration, one of the tectonic shifts in infrastructure today is the adoption of PaaS (Platform-as-a-Service) services. PaaS services are popular because as developers, we do not have to manage the underlying infrastructure as the usage of the service grows or shrinks. The infrastructure provider is responsible for the overall availability of the service irrespective of the scale. In the public cloud world, the provider is highly incentivized to provide and operationalize such services since it’s a win-win situation in terms of ROI — developers are happy to adopt and the CSP has higher margins for these kinds of services.
There are many kinds of PaaS services. Let’s look at AWS services in particular. Services such as Elastic Beanstalk are platforms to deploy custom applications to AWS. Applications implemented as AWS Lambda functions (aka serverless functions) and deployed through API Gateways are also PaaS. Slightly more nuanced services are AWS S3 or AWS RDS since the underlying service is core infrastructure (Storage for S3 and Database for RDS). Thus, these can be positioned as IaaS. But the operational complexity is abstracted out for developers and in our view, we can classify these as PaaS services.
What is the security model of a PaaS service?
AWS has a very rich model for enforcing security. At a high level, these include the following.
- Authentication and Authorization. These are described in detail here: Policy Evaluation Logic.
- Identity-based: Policies attached to an IAM identity (user, group, and role etc).
- Resource-based: Policies attached to the resource which is being accessed by an AWS user.
- Data at rest: Support for data at rest encryption wherever applicable (e.g RDS, S3)
- Data in transit: Support for end to end Transport Layer Security (TLS).
The following is an example S3 bucket strategy, where access is restricted to specific IAM users and roles.
S3 bucket with encryption of data at restTransit - SSLPrivate bucket - with IAM policy to designate a specific set of users
Here is another example where access to S3 bucket is restricted by a resource based policy to specific VPCs and VPC endpoints. This is useful to restrict access to S3 buckets to specific VPCs in your infrastructure. You can also specify IP address whitelists.
S3 bucket Private bucket - access restricted to a single VPC
While these are a required set of tools, these are not sufficient in terms of the needs of enterprise customers as exemplified through all the breaches over the years. Restricting access to specific VPCs is a step in that direction but these are simply not adequate since they are too coarse.
Enterprises need richer set of tools to control the lateral access to S3 and other PaaS services. One such tool is micro-segmentation. Given that these are PaaS services provided by the cloud provider, no third party tool has access to the host providing the PaaS service. Obviously host based security tools cannot help here by definition but network could be a great leverage point here. Hence, the only possible approach is network security.
Micro-segmentation in Public Cloud PaaS
How can enterprises achieve micro-segmentation in Public Cloud PaaS workloads? Think of the following use cases -
- Allow only EC2 instances running CentOS 7.x to download a critical patch from an internal S3 bucket.
- Allow traffic from a non-AWS data center (say Azure VNet) to access the S3 bucket. In this case, the set of source IP Addresses themselves are dynamic. So it is not trivial to keep updating the IP whitelist feature in S3.
Identity ExplainedOne can achieve micro-segmentation through IAM policy in certain scenarios. For e.g., one can assign the instances running CentOS 7.X with a specific IAM policy. Similarly, the resources in the non-AWS hybrid environments could share an IAM policy to be used for S3. As you can imagine, this is simply a non-trivial exercise.
Apart from the complexity in managing per-user policy vs policies associated with a Role (since a group of users or principals will be affected when there are changes), IAM credentials are not tied to a resource. They are in fact tied to a principal which can itself laterally move. For e.g, let’s say the IAM access keys intended for a specific user are leaked. The attacker can now assume those credentials and thus take over as the user.
Overcoming complexity in PaaS micro-Segmentation
At Valtix, we are working with customers to provide network security at a granular level for enterprise use cases. With our solution, enterprises can create “tag” based Address Objects to restrict access to internal S3 buckets in a multi-cloud/hybrid-cloud environment. These tags can be native attributes such as “VPC”, “Security Group”, “Instance”, etc., In addition, they can be “user defined tags” such as classic “Key/Value” pairs. This is all in an environment where there is full end-to-end TLS encryption between the client and the PaaS workload managed by AWS.
Let’s revisit the challenges described above. Now with Valtix, we would have achieved the above use cases in the following way.
- Define an Address Object based on the Tag “OS=CentOS 7.X”.
- Defined an Address Object for the Azure VNet “VNet=VNetb”.
- Allow these two Address Objects to access the internal S3.
This is being achieved in forwarding mode of Valtix firewall (note that Valtix firewall can work in forwarding mode for some rules and proxy mode for other rules) using IP addresses and HTTP headers.
This works in conjunction with existing security mechanism for S3, including IAM based policies, resource based policies for the S3 bucket. With Valtix’s security platform, customers can greatly enhance their security posture by implementing advanced micro-segmentation based on both user defined tags of the resources as well as organic attributes across different cloud providers.
Going beyond micro-segmentation: DPI for PaaS applications
In the previous example, we have shown in forwarding mode how to achieve micro-segmentation. Valtix also provides full TLS In / TLS Out proxy for PaaS workloads. With this configuration in place, we can enforce full IPS and WAF policies for both inbound and outbound traffic. An important use case in this architecture is protection against exfiltration of critical data. Another use case is malware detection for both inbound and outbound traffic.
Back to Capital One breach. How deploying Valtix could have prevented the breach.
The Capital One breach has been well documented here. Following is the high level timeline of events.
- One of the Apps in Capital One deployed in AWS has a WAF that is misconfigured.The App server’s IAM Policies has full access to the internal S3 buckets of Capital One.
- The attacker who has deep expertise in AWS, exploited the vulnerability to invoke the EC2 metadata service bypassing the misconfigured WAF. The specific vulnerability is known as Server Side Request Forgery (SSRF) due to which WAFs can be tricked to invoke internal services to retrieve data. In this case, the retrieved data is the temporary IAM credentials of the app server.
- Attacker uses the stolen IAM credentials and retrieves data from the internal S3 buckets.
There are various recommendations provided in various blogs as to how this can be prevented in future and by other organizations who may have similar architectures and vulnerabilities. Without repeating and going over all those recommendations, we will focus on Valtix’s approach in providing “multiple layers in defense” for enterprises leveraging the network to enforce security policy. Valtix’s approach is also complementary to the tools provided by the CSP, namely, resource based policies and identity based policies.
At a high level, such financial service organizations can leverage Valtix’s solution in conjunction with AWS (and other CSP) tools.
- Enforce strict S3 bucket based policies. Restrict the VPCs which can access specific S3 buckets based on usage. This step would have prevented the lateral access of S3 using the stolen IAM credentials from the attacker’s virtual machine.
- Enforce strict IAM based policies to restrict access to data that is necessary to the IAM principals.
- Deploy Valtix’ discovery solution that builds an evergreen model of your apps and infrastructure. Tag your applications and create the corresponding policy. Valtix will ensure that the policies move with your apps as the footprint changes or increases across your public cloud deployments.
- Use Valtix’s WAF solution to protect the Application. Build WAF profiles to protect against SSRF attacks. These profiles can be used in all of your multi-cloud deployments. You can also extend from a Valtix provided cloud friendly WAF profile that has this policy enabled by default.
- Implement micro-segmentation using tag-based Address Objects to enforce fine grained access to S3 within the AWS infrastructure.
- Implement full DPI of S3 services. This is optional. This requires changes in the applications accessing S3 to go via proxy. Access S3 by an internal Valtix’ powered proxy endpoint. Enable rich IPS policies to defend against exfiltration of data based on configured regular expressions. This step would have prevented the attacker who assumed the IAM credentials of the app server from downloading the consumer application data from S3.
Enterprises want a single platform where they can:
- Discover their applications across all of their Public Cloud accounts and regions
- Deploy network security by following the apps and workloads as they grow or shrink
- Defend by enforcing comprehensive network security services based on the workloads
Valtix’s platform consisting of Discover-Deploy-Defend is fundamentally changing the way network security is being done in public clouds. As Valtix continues to address the needs of enterprises to secure their public cloud infrastructures by adding new game-changing features, we’ll continue posting our response here in our blogs. Alternatively, if you’d like to get an update on where we’re headed and our roadmap sooner, contact us here and let’s get a demo or meet to discuss your specific needs.