# README
Cloud Reference Notes
AWS Reference Notes is a compilation of the **Note-able** sections of AWS services[^1]. It is compiled by parsing all sections of [AWS Documentation](https://docs.aws.amazon.com/) and extracting specific admonitions (eg. **Note**, **Important**, **Considerations**) which are then organized and compiled here. AWS Reference Notes exists because I observed that any section of the AWS docs that start with a **Note** was something that was worth paying attention to. These sections documented gotchas, limits, and other caveats of a particular service. When not observed, they can take anywhere on the order of hours to weeks to work around. ### Layout Each AWS Service is organized via the following categories ``` - {service} - Common - Topics ``` #### Common Common refers to parts of an AWS service that is shared among all services. Examples of `Common` sections: * Getting Started * Monitoring * Security * Networking * Configuration * Troubleshooting * Resources and Tags * Working with other services #### Topics Topics refers to parts of an AWS service that is service specific. This includes service specific components, features, and runtimes. Examples of `Topics` sections: * ECS Clusters * S3 Clacier Vaults * KMS Keys ### Acknowledgements All content in this site (besides this page) is generated from the official AWS Developer Docs, licensed under the [Creative Commons Attribution 4.0 International](https://github.com/open-guides/og-aws/blob/master/LICENSE.txt) ### Contributing This project is hot off the press as of 2023-05-14. All the notes in this project are auto-generated from [aws-doc-extractor](https://github.com/kevinslin/aws-doc-extractor) See the repo for instructions on contributing. Feel free to also report any issues or feature request in this repo. You can see the [roadmap](https://github.com/users/kevinslin/projects/3/views/1) to see upcoming changes. [^1]: NOTE: AWS Reference Notes currently has 66 services respresented # Amazon EC2 # Common # Monitor {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Monitor memory and disk metrics * **Important**\ The CloudWatch monitoring scripts are deprecated. We strongly recommend that you use the CloudWatch agent to collect metrics and logs. For more information, see [Collect Metrics from Amazon EC2 Instances and On-Premises Servers with the CloudWatch Agent](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html) in the *Amazon CloudWatch User Guide*.\ If you are still migrating from the deprecated monitoring scripts to the agent, and require information about the monitoring scripts, see [Deprecated: Collect metrics using the CloudWatch monitoring scripts](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2/monitoring-scripts-intro.md). # Networking {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Regions and Zones * **Considerations**\ Some AWS resources might not be available in all Regions. Ensure that you can create the resources that you need in the desired Regions before you launch an instance. * **Considerations** ## Instance IP addressing * **Note**\ You can create a VPC with a publicly routable CIDR block that falls outside of the private IPv4 address ranges specified in RFC 1918. However, for the purposes of this documentation, we refer to private IPv4 addresses (or 'private IP addresses') as the IP addresses that are within the IPv4 CIDR range of your VPC. * **Note**\ Instances that access other instances through their public NAT IP address are charged for regional or Internet data transfer, depending on whether the instances are in the same Region. * **Considerations** ## Instance hostname types * **Note**\ Changing the subnet settings doesn't change the configuration of EC2 instances that are already launched in the subnet. * **Important**\ To change the **Use resource based naming as guest OS hostname** setting, you must first stop the instance. To change the **Answer DNS hostname IPv4 (A record) request** or **Answer DNS hostname IPv6 (AAAA record) requests** settings, you don't have to stop the instance. To modify any of the settings for non-EBS backed EC2 instance types, you cannot stop the instance. You must terminate the instance and launch a new instance with the desired Hostname type and DNS Hostname configurations. ## Bring your own IP addresses * **Note**\ The following steps describe how to bring your own IP address range for use in Amazon EC2 only. For steps to bring your own IP address range for use in AWS Global Accelerator, see [Bring your own IP addresses (BYOIP)](https://docs.aws.amazon.com/global-accelerator/latest/dg/using-byoip.html) in the *AWS Global Accelerator Developer Guide*. * **Note**\ A ROA is not required for non-publicly advertised IPv6 address space. * **Note**\ The Common Name is not needed for AWS provisioning. It can be any internal or public domain name. * **Important**\ If you are creating a ROA object for Amazon VPC IP Address Manager (IPAM), when you create the ROAs, for IPv4 CIDRs you must set the maximum length of an IP address prefix to `/24`. For IPv6 CIDRs, if you are adding them to an advertisable pool, the maximum length of an IP address prefix must be `/48`. This ensures that you have full flexibility to divide your public IP address across AWS Regions. IPAM enforces the maximum length you set. For more information about BYOIP addresses to IPAM, see [Tutorial: BYOIP address CIDRs to IPAM](https://docs.aws.amazon.com/vpc/latest/ipam/tutorials-byoip-ipam.html) in the *Amazon VPC IPAM User Guide*. * **Note**\ This step is not required for non-publicly advertised IPv6 address space. * **Important**\ You can only specify whether an address range is publicly advertised during provisioning. You cannot change the advertisable status later on. ## Elastic IP addresses * **Note**\ If you contacted AWS support to set up reverse DNS for an Elastic IP (EIP) address, you can remove the reverse DNS, but you can’t release the Elastic IP address because it’s been locked by AWS support. To unlock the Elastic IP address, contact [AWS Support](https://console.aws.amazon.com/support/home#/). Once the Elastic IP address is unlocked, you can release the Elastic IP address. * **Considerations** * **Note**\ If you receive the following error when you run the command, you can submit a [Request to remove email sending limitations](http://aws.amazon.com/forms/ec2-email-limit-rdns-request) to customer support for assistance.\ *The address with allocation id cannot be released because it is locked to your account*. ## Network interfaces * **Important**\ For EC2 instances in an IPv6-only subnet, if you attach a secondary network interface to the instance, the private DNS hostname of the second network interface will resolve to the first IPv6 address on the instance's first network interface. For more information about EC2 instance private DNS hostnames, see [Amazon EC2 instance hostname types](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2/ec2-instance-naming.md). ## Elastic Fabric Adapter * **Note**\ The OS-bypass capabilities of EFAs are not supported on Windows instances. If you attach an EFA to a Windows instance, the instance functions as an Elastic Network Adapter, without the added EFA capabilities. * **Note**\ Libfabric is a core component of the OpenFabrics Interfaces (OFI) framework, which defines and exports the user-space API of OFI. For more information, see the [Libfabric OpenFabrics](https://ofiwg.github.io/libfabric/) website. * **Note**\ Ubuntu 20.04 supports peer direct support when used with `dl1.24xlarge` instances. ## Placement groups * **Note**\ You can tag a placement group on creation using the command line tools only. ## Network MTU * **Important**\ Path MTU Discovery does not guarantee that jumbo frames will not be dropped by some routers. An internet gateway in your VPC will forward packets up to 1500 bytes only. 1500 MTU packets are recommended for internet traffic. # Security {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## AWS PrivateLink * **Important**\ When a non-default policy is applied to an interface VPC endpoint for Amazon EC2, certain failed API requests, such as those failing from `RequestLimitExceeded`, might not be logged to AWS CloudTrail or Amazon CloudWatch. # Resources and tags {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Recycle Bin * **Note**\ Some resources might still appear in the Recycle Bin console or in the AWS CLI and API output for a short period after their retention periods have expired and they have been permanently deleted. You are not billed for these resources. Billing stops as soon as the retention period expires. ## List and filter your resources * **Important**\ You cannot use a private window in Firefox to access Amazon EC2 Global View. ## Tag your resources * **Note**\ After you delete a resource, its tags might remain visible in the console, API, and CLI output for a short period. These tags will be gradually disassociated from the resource and be permanently deleted. * **Note**\ If you've just enabled reporting, data for the current month is available for viewing after 24 hours. * **Note**\ Elastic IP addresses that are tagged do not appear on your cost allocation report. * **Note**\ The way you enter JSON-formatted parameters on the command line differs depending on your operating system.\ Linux, macOS, or Unix and Windows PowerShell – Use single quotes (') to enclose the JSON data structure. Windows – Omit the single quotes when using the commands with the Windows command line. For more information, see [Specifying parameter values for the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-using-param.html). ## Service quotas * **Note**\ This restriction does not apply to outbound traffic sent over port 25 to:\ IP addresses in the primary CIDR block of the VPC in which the originating network interface exists. IP addresses in the CIDRs defined in [RFC 1918](https://datatracker.ietf.org/doc/html/rfc1918), [RFC 6598](https://datatracker.ietf.org/doc/html/rfc6598), and [RFC 4193](https://datatracker.ietf.org/doc/html/rfc4193). # Troubleshoot {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Connect to your instance * **Note**\ If you get an error stating that the file system is corrupt, run the following command to use the **fsck** utility to check the file system and repair any issues: * **Note**\ If the name of your new key pair and corresponding private key file is different from the name of the original key pair, ensure that you specify the name of the new private key file when you connect to your instance. ## Stop your instance * **Note**\ You can force an instance to stop using the console only while the instance is in the `stopping` state. You can force an instance to stop using the AWS CLI while the instance is in any state, except `shutting-down` and `terminated`. * **Important**\ Creating a replacement instance is recommended if it's registering [system status checks](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2/monitoring-instances-status-check.md) only, as instance status checks will result in the AMI copying over an exact replica of the broken OS. Once you've confirmed the status message, create the AMI and launch a new instance using the new AMI. ## Troubleshoot an unreachable instance * **Note**\ Only the most recent 64 KB of posted output is stored, which is available for at least 1 hour after the last posting. ## Boot from the wrong volume * **Important**\ If you intend to detach the volume with the new label and return it to another instance to use as the root volume, you must perform the above procedure again and change the volume label back to its original value. Otherwise, the other instance does not boot because the ramdisk is unable to find the volume with the label `/`. ## Send a diagnostic interrupt * **Note**\ By default, the crash dump file is saved to `/var/crash/`. To change the location, modify the `/etc/kdump.conf` file using your preferred text editor. * **Note**\ By default, the crash dump file is saved to `/var/crash/`. To change the location, modify the `/etc/kdump.conf` file using your preferred text editor. * **Note**\ On instances based on Intel and AMD processors, the `send-diagnostic-interrupt` command sends an *unknown non-maskable interrupt* (NMI) to the instance. You must configure the kernel to crash when it receives the unknown NMI. Add the following to your configuration file. # Topics # Amazon Machine Images {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Find a Linux AMI * **Important**\ Omitting the `--owners` flag from the describe-images command returns all images for which you have launch permissions, regardless of ownership. * **Note**\ Running instances are not affected when you change the AMI ID pointed to by the Systems Manager parameter. ## Paid AMIs * **Important**\ Amazon DevPay is no longer accepting new sellers or products. AWS Marketplace is now the single, unified e-commerce platform for selling software and services through AWS. For information about how to deploy and sell software from AWS Marketplace, see [Selling in AWS Marketplace](http://aws.amazon.com/marketplace/partners/management-tour). AWS Marketplace supports AMIs backed by Amazon EBS. * **Important**\ You can't use a support product with Reserved Instances. You always pay the price that's specified by the seller of the support product. ## Use encryption with EBS-backed AMIs * **Note**\ Enabling [encryption by default](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2/EBSEncryption.md#encryption-by-default) has the same effect as setting the `Encrypted` parameter to `true` for all snapshots in the AMI. * **Note**\ You can also copy an image with multiple snapshots and configure the encryption state of each individually. ## Amazon Linux * **Important**\ Your instance must have access to the internet in order to access the repository. * **Important**\ If you lock to a version of the repositories that is not the latest, you do not receive further updates. To receive a continuous flow of updates, you must use the latest AMI, or consistently update your AMI with the repositories pointed to latest. * **Note**\ This command does not remove packages that were installed as dependencies of the extra. * **Important**\ This command is intended for advanced users. Improper usage of this command could cause package compatibility conflicts. ## User provided kernels * **Note**\ We continue to provide hd00 AKIs for backward compatibility in Regions where they were previously available. ## Configure the MATE desktop connection * **Important**\ `xrdp` is the remote desktop software bundled in the AMI. By default, `xrdp` uses a self-signed TLS certificate to encrypt remote desktop sessions. Neither AWS nor the `xrdp` maintainers recommend using self-signed certificates in production. Instead, obtain a certificate from an appropriate certificate authority (CA) and install it on your instances. For more information about TLS configuration, see [TLS security layer](https://github.com/neutrinolabs/xrdp/wiki/TLS-security-layer) on the `xrdp` wiki. * **Note**\ This command generates a certificate that is valid for 365 days. # Instances {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Instances and AMIs * **Note**\ Some AWS resources, such as Amazon EBS volumes and Elastic IP addresses, incur charges regardless of the instance's state. For more information, see [Avoiding Unexpected Charges](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/checklistforunwantedcharges.html) in the *AWS Billing User Guide*. For more information about Amazon EBS costs, see [Amazon EBS pricing](http://aws.amazon.com/ebs/pricing/). ## Instance types * **Note**\ Previous generation instances are still fully supported and retain the same features and functionality. We encourage you to use the latest generation of instances to get the best performance. ## Mac instances * **Note**\ The **unit of billing** is the **dedicated host**. The instances running on that host have no additional charge. * **Important**\ Multiple users can access the OS simultaneously. Typically there is a 1:1 user:GUI session due to the built-in Screen Sharing service on port 5900. Using SSH within macOS supports multiple sessions up until the "Max Sessions" limit in the sshd\_config file. * **Note**\ macOS 10.14 and later only allows control if Screen Sharing is enabled through [System Preferences](https://support.apple.com/guide/remote-desktop/enable-remote-management-apd8b1c65bd/mac). * **Note**\ The current build of displayplacer is not supported on M1 Mac instances. * **Note**\ With this procedure on macOS Big Sur, you can only perform minor updates such as updating from macOS Big Sur 11.7.3 to macOS Big Sur 11.7.4. For macOS Monterey or above, you can perform major software updates. * **Note**\ Do not exit this SSH session until the following VNC connection and GUI steps are completed. When the instance is restarted, the connection will close automatically. * **Note**\ EC2 System Monitoring for macOS is not currently supported on M1 Mac instances. ## Instance lifecycle * **Note**\ The table indicates billing for instance usage only. Some AWS resources, such as Amazon EBS volumes and Elastic IP addresses, incur charges regardless of the instance's state. For more information, see [Avoiding Unexpected Charges](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/checklistforunwantedcharges.html) in the *AWS Billing User Guide*. * **Note**\ Rebooting an instance doesn't start a new instance billing period because the instance stays in the `running` state. ## Amazon EI * **Note**\ Starting April 15, 2023, AWS will not onboard new customers to Amazon Elastic Inference (EI), and will help current customers migrate their workloads to options that offer better price and performance. After April 15, 2023, new customers will not be able to launch instances with Amazon EI accelerators in Amazon SageMaker, Amazon ECS, or Amazon EC2. However, customers who have used Amazon EI at least once during the past 30-day period are considered current customers and will be able to continue using the service. # Storage {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-ec2-user-guide.git) {% endhint %} ## Instance volume limits * **Important**\ Attaching more than 40 volumes to a Linux instance is supported on a best effort basis only and is not guaranteed. ## Root device volume * **Important**\ Only the following instance types support an instance store volume as the root device: C3, D2, G2, I2, M3, and R3. ## Block device mappings * **Note**\ For M3 instances, you must specify instance store volumes in the block device mapping for the instance when you launch it. When you launch an M3 instance, instance store volumes specified in the block device mapping for the AMI may be ignored if they are not specified as part of the instance block device mapping. * **Note**\ For instances launched before the release of the 2009-10-31 API, AWS can't display the block device mapping. You must detach and reattach the volumes so that AWS can display the block device mapping. ## Torn write prevention * **Note**\ You do not need to disable I/O merging for MySQL and MariaDB workloads. * **Note**\ The command returns the vendor-specific information in hex with ASCII interpretation. You might need to build a tool, similar to `ebsnvme-id`, into your applications that can read and parse the output. * **Note**\ The bytes in the following table represent the offset in bytes from the beginning of the NVMe namespace vendor specific data. * **Note**\ There is no performance impact on workloads that do not support torn write prevention. You do not need to make any changes for these workloads.\ Workloads that do support torn write prevention, but are not configured to use it, continue to use the doublewrite buffer and do not receive any performance benefits. * **Note**\ For MySQL and MariaDB, you must use `-C 16384` to match the database page size. Setting allocation granularity to a value other than a multiple of the page size can result in allocations that might be mismatched with torn write prevention boundaries of the storage device. * **Important**\ If you are using Logical Volume Manager (LVM) or other storage virtualization layer, make sure that the starting offsets of the volumes are aligned on 16 KiB multiples. This is relative to the underlying NVMe storage to account for the metadata headers and superblocks used by the storage virtualization layer. If you add an offset to the LVM physical volume, it can cause misalignment between the file system allocations and the NVMe device's offsets, which would invalidate torn write prevention. For more information, see `--dataalignmentoffset` in the [Linux manual page](https://man7.org/linux/man-pages/man8/pvcreate.8.html). # Amazon EC2 Image Builder # Common # Topics # Get started {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/ec2-image-builder-user-guide.git) {% endhint %} ## Prerequisites * **Note**\ If you want to copy an image created with Image Builder to another account, you must create the `EC2ImageBuilderDistributionCrossAccountRole` role in all of the target accounts, and attach the [Ec2ImageBuilderCrossAccountDistributionAccess policy](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/security-iam-awsmanpol.md#sec-iam-manpol-Ec2ImageBuilderCrossAccountDistributionAccess) managed policy to the role. For more information, see [Share EC2 Image Builder resources](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/manage-shared-resources.md). ## Create an image pipeline (AMI) * **Note**\ To specify a subnet to use for a private VPC, you can create your own custom infrastructure configuration, or use settings that you have already created. * **Tip**\ To prevent dependency errors when you delete resources, make sure to delete your resources in the following order:\ Image pipeline Image recipe All remaining resources ## Create an image pipeline (Docker) * **Tip**\ To prevent dependency errors when you delete resources, make sure to delete your resources in the following order:\ Image pipeline Image recipe All remaining resources # AWSTOE component manager {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/ec2-image-builder-user-guide.git) {% endhint %} ## Get started with AWSTOE * **Important**\ AWSTOE is invoked directly from its download location. There is no need for a separate install step. This also means that AWSTOE can make changes to the local environment.\ To ensure that you isolate changes during component development, we recommend that you use an EC2 instance to develop and test AWSTOE components. * **Note**\ The AWSTOE application can validate only the component syntax for the current operating system. For example, when running `awstoe.exe` on Windows, you cannot validate the syntax for a Linux document that uses the `ExecuteBash` action module. ## Use component documents * **Tip**\ The service that uses your component to build an image might implement rules about what phases to use for their build process, and when those phases are allowed to run. This is important to consider when you design your component. * **Note**\ The log folder might also contain other temporary files that are not covered here. ## Action modules * **Note**\ All action modules use the same account as the Systems Manager agent when they run, which is `root` on Linux, and `NT Authority\SYSTEM` on Windows. * **Note**\ All folders in the destination path must exist prior to download, or the download fails. * **Note**\ For the following examples, the Windows folder path can be replaced with a Linux path. For example, `C:\myfolder\package.zip` can be replaced with `/myfolder/package.zip`. ## STIG components * **Note**\ The STIG-Build-Windows-Medium components include all STIG settings that AWSTOE applies for STIG-Build-Windows-Low components, in addition to the STIG settings that apply specifically for Category II vulnerabilities. * **Note**\ The STIG-Build-Windows-High components include all STIG settings that AWSTOE applies for STIG-Build-Windows-Low and STIG-Build-Windows-Medium components, in addition to the STIG settings that apply specifically for Category I vulnerabilities. * **Note**\ The STIG-Build-Linux-Medium components include all STIG settings that AWSTOE applies for STIG-Build-Linux-Low components, in addition to the STIG settings that apply specifically for Category II vulnerabilities. * **Note**\ The STIG-Build-Linux-High components include all STIG settings that AWSTOE applies for STIG-Build-Linux-Low and STIG-Build-Linux-Medium components, in addition to the STIG settings that apply specifically for Category I vulnerabilities. * **Note**\ You can review the reports with one of the [STIG Viewing Tools](https://public.cyber.mil/stigs/srg-stig-tools/). These tools are available online via the DoD Cyber Exchange. * **Note**\ AWSTOE currently supports SCAP compliance validation for Windows Server 2012 R2, 2016, and 2019. * **Note**\ AWSTOE currently supports SCAP compliance validation for RHEL 7/8 and Ubuntu 18. The SCC application currently supports the x86 architecture for validation. ## Command reference * **Note**\ Some AWSTOE action modules require elevated permissions to run on a Linux server. To use elevated permissions, prefix the command syntax with sudo, or run the sudo su command one time when you log in before running the commands linked below. For more information about AWSTOE action modules, see [Action modules supported by AWSTOE component manager](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/toe-action-modules.md). # Manage resources {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/ec2-image-builder-user-guide.git) {% endhint %} ## Components * **Note**\ When a component that is managed by Amazon reaches the end of its support lifespan, it is no longer maintained. About four weeks before this occurs, any accounts that are using the component receive notification, and a list of the affected recipes in their account from their AWS Health Dashboard. To learn more about AWS Health, see [AWS Health User Guide](https://docs.aws.amazon.com/health/latest/ug/). * **Note**\ While AWSTOE allows you to define many phases in a component document, Image Builder has strict rules about what phases it runs, and during which stages it runs them. For a component to run during the build stage, the component document must define at least one of these phases: `build` or `validate`. For a component to run during the test stage, the component document must define the `test` phase, and no other phases.\ Since Image Builder runs the stages independently, chaining references in AWSTOE documents cannot cross stage boundaries. You cannot chain a value from a phase that runs in the build stage to a phase that runs in the test stage. You can, however, define input parameters to the intended target, and pass in values through the command line. For more information about setting component parameters in your Image Builder recipes, see [Manage AWSTOE component parameters with EC2 Image Builder](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/manage-component-parameters.md). * **Tip**\ When you have many components to keep track of, tagging helps you to identify a specific component or version based on the tags you've assigned to it. For more information about tagging your resources using Image Builder commands in the AWS CLI, see the [Tag resources](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/tag-resources.md) section of this guide. ## Recipes * **Tip**\ You can use Amazon managed components in your recipes, or you can develop your own custom components with the AWS Task Orchestrator and Executor (AWSTOE) application. To get started, see [Get started with AWSTOE](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/toe-get-started.md). ## Images * **Tip**\ When you have multiple resources of the same type, tagging helps you to identify a specific resource based on the tags you've assigned to it. For more information about tagging your resources using Image Builder commands in the AWS CLI, see the [Tag resources](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/tag-resources.md) section of this guide. ## Infrastructure configurations * **Tip**\ When you have multiple resources of the same type, tagging helps you to identify a specific resource based on the tags you've assigned to it. For more information about tagging your resources using Image Builder commands in the AWS CLI, see the [Tag resources](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/tag-resources.md) section of this guide. ## Distribution settings * **Tip**\ When you have multiple resources of the same type, tagging helps you to identify a specific resource based on the tags you've assigned to it. For more information about tagging your resources using Image Builder commands in the AWS CLI, see the [Tag resources](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_ec2_image_builder/tag-resources.md) section of this guide. ## Import and export VM images * **Note**\ You must include the `file://` notation at the beginning of the JSON file path. The path for the JSON file should follow the appropriate convention for the base operating system where you are running the command. For example, Windows uses the backslash (\\) to refer to the directory path, and Linux uses the forward slash (/). ## Share resources * **Note**\ To set the correct policies for sharing and unsharing a resource, the resource owner must have `imagebuilder:put*` permissions. * **Note**\ To unshare a component, image, or recipe, the consumer cannot have any dependencies on them. The consumer must remove any dependencies on the shared resources before the owner can unshare them. ## Delete resources * **Tip**\ To prevent dependency errors when you delete resources, make sure to delete your resources in the following order:\ Image pipeline Image recipe All remaining resources * **Important**\ Make sure there are no recipes that reference the component build version in any way before you delete it. Failing to do so could cause pipeline failures. # Manage pipelines {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/ec2-image-builder-user-guide.git) {% endhint %} ## Use cron expressions * **Note**\ Cron expressions use Universal Coordinated Time (UTC). For more information about UTC time, and to find the offset for your time zone, see [Time Zone Abbreviations – Worldwide List](https://www.timeanddate.com/time/zones/). * **Tip**\ If you don't want your pipeline job to extend into the next day while it's running, make sure that you factor in time for your build when you specify the start time. * **Tip**\ If you don't want your pipeline job to extend into the next day while it's running, make sure that you factor in time for your build when you specify the start time. ## Use EventBridge rules * **Note**\ Event buses are specific to a Region. The rule and the target must be in the same Region. * **Note**\ To learn more about settings for rate expression rules that are not covered in this example, see [Rate expressions](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule-schedule.html#eb-rate-expressions) in the *Amazon EventBridge User Guide*. # Integrate products and services {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/ec2-image-builder-user-guide.git) {% endhint %} ## AWS Marketplace * **Note**\ Data product providers must meet the AWS Data Exchange eligibility requirements. For more information, see [Providing Data Products on AWS Data Exchange](https://docs.aws.amazon.com/marketplace/latest/userguide/providing-data-sets.html) in the *AWS Data Exchange User Guide*. * **Note**\ If you're looking for a product that you just subscribed to, and you don't see it in the list, use the refresh button at the top of the tab to refresh your results. It might take a few minutes for a new subscription to appear in the list. * **Note**\ If your image product subscription includes an AWSTOE build component, you can select it from the **Build components** list. Select `Third party managed` from the component owner type list to see it. If your product subscription includes an AWSTOE test component, follow the same procedure for the **Test components** list. ## Amazon Simple Notification Service * **Note**\ If your SNS topic is encrypted, the key that encrypts this topic must reside in the account where the Image Builder service runs. Image Builder can't send notifications to SNS topics that are encrypted with keys from other accounts. # Security in EC2 Image Builder {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/ec2-image-builder-user-guide.git) {% endhint %} ## Best practices * **Note**\ We recommend that you configure all EC2 instances that Image Builder launches from a pipeline build to use IMDSv2 so that instance metadata retrieval requests require a signed token header. * **Important**\ If you override **User data** in your recipe, the script doesn't run. In that case, make sure that you include a command in your user data that creates an empty file named `perform_cleanup`. Image Builder detects this file and runs the clean-up script prior to creating the new image. * **Important**\ Skipping sections in the clean up script can result in sensitive information, such as owner account details or SSH keys being included in the final image, and in any instance launched from that image. You might also experience problems with launching in different Availability Zones, Regions, or accounts. * **Note**\ The files that you create to skip a section of the clean up script should not include a file extension. For example, if you want to skip the `CLOUD_INIT_FILES` section of the script, but you create a file named `skip_cleanup_cloudinit_files.txt`, Image Builder will not recognize the skip file. # AWS End-of-Support Migration Program (EMP) for Windows Server # Common # Topics # Get started {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/emp-user-guide.git) {% endhint %} ## Package an application * **Note**\ If the criteria for standard packaging is met, then we recommend that you apply this packaging model instead of the GRP packaging model. * **Note**\ This step is required only if there is a benefit to test the package in an on-premises environment to validate that it works as expected before migrating the application to AWS, or if it is required to troubleshoot any issues. It helps identify whether issues are the result of the AWS environment setup, the EMP package, or the target operating system. ## Deploy an application * **Note**\ If you do not specify a name when you create a profile, it will default to `default` by both the AWS CLI and AWS Tools for PowerShell. You are not required to update the `metadata.json` file found in the root of the EMP package. If you specify a new name for the profile at a later time, update the `AWSProfileName` property in the `metadata.json` file. # Working with EMP packages {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/emp-user-guide.git) {% endhint %} ## Compatibility package features * **Note**\ This feature, when enabled, causes the printer driver host for 32-bit applications (`SPLWOW64.exe`) to fail, and must be excluded if the packaged application launches it. * **Note**\ Apply the Microsoft server naming conventions described at . These are the only naming conventions supported. ## Edit, upgrade, and maintain packages * **Important**\ You must use the Package Editor to update an EMP package on the same architecture on which the original package was created. For example, if the package was created on an x86 machine, then the Editor must update the package on an x86 machine. ## Update a deployed package * **Important**\ If you attempt to use the `/deploydir` switch when a package has already been deployed, a `Failed to deploy' exit code -1` error will be returned. The `/update` switch must be used to update the package to the latest version, or the `/uninstall` switch must be used to remove the package first. * **Note**\ The `/update` switch preserves any values that appear in the registry that are not specified in the source file. If the file type associations source file (`FileAssociations.xml`) in the new package is different from the one in the currently deployed package, `/update` deletes the registry values that do not appear in `FileAssociations.xml` and updates values and types that have changed. ## Uninstall a package * **Note**\ If you run this command from the deployed location, the uninstall will be incomplete. Verify that the package is uninstalled from the correct source path. ## Enable logging * **Note**\ `LocalAppData` resolves to a special location for the `SYSTEM` account: `C:\Windows\System32\config\systemprofile\AppData\Local`. ## Enable out-of-process COM * **Note**\ The `COMVirtualization` feature is not required if the application uses in-process COM objects. Applications that use in-process COM objects behave as expected without enabling the feature. ## Exclude or detach a process * **Important**\ The virtualization and redirection engine of the package will detach from the parent process once virtualization is complete. Any child processes will not be virtualized, and the detached process will not benefit from `DEPOptOut` or `HandleInvalidHandle`, even if these features are enabled. # AWS ParallelCluster # Common # Topics # AWS ParallelCluster version 2 {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-parallelcluster-user-guide.git) {% endhint %} ## Development * **Important**\ The following sections include instructions for using a custom version of the cookbook recipes and a custom AWS ParallelCluster node package. This information covers an advanced method of customizing AWS ParallelCluster, with potential issues that can be hard to debug. The AWS ParallelCluster team highly recommends using the scripts in [Custom Bootstrap Actions](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_parallel_cluster/pre_post_install.md) for customization, because post-install hooks are generally easier to debug and more portable across releases of AWS ParallelCluster. ## Troubleshooting * **Note**\ Starting with version 2.11.5, AWS ParallelCluster doesn't support the use of SGE or Torque schedulers. # AWS Serverless Application Model (AWS SAM) # Common # Getting started {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Installing the AWS SAM CLI * **Important**\ You must have Homebrew installed on your Linux machine. For install instructions, see [Installing Homebrew to use with the AWS SAM CLI](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/install-homebrew.md). * **Note**\ To invoke the AWS SAM CLI with the sam command, the installer automatically creates a symlink between `/usr/local/bin/sam` and `/usr/local/aws-sam-cli/sam`. * **Important**\ You must have Homebrew installed on your machine. For install instructions, see [Installing Homebrew to use with the AWS SAM CLI](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/install-homebrew.md). ## Tutorial: Hello World application * **Note**\ If you don't have Python on your local machine, use the sam build --use-container command instead. The AWS SAM CLI will create a Docker container that includes your function's runtime and dependencies. This command requires Docker on your local machine. To install Docker, see [Installing Docker](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/install-docker.md). * **Note**\ This step requires AWS credentials configuration. For more information, see [Step 5: Use the AWS CLI to configure AWS credentials](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/prerequisites.md#prerequisites-configure-credentials) in [AWS SAM prerequisites](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/prerequisites.md). * **Note**\ This step is optional since it requires Docker on your local machine. * **Important**\ To use the AWS SAM CLI for local testing, you must have Docker installed and configured. For more information, see [Installing Docker](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/install-docker.md). # Monitoring {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Working with logs * **Note**\ The `sam logs` command works for all AWS Lambda functions, not just the ones you deploy using AWS SAM. # Topics # AWS SAM specification {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Generated resources * **Note**\ The `LogicalIds` of some generated resources include a unique hash value to avoid namespace clashes. The `LogicalIds` of these resources are derived when the stack is created. You can retrieve them only after the stack has been created using the AWS Management Console, AWS CLI, or one of the AWS SDKs. We don't recommend referencing these resources by `LogicalId` because the hash values might change. * **Note**\ Not all generated AWS CloudFormation resources have referenceable properties. For those resources, you must use the `LogicalId`. # Authoring {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Validating AWS SAM template files * **Note**\ The `sam validate` command requires AWS credentials to be configured. For more information, see [Configuration and Credential Files](https://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html). ## Using nested applications * **Note**\ The maximum number of applications that can be nested in a serverless application is 200.\ The maximum number of parameters a nested application can have is 60. * **Important**\ Applications that contain nested applications hosted in the AWS Serverless Application Repository inherit the nested applications' sharing restrictions.\ For example, suppose an application is publicly shared, but it contains a nested application that's only privately shared with the AWS account that created the parent application. In this case, if your AWS account doesn't have permission to deploy the nested application, you aren't able to deploy the parent application. For more information about permissions to deploy applications, see [Application Deployment Permissions](https://docs.aws.amazon.com/serverlessrepo/latest/devguide/serverless-app-consuming-applications.html#application-deployment-permissions) and [Publishing Applications](https://docs.aws.amazon.com/serverlessrepo/latest/devguide/serverless-app-publishing-applications.html) in the *AWS Serverless Application Repository Developer Guide*. * **Note**\ When you deploy an application that contains nested applications, you must acknowledge that. You do this by passing CAPABILITY\_AUTO\_EXPAND to the [CreateCloudFormationChangeSet API](https://docs.aws.amazon.com/goto/WebAPI/serverlessrepo-2017-09-08/CreateCloudFormationChangeSet),or using the AWS CLI command.\ For more information about acknowledging nested applications, see [Acknowledging IAM Roles, Resource Policies, and Nested Applications when Deploying Applications](https://docs.aws.amazon.com/serverlessrepo/latest/devguide/acknowledging-application-capabilities.html) in the *AWS Serverless Application Repository Developer Guide*. ## Orchestrating applications * **Note**\ To manage AWS SAM templates that contain Step Functions state machines, you must use version 0.52.0 or later of the AWS SAM CLI. To check which version you have, execute the command `sam --version`. ## Code signing * **Note**\ In order to successfully sign your code with the `sam package` or `sam deploy` commands, versioning must be enabled for the Amazon S3 bucket you use with these commands. If you are using the Amazon S3 Bucket that AWS SAM creates for you, versioning is enabled automatically. For more information about Amazon S3 bucket versioning and instructions for enabling versioning on an Amazon S3 bucket that you provide, see [Using versioning in Amazon S3 buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html) in the *Amazon Simple Storage Service User Guide*. * **Note**\ In order to successfully sign your code with the `sam package` or `sam deploy` commands, versioning must be enabled for the Amazon S3 bucket you use with these commands. If you are using the Amazon S3 Bucket that AWS SAM creates for you, versioning is enabled automatically. For more information about Amazon S3 bucket versioning and instructions for enabling versioning on an Amazon S3 bucket that you provide, see [Using versioning in Amazon S3 buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html) in the *Amazon Simple Storage Service User Guide*. # Building {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Building applications * **Note**\ If you specify a multi-architecture base image in your Dockerfile, AWS SAM builds your container image for your host machine's architecture. To build for a different architecture, specify a base image that uses the specific target architecture. ## Building layers * **Note**\ When you create a custom layer, AWS Lambda depends on environment variables to find your layer code. Lambda runtimes include paths in the `/opt` directory where your layer code is copied into. Your project's build artifact folder structure must match the runtime's expected folder structure so your custom layer code can be found.\ For example, for Python you can place your code in the `python/` subdirectory. For NodeJS, you can place your code in the `nodejs/node_modules/` subdirectory.\ For more information, see [Including library dependencies in a layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-path) in the *AWS Lambda Developer Guide*. * **Note**\ If you don't include the `Metadata` resource attribute section, AWS SAM doesn't build the layer. Instead, it copies the build artifacts from the location specified in the `CodeUri` property of the layer resource. For more information, see the [ContentUri](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/sam-resource-layerversion.md#sam-layerversion-contenturi) property of the `AWS::Serverless::LayerVersion` resource type. ## Building custom runtimes * **Note**\ We recommend building Lambda functions with Cargo Lambda. To learn more, see [Building Rust Lambda functions with Cargo Lambda](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/building-rust.md). # Testing and debugging {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Invoking functions locally * **Note**\ The sam local invoke command corresponds to the AWS Command Line Interface (AWS CLI) command . You can use either command to invoke a Lambda function. ## Running API Gateway locally * **Note**\ *Hot reloading* is when only the files that changed are refreshed, and the state of the application remains the same. In contrast, *live reloading* is when the entire application is refreshed, and the state of the application is lost. ## Step-through debugging Lambda functions locally * **Note**\ If you're using `sam local start-api`, the local API Gateway instance exposes all of your Lambda functions. However, because you can specify a single debug port, you can only debug one function at a time. You need to call your API before the AWS SAM CLI binds to the port, which allows the debugger to connect. # Deploying {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Using OIDC with AWS SAM pipeline * **Note**\ If the identity provider (IdP) resource already exists in your AWS account, AWS SAM references it instead of creating a new resource. # Publishing {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Metadata section properties * **Note**\ AWS CloudFormation [intrinsic functions](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html) aren't supported by the `AWS::ServerlessRepo::Application` metadata key. # Example applications {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Process Amazon S3 events * **Note**\ With this example application, you perform steps in a slightly different order than in previous examples. The reason for this is that this example requires that AWS resources are created and IAM permissions are configured *before* you can test the Lambda function locally. We're going to leverage AWS CloudFormation to create the resources and configure the permissions for you. Otherwise, you would need to do this manually before you can test the Lambda function locally.\ Because this example is more complicated, be sure that you're familiar with installing the previous example applications before executing this one. # AWS CDK {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Getting started * **Note**\ The AWS SAM CLI supports AWS CDK v1 starting from version 1.135.0 and AWS CDK v2 starting from version 2.0.0. # AWS SAM reference {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-sam-developer-guide.git) {% endhint %} ## Installing Docker * **Note**\ Docker is required only for testing your applications locally and for building deployment packages using the `--use-container` option. * **Note**\ On Linux, to build and run Lambda functions with a different instruction set architecture than your host machine, there are additional steps to configure Docker. For example, to run `arm64` functions on an `x86_64` machine, you can run the following command to configure the Docker daemon: `docker run --rm --privileged multiarch/qemu-user-static --reset -p yes`. * **Note**\ Docker Desktop is officially supported, but starting with AWS SAM CLI version 1.47.0, you can use alternatives as long as they use the Docker runtime. * **Note**\ AWS SAM officially supports Docker Desktop. However, starting with AWS SAM CLI version 1.47.0, you can use alternatives as long as they use the Docker runtime. ## Installing Homebrew * **Note**\ Installing Homebrew changes your environment's default Python version to the one that Homebrew installs. ## Image repositories * **Note**\ Prior to version 1.22.0 of the AWS SAM CLI, DockerHub was the default repository that the AWS SAM CLI pulled the container image from. Starting with version 1.22.0, the default repository changed to Amazon Elastic Container Registry Public (Amazon ECR Public). To pull a container image from a repository other than the current default, you can use the [sam build](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_model_aws_sam/sam-cli-command-reference-sam-build.md) command with the --build-image option. The examples at the end of this topic show how to build applications using DockerHub repository images. * **Note**\ Amazon ECR Public replaced DockerHub starting with the AWS SAM CLI version 1.22.0. If you are using an earlier version of the AWS SAM CLI, we recommend that you upgrade. ## Deploying gradually * **Note**\ If you enable gradual deployments through your AWS SAM template, a CodeDeploy resource is automatically created for you. You can view the CodeDeploy resource directly through the AWS Management Console. ## Important notes * **Important**\ Although the `pip install aws-sam-cli` command also works on 64-bit Windows, we recommend that you use the [64-bit MSI](https://github.com/aws/aws-sam-cli/releases/latest/download/AWS_SAM_CLI_64_PY3.msi) to install AWS SAM CLI on 64-bit systems. # AWS Serverless Application Repository # Common # Topics # Publishing Applications {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-serverlessrepo-developer-guide.git) {% endhint %} ## Using AWS SAM with the AWS Serverless Application Repository * **Important**\ If your application template contains one of the following custom IAM roles or resource policies, your application doesn't show up in search results by default. Also, customers need to acknowledge the application's custom IAM roles or resource policies before they can deploy the application. For more information, see [Acknowledging Application Capabilities](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_repository/acknowledging-application-capabilities.md).\ The list of resources that this applies to are:\ \*\*IAM roles: \*\*[AWS::IAM::Group](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iam-group.html), [AWS::IAM::InstanceProfile](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-instanceprofile.html), [AWS::IAM::Policy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-policy.html), and [AWS::IAM::Role](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html). \*\*Resource policies: \*\* [AWS::Lambda::LayerVersionPermission](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-layerversionpermission.html), [AWS::Lambda::Permission](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-permission.html), [AWS::Events::EventBusPolicy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-eventbuspolicy.html), [AWS::IAM:Policy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-policy.html), [AWS::ApplicationAutoScaling::ScalingPolicy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-applicationautoscaling-scalingpolicy.html), [AWS::S3::BucketPolicy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-policy.html), [AWS::SQS::QueuePolicy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sqs-policy.html), and [AWS::SNS:TopicPolicy](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-policy.html). If your application contains the [AWS::Serverless::Application](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template.html#serverless-sam-template-application) resource, customers need to acknowledge that the application contains a **nested application** before they can deploy the application. For more information about nested applications, see [Nested Applications](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-nested-applications.html) in the *AWS Serverless Application Model Developer Guide*. For more information about acknowledging capabilities, see [Acknowledging Application Capabilities](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_serverless_application_repository/acknowledging-application-capabilities.md). ## How to Publish Applications * **Important**\ The information that you enter when you publish an application isn't encrypted. This information includes data such as the author name. If you have personally identifiable information that you don't want to be stored or made public, we recommend that you don't enter this information when publishing your application. * **Note**\ In order to share an application publicly, it must have both the `SemanticVersion` and `LicenseUrl` properties set. ## Verified Author Badge * **Note**\ The verified author badge is displayed for all applications that match both the AWS account and author name. Because AWS accounts can have multiple authors, badges aren't be displayed on applications that have a different author name. To have author badges displayed on applications with different author names, you must submit another request for that author. # Deploying Applications {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-serverlessrepo-developer-guide.git) {% endhint %} ## Application Deployment Permissions * **Important**\ Applications that contain nested applications inherit the nested applications' sharing restrictions. For example, suppose an application is publicly shared, but it contains a nested application that's only privately shared with the AWS account that created the parent application. In this case, if your AWS account doesn't have permission to deploy the nested application, then you aren't able to deploy the parent application. For more information about nested applications, see [Nested Applications](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-nested-applications.html) in the *AWS Serverless Application Model Developer Guide*. # Amazon S3 # Common # Getting started {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Step 1: Create a bucket * **Note**\ You are not charged for creating a bucket. You are charged only for storing objects in the bucket and for transferring objects in and out of the bucket. The charges that you incur through following the examples in this guide are minimal (less than $1). For more information about storage charges, see [Amazon S3 pricing](http://aws.amazon.com/s3/pricing/). ## Step 3: Download an object * **Note**\ You can only download one object at a time. Objects with key names ending with period(s) "." downloaded using the Amazon S3 console will have the period(s) "." removed from the key name of the downloaded object. To download an object with the key name ending in period(s) "." retained in the downloaded object, you must use the AWS Command Line Interface (AWS CLI), AWS SDKs, or REST API. For AWS CLI, REST API, and AWS SDK information and examples, see [Downloading an object](https://docs.aws.amazon.com/AmazonS3/latest/userguide/download-objects.html). ## Access control * **Important**\ If your bucket uses the bucket owner enforced setting for S3 Object Ownership, you must use policies to grant access to your bucket and the objects in it. With the bucket owner enforced setting enabled, requests to set access control lists (ACLs) or update ACLs fail and return the `AccessControlListNotSupported` error code. Requests to read ACLs are still supported. # Tutorials {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Transforming data with S3 Object Lambda * **Note**\ For simplicity, this tutorial creates and uses an IAM user. After completing this tutorial, remember to [Delete the IAM user](#ol-upper-step8-delete-user). For production use, we recommend that you follow the [Security best practices in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) in the *IAM User Guide*. A best practice requires human users to use federation with an identity provider to access AWS with temporary credentials. Another best practice is to require workloads to use temporary credentials with IAM roles to access AWS. To learn about using AWS IAM Identity Center (successor to AWS Single Sign-On) to create users with temporary credentials, see [Getting started](https://docs.aws.amazon.com/singlesignon/latest/userguide/getting-started.html) in the *AWS IAM Identity Center (successor to AWS Single Sign-On) User Guide*.\ This tutorial also uses full-access AWS managed policies. For production use, we recommend that you instead grant only the minimum permissions necessary for your use case, in accordance with [security best practices](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/security-best-practices.md#security-best-practices-prevent). * **Note**\ The preceding example Lambda function loads the entire requested object into memory before transforming it and returning it to the client. Alternatively, you can stream the object from S3 to avoid loading the entire object into memory. This approach can be useful when working with large objects. For more information about streaming responses with Object Lambda Access Points, see the streaming examples in [Working with `GetObject` requests in Lambda](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/olap-writing-lambda.md#olap-getobject-response). ## Detecting and redacting PII data * **Note**\ For simplicity, this tutorial creates and uses an IAM user. After completing this tutorial, remember to [Delete the IAM user](#ol-pii-step8-delete-user). For production use, we recommend that you follow the [Security best practices in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) in the *IAM User Guide*. A best practice requires human users to use federation with an identity provider to access AWS with temporary credentials. Another best practice is to require workloads to use temporary credentials with IAM roles to access AWS. To learn about using AWS IAM Identity Center (successor to AWS Single Sign-On) to create users with temporary credentials, see [Getting started](https://docs.aws.amazon.com/singlesignon/latest/userguide/getting-started.html) in the *AWS IAM Identity Center (successor to AWS Single Sign-On) User Guide*.\ This tutorial also uses full-access policies. For production use, we recommend that you instead grant only the minimum permissions necessary for your use case, in accordance with [security best practices](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/security-best-practices.md#security-best-practices-prevent). ## Hosting video streaming * **Note**\ When you register a domain, it costs money immediately and it's irreversible. You can choose not to auto-renew the domain, but you pay up front and own it for the year. For more information, see [Registering a new domain](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html) in the *Amazon Route 53 Developer Guide*. * **Note**\ When you register a domain, it costs money immediately and it's irreversible. You can choose not to auto-renew the domain, but you pay up front and own it for the year. For more information, see [Registering a new domain](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html) in the *Amazon Route 53 Developer Guide*. * **Important**\ If you want to transfer the domain between AWS accounts or transfer the domain to another registrar, don't delete the domain and expect to immediately reregister it. Instead, see the applicable documentation in the *Amazon Route 53 Developer Guide*:\ [Transferring a domain to a different AWS account](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-transfer-between-aws-accounts.html) [Transferring a domain from Amazon Route 53 to another registrar](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-transfer-from-route-53.html) ## Batch-transcoding videos * **Note**\ Before you start creating an S3 Batch Operations job, make sure that the **Create job from manifest** button is enabled. For more information, see [Check the inventory report for your S3 video source bucket](#batchops-s3-step5-manifest). If the **Create job from manifest** button is disabled, the first inventory report has not been delivered and you must wait until the button is enabled. After you configure Amazon S3 Inventory for your S3 source bucket in [Step 5](#batchops-s3-step5), it can take up to 48 hours to deliver the first inventory report. ## Configuring a static website * **Important**\ Amazon S3 now applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for every bucket in Amazon S3. Starting January 5, 2023, all new object uploads to Amazon S3 are automatically encrypted at no additional cost and with no impact on performance. The automatic encryption status for S3 bucket default encryption configuration and for new object uploads is available in AWS CloudTrail logs, S3 Inventory, S3 Storage Lens, the Amazon S3 console, and as an additional Amazon S3 API response header in the AWS Command Line Interface and AWS SDKs. For more information, see [Default encryption FAQ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-encryption-faq.html). * **Important**\ The following policy is an example only and allows full access to the contents of your bucket. Before you proceed with this step, review [How can I secure the files in my Amazon S3 bucket?](https://aws.amazon.com/premiumsupport/knowledge-center/secure-s3-resources/) to ensure that you understand the best practices for securing the files in your S3 bucket and risks involved in granting public access. * **Note**\ Amazon S3 does not support HTTPS access to the website. If you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3.\ For more information, see [How do I use CloudFront to serve a static website hosted on Amazon S3?](http://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website/) and [Requiring HTTPS for communication between viewers and CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-viewers-to-cloudfront.html). ## Configuring a static website using a custom domain * **Note**\ Amazon S3 website endpoints do not support HTTPS or access points. If you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3.\ For more information, see [How do I use CloudFront to serve a static website hosted on Amazon S3?](http://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website/) and [Requiring HTTPS for communication between viewers and CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-viewers-to-cloudfront.html). * **Important**\ The following policy is an example only and allows full access to the contents of your bucket. Before you proceed with this step, review [How can I secure the files in my Amazon S3 bucket?](https://aws.amazon.com/premiumsupport/knowledge-center/secure-s3-resources/) to ensure that you understand the best practices for securing the files in your S3 bucket and risks involved in granting public access. * **Note**\ Amazon S3 does not support HTTPS access to the website. If you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3.\ For more information, see [How do I use CloudFront to serve a static website hosted on Amazon S3?](http://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website/) and [Requiring HTTPS for communication between viewers and CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-viewers-to-cloudfront.html). * **Note**\ Changes generally propagate to all Route 53 servers within 60 seconds. When propagation is done, you can route traffic to your Amazon S3 bucket by using the names of the alias records that you created in this procedure. * **Note**\ Changes generally propagate to all Route 53 servers within 60 seconds. When propagation is done, you can route traffic to your Amazon S3 bucket by using the names of the alias records that you created in this procedure. # Security {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Data encryption * **Important**\ Amazon S3 now applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for every bucket in Amazon S3. Starting January 5, 2023, all new object uploads to Amazon S3 are automatically encrypted at no additional cost and with no impact on performance. The automatic encryption status for S3 bucket default encryption configuration and for new object uploads is available in AWS CloudTrail logs, S3 Inventory, S3 Storage Lens, the Amazon S3 console, and as an additional Amazon S3 API response header in the AWS Command Line Interface and AWS SDKs. For more information, see [Default encryption FAQ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-encryption-faq.html). ## AWS PrivateLink for Amazon S3 * **Important**\ To take advantage of the lowest cost network path when using **Enable private DNS only for inbound endpoints**, a gateway endpoint must be present in your VPC. The presence of a gateway endpoint helps ensure that in-VPC traffic always routes over the AWS private network when the **Enable private DNS only for inbound endpoints** option is selected. You must maintain this gateway endpoint while you have the **Enable private DNS only for inbound endpoints** option selected. If you want to delete your gateway endpoint you must first clear **Enable private DNS only for inbound endpoints**.\ If you want to update an existing interface endpoint to **Enable private DNS only for inbound endpoints**, first confirm that your VPC has an S3 gateway endpoint. For more information about gateway endpoints and managing private DNS names, see [Gateway VPC endpoints](https://docs.aws.amazon.com/vpc/latest/privatelink/vpce-gateway.html) and [Manage DNS names](https://docs.aws.amazon.com/vpc/latest/privatelink/manage-dns-names.html) respectively in the *AWS PrivateLink Guide*. * **Note**\ To specify the AWS account ID of the resource being accessed, you can use either the `aws:ResourceAccount` or the `s3:ResourceAccount` key in your IAM policy. However, be aware that some AWS services rely on access to AWS managed buckets. Therefore, using the `aws:ResourceAccount` or `s3:ResourceAccount` key in your IAM policy might also affect access to these resources. * **Important**\ When applying the following Amazon S3 bucket policy to restrict access to only certain VPC endpoints, you might block your access to the bucket without intending to do so. Bucket policies that are intended to specifically limit bucket access to connections originating from your VPC endpoint can block all connections to the bucket. For information about how to fix this issue, see [My bucket policy has the wrong VPC or VPC endpoint ID. How can I fix the policy so that I can access the bucket?](https://aws.amazon.com/premiumsupport/knowledge-center/s3-regain-access/) in the *AWS Support Knowledge Center*. Before using the following example policy, replace the VPC endpoint ID with an appropriate value for your use case. Otherwise, you won't be able to access your bucket. This policy disables *console* access to the specified bucket, because console requests don't originate from the specified VPC endpoint. ## Controlling object ownership * **Note**\ If other AWS accounts need access to objects after uploading, you must grant additional permissions to those accounts through bucket policies. For more information, see [Example walkthroughs: Managing access to your Amazon S3 resources](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/example-walkthroughs-managing-access.md). # Monitoring Amazon S3 {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Logging with CloudTrail * **Note**\ S3 does not support delivery of CloudTrail logs to the requester or the bucket owner for VPC endpoint requests when the VPC endpoint policy denies them. * **Important**\ Newer Amazon S3 features are not supported for SOAP. We recommend that you use either the REST API or the AWS SDKs. ## Logging server access * **Note**\ Server access logs don't record information about wrong-region redirect errors for Regions that launched after March 20, 2019. Wrong-region redirect errors occur when a request for an object or bucket is made outside the Region in which the bucket exists. * **Note**\ Amazon S3 does not support delivery of server access logs to the requester or the bucket owner for VPC endpoint requests when the VPC endpoint policy denies them. # Developing with Amazon S3 {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Using the AWS CLI * **Note**\ Services in AWS, such as Amazon S3, require that you provide credentials when you access them. The service can then determine whether you have permissions to access the resources that it owns. The console requires your password. You can create access keys for your AWS account to access the AWS CLI or API. However, we don't recommend that you access AWS using the credentials for your AWS account. Instead, we recommend that you use AWS Identity and Access Management (IAM). Create an IAM user, add the user to an IAM group with administrative permissions, and then grant administrative permissions to the IAM user that you created. You can then access AWS using a special URL and the credentials of that IAM user. For instructions, go to [Creating Your First IAM user and Administrators Group](https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html) in the *IAM User Guide*. ## Error handling * **Note**\ SOAP support over HTTP is deprecated, but SOAP is still available over HTTPS. New Amazon S3 features are not supported for SOAP. Instead of using SOAP, we recommend that you use either the REST API or the AWS SDKs. # Troubleshooting {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Troubleshoot Access Denied (403 Forbidden) errors * **Note**\ `Allow` statements in a bucket policy apply only to objects that are [owned by the same bucket-owning account](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html). However, `Deny` statements in a bucket policy apply to all objects regardless of object ownership. * **Note**\ If you get locked out of a bucket because of an incorrect bucket policy, [sign in to the AWS Management Console by using your root user credentials.](https://docs.aws.amazon.com/signin/latest/userguide/introduction-to-root-user-sign-in-tutorial.html) To regain access to your bucket, make sure to delete the bucket policy by using your root user credentials. ## Troubleshoot lifecycle issues * **Note**\ Amazon S3 doesn’t transition objects that are smaller than 128 KB from the S3 Standard or S3 Standard-IA storage class to the S3 Intelligent-Tiering, S3 Standard-IA, or S3 One Zone-IA storage class. * **Note**\ Amazon S3 rounds the transition or expiration date of an object to midnight UTC the next day. For more information, see [Lifecycle rules: Based on an object's age](https://docs.aws.amazon.com/AmazonS3/latest/dev/intro-lifecycle-rules.html#intro-lifecycle-rules-number-of-days). For S3 objects that are protected by Object Lock, current versions are not permanently deleted. Instead, a delete marker is added to the objects, making them noncurrent. Noncurrent versions are then preserved and are not permanently expired. * **Note**\ For S3 objects that are protected by Object Lock, current versions are not permanently deleted. Instead, a delete marker is added to the objects, making them noncurrent. Noncurrent versions are then preserved and are not permanently expired. * **Note**\ If the S3 bucket is protected by [AWS Backup](https://docs.aws.amazon.com/aws-backup/latest/devguide/s3-backups.html) or [S3 Replication](https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication.html), you might also be able to use these features to recover your expired objects. ## Troubleshoot replication * **Note**\ If the destination bucket's object ownership settings include **Bucket owner enforced**, then you don't need to update the setting to **Change object ownership to the destination bucket owner** in the replication rule. The object ownership change will occur by default. For more information about changing replica ownership, see [Changing the replica owner](https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-change-owner.html). ## Troubleshoot server access logging * **Note**\ We recommend that you choose a target bucket that's different from the source bucket. When the source bucket and target bucket are the same, additional logs are created for the logs that are written to the bucket, which can increase your storage bill. These extra logs about logs can also make it difficult to find the particular logs that you're looking for. For simpler log management, we recommend saving access logs in a different bucket. For more information, see [How do I enable log delivery?](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/ServerLogs.md#server-access-logging-overview). ## Troubleshoot versioning * **Note**\ For S3 Object Lock-enabled buckets, a `DELETE` object request with a protected object version ID causes a 403 Access Denied error. A `DELETE` object request without a version ID adds a delete marker as the newest version of the object with a 200 OK response. Objects protected by Object Lock cannot be permanently deleted until their retention periods and legal holds are removed. For more information, see [How S3 Object Lock works](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/object-lock-overview.md). ## Get Amazon S3 request IDs for AWS Support * **Note**\ HTTPS requests are encrypted and hidden in most packet captures. * **Note**\ By default, the returned log contains only error information. To get the request IDs, the config file must have `AWSLogMetrics` (and optionally, `AWSResponseLogging`) added. # Topics # Working with buckets {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Buckets overview * **Note**\ Objects that belong to a bucket that you create in a specific AWS Region never leave that Region, unless you explicitly transfer them to another Region. For example, objects that are stored in the Europe (Ireland) Region never leave it. * **Note**\ If you see an `Error` when you list your buckets and their public access settings, you might not have the required permissions. Make sure that you have the following permissions added to your user or role policy: * **Note**\ There are also object-level configurations. For example, you can configure object-level permissions by configuring an access control list (ACL) specific to that object. ## Naming rules * **Note**\ Before March 1, 2018, buckets created in the US East (N. Virginia) Region could have names that were up to 255 characters long and included uppercase letters and underscores. Beginning March 1, 2018, new buckets in US East (N. Virginia) must conform to the same rules applied in all other Regions. ## Methods for accessing a bucket * **Note**\ Virtual-hosted-style and path-style requests use the S3 dot Region endpoint structure (`s3.Region`), for example, `https://my-bucket.s3.us-west-2.amazonaws.com`. However, some older Amazon S3 Regions also support S3 dash Region endpoints `s3-Region`, for example, `https://my-bucket.s3-us-west-2.amazonaws.com`. If your bucket is in one of these Regions, you might see `s3-Region` endpoints in your server access logs or AWS CloudTrail logs. We recommend that you do not use this endpoint structure in your requests. * **Important**\ Update (September 23, 2020) – To make sure that customers have the time that they need to transition to virtual-hosted–style URLs, we have decided to delay the deprecation of path-style URLs. For more information, see [Amazon S3 Path Deprecation Plan – The Rest of the Story](https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/) in the *AWS News Blog*. * **Note**\ If your access point name includes dash (-) characters, include the dashes in the URL and insert another dash before the account ID. For example, to use an access point named `finance-docs` owned by account `123456789012` in Region `us-west-2`, the appropriate URL would be `https://finance-docs-123456789012.s3-accesspoint.us-west-2.amazonaws.com`. S3 access points don't support access by HTTP, only secure access by HTTPS. ## Emptying a bucket * **Note**\ You can't remove objects from a bucket that has versioning enabled. Amazon S3 adds a delete marker when you delete an object, which is what this command does. For more information about S3 Bucket Versioning, see [Using versioning in S3 buckets](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/Versioning.md). ## Deleting a bucket * **Important**\ Bucket names are unique. If you delete a bucket, another AWS user can use the name. If you want to continue to use the same bucket name, don't delete the bucket. We recommend that you empty the bucket and keep it. * **Note**\ For buckets without versioning enabled, you can delete all objects directly and then delete the bucket. For buckets with versioning enabled, you must delete all object versions before deleting the bucket. ## Setting default bucket encryption * **Important**\ Amazon S3 now applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for every bucket in Amazon S3. Starting January 5, 2023, all new object uploads to Amazon S3 are automatically encrypted at no additional cost and with no impact on performance. The automatic encryption status for S3 bucket default encryption configuration and for new object uploads is available in AWS CloudTrail logs, S3 Inventory, S3 Storage Lens, the Amazon S3 console, and as an additional Amazon S3 API response header in the AWS Command Line Interface and AWS SDKs. For more information, see [Default encryption FAQ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-encryption-faq.html). * **Note**\ We've changed buckets to encrypt new object uploads automatically. If you previously created a bucket without default encryption, Amazon S3 will enable encryption by default for the bucket using SSE-S3. There will be no changes to the default encryption configuration for an existing bucket that already has SSE-S3 or SSE-KMS configured. If you want to encrypt your objects with SSE-KMS, you must change the encryption type in your bucket settings. For more information, see [Protecting data using server-side encryption with AWS Key Management Service keys (SSE-KMS)](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/UsingKMSEncryption.md). * **Note**\ Amazon S3 buckets with default bucket encryption using SSE-KMS cannot be used as destination buckets for [Logging requests using server access logging](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/ServerLogs.md). Only SSE-S3 default encryption is supported for server access log destination buckets. ## Using Requester Pays * **Important**\ If you enable Requester Pays on a bucket, anonymous access to that bucket is not allowed. # Working with objects {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Creating object keys * **Note**\ Object key names with the value "soap" aren't supported for [virtual-hosted-style requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#virtual-hosted-style-access). For object key name values where "soap" is used, a [path-style URL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access) must be used instead. * **Note**\ Objects with key names ending with period(s) "." downloaded using the Amazon S3 console will have the period(s) "." removed from the key name of the downloaded object. To download an object with the key name ending in period(s) "." retained in the downloaded object, you must use the AWS Command Line Interface (AWS CLI), AWS SDKs, or REST API.\ In addition, be aware of the following prefix limitations:\ Objects with a prefix of "./" must be uploaded or downloaded with the AWS Command Line Interface (AWS CLI), AWS SDKs, or REST API. You cannot use the Amazon S3 console. Objects with a prefix of "../" cannot be uploaded using the AWS Command Line Interface (AWS CLI) or Amazon S3 console. ## Working with metadata * **Note**\ The `PUT` request header is limited to 8 KB in size. Within the `PUT` request header, the system-defined metadata is limited to 2 KB in size. The size of system-defined metadata is measured by taking the sum of the number of bytes in the US-ASCII encoding of each key and value. * **Note**\ SOAP support over HTTP is deprecated, but SOAP is still available over HTTPS. New Amazon S3 features are not supported for SOAP. Instead of using SOAP, we recommend that you use either the REST API or the AWS SDKs. * **Note**\ The `PUT` request header is limited to 8 KB in size. Within the `PUT` request header, the user-defined metadata is limited to 2 KB in size. The size of user-defined metadata is measured by taking the sum of the number of bytes in the UTF-8 encoding of each key and value. ## Uploading objects * **Note**\ If you rename an object or change any of the properties in the Amazon S3 console, for example **Storage Class**, \*\* Encryption\*\*, or **Metadata**, a new object is created to replace the old one. If S3 Versioning is enabled, a new version of the object is created, and the existing object becomes an older version. The role that changes the property also becomes the owner of the new object (or object version). ## Using multipart upload * **Note**\ After you initiate a multipart upload and upload one or more parts, you must either complete or stop the multipart upload to stop getting charged for storage of the uploaded parts. Only *after* you either complete or stop a multipart upload will Amazon S3 free up the parts storage and stop charging you for the parts storage.\ After stopping a multipart upload, you cannot upload any part using that upload ID again. If any part uploads were in-progress, they can still succeed or fail even after you stop the upload. To make sure you free all storage consumed by all parts, you must stop a multipart upload only after all part uploads have been completed. * **Important**\ If you are using a multipart upload with additional checksums, the multipart part numbers must use consecutive part numbers. When using additional checksums, if you try to complete a multipart upload request with nonconsecutive part numbers, Amazon S3 generates HTTP `500 Internal Server Error` error. * **Note**\ It is possible for some other request received between the time you initiated a multipart upload and completed it to take precedence. For example, if another operation deletes a key after you initiate a multipart upload with that key, but before you complete it, the complete multipart upload response might indicate a successful object creation without you ever seeing the object. * **Note**\ To minimize your storage costs, we recommend that you configure a lifecycle rule to delete incomplete multipart uploads after a specified number of days by using the `AbortIncompleteMultipartUpload` action. For more information about creating a lifecycle rule to delete incomplete multipart uploads, see [Configuring a bucket lifecycle configuration to delete incomplete multipart uploads](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/mpu-abort-incomplete-mpu-lifecycle-config.html). ## Copying objects * **Note**\ Copying objects across locations incurs bandwidth charges. If the source object is archived in `S3 Glacier Flexible Retrieval` or `S3 Glacier Deep Archive`, you must first restore a temporary copy before you can copy the object to another bucket. For information about archiving objects, see [Transitioning to the S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive storage classes (object archival)](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/lifecycle-transition-general-considerations.md#before-deciding-to-archive-objects). * **Note**\ Objects encrypted with customer-provided encryption keys (SSE-C) cannot be copied or moved using the S3 console. To copy or move objects encrypted with SSE-C, use the AWS CLI, AWS SDK, or the Amazon S3 REST API. When copying an object by using the Amazon S3 console, you must grant permission `s3:ListAllMyBuckets`. The console needs this permission to validate the copy operation. * **Note**\ This action creates a copy of all specified objects with updated settings, updates the last-modified date in the specified location, and adds a delete marker to the original object. When moving folders, wait for the move action to finish before making additional changes in the folders. This action updates metadata for bucket versioning, encryption, Object Lock features, and archived objects. ## Downloading an object * **Note**\ You can only download one object at a time. Objects with key names ending with period(s) "." downloaded using the Amazon S3 console will have the period(s) "." removed from the key name of the downloaded object. To download an object with the key name ending in period(s) "." retained in the downloaded object, you must use the AWS Command Line Interface (AWS CLI), AWS SDKs, or REST API. For AWS CLI, REST API, and AWS SDK information and examples, see [Downloading an object](https://docs.aws.amazon.com/AmazonS3/latest/userguide/download-objects.html). * **Note**\ Your network connection remains open until you read all of the data or close the input stream. We recommend that you read the content of the stream as quickly as possible. ## Checking object integrity * **Important**\ If you're using a multipart upload with additional checksums, the multipart part numbers must use consecutive part numbers. When using additional checksums, if you try to complete a multipart upload request with nonconsecutive part numbers, Amazon S3 generates an HTTP `500 Internal Server Error` error. * **Important**\ If you're using S3 Object Lambda, all requests to S3 Object Lambda are signed using `s3-object-lambda` instead of `s3`. This behavior affects the signature of trailing checksum values. For more information about S3 Object Lambda, see [Transforming objects with S3 Object Lambda](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/transforming-objects.md). * **Note**\ Objects that are larger than the size limitations of the `CopyObject` API operation must use multipart copy commands. * **Important**\ When you perform some operations using the AWS Management Console, Amazon S3 uses a multipart upload if the object is greater than 16 MB in size. In this case, the checksum is not a direct checksum of the full object, but rather a calculation based on the checksum values of each individual part.\ For example, consider an object 100 MB in size that you uploaded as a single-part direct upload using the REST API. The checksum in this case is a checksum of the entire object. If you later use the console to rename that object, copy it, change the storage class, or edit the metadata, Amazon S3 uses the multipart upload functionality to update the object. As a result, Amazon S3 creates a new checksum value for the object that is calculated based on the checksum values of the individual parts.\ **The preceding list of console operations is not a complete list of all the possible actions that you can take in the AWS Management Console that result in Amazon S3 updating the object using the multipart upload functionality.** Keep in mind that whenever you use the console to act on objects over 16 MB in size, the checksum value might not be the checksum of the entire object. ## Using presigned URLs * **Important**\ For all Regions that launched after March 20, 2019, if a request arrives at the wrong Amazon S3 location, Amazon S3 returns an HTTP 400 Bad Request error. * **Note**\ If you created a presigned URL using a temporary token, the URL expires when the token expires, even if the URL was created with a later expiration time. Because presigned URLs grant access to your Amazon S3 buckets to whoever has the URL, we recommend that you protect them appropriately. # Working with access points {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Configuring IAM policies * **Important**\ Adding an S3 access point to a bucket doesn't change the bucket's behavior when the bucket is accessed directly through the bucket's name or Amazon Resource Name (ARN). All existing operations against the bucket will continue to work as before. Restrictions that you include in an access point policy apply only to requests made through that access point. * **Note**\ Permissions granted in an access point policy are effective only if the underlying bucket also allows the same access. You can accomplish this in two ways:\ **(Recommended)** Delegate access control from the bucket to the access point, as described in [Delegating access control to access points](#access-points-delegating-control). Add the same permissions contained in the access point policy to the underlying bucket's policy. The Example 1 access point policy example demonstrates how to modify the underlying bucket policy to allow the necessary access. * **Note**\ For the access point policy to effectively grant access to *`Jane`*, the underlying bucket must also allow the same access to *`Jane`*. You can delegate access control from the bucket to the access point as described in [Delegating access control to access points](#access-points-delegating-control). Or, you can add the following policy to the underlying bucket to grant the necessary permissions to Jane. Note that the `Resource` entry differs between the access point and bucket policies. ## Creating access points * **Note**\ Because you might want to publicize your access point name so that other users can use the access point, avoid including sensitive information in the access point name. Access point names are published in a publicly accessible database known as the Domain Name System (DNS). # Working with Multi-Region Access Points {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Creating Multi-Region Access Points * **Note**\ For an application or user to be able to access an object through a Multi-Region Access Point, both of the following policies must permit the request:\ The access policy for the Multi-Region Access Point The access policy for the underlying bucket that contains the object When the two policies are different, the more restrictive policy takes precedence.\ To simplify permissions management for Multi-Region Access Points, you can delegate access control from the bucket to the Multi-Region Access Point. For more information, see [Multi-Region Access Point policy examples](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/MultiRegionAccessPointPermissions.md#MultiRegionAccessPointPolicyExamples). ## Using Multi-Region Access Points * **Note**\ The Multi-Region Access Point alias and ARN cannot be used interchangeably. # Managing storage {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Using S3 Versioning * **Note**\ The SOAP API does not support S3 Versioning. SOAP support over HTTP is deprecated, but it is still available over HTTPS. New Amazon S3 features are not supported for SOAP. Normal Amazon S3 rates apply for every version of an object stored and transferred. Each version of an object is the entire object; it is not just a diff from the previous version. Thus, if you have three versions of an object stored, you are charged for three objects. * **Important**\ If you have an object expiration lifecycle configuration in your unversioned bucket and you want to maintain the same permanent delete behavior when you enable versioning, you must add a noncurrent expiration configuration. The noncurrent expiration lifecycle configuration manages the deletes of the noncurrent object versions in the version-enabled bucket. (A version-enabled bucket maintains one current, and zero or more noncurrent, object versions.) For more information, see [Setting lifecycle configuration on a bucket](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/how-to-set-lifecycle-configuration-intro.md). ## Using AWS Backup for Amazon S3 * **Note**\ We recommend that you [set a lifecycle expiration rule for versioning-enabled buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html#lifecycle-config-conceptual-ex6) that are being backed up. If you do not set a lifecycle expiration period, your Amazon S3 storage costs might increase because AWS Backup retains all versions of your Amazon S3 data. ## Working with archived objects * **Note**\ Unlike in the S3 Glacier Flexible Retrieval and S3 Glacier Deep Archive storage classes, restore requests for S3 Intelligent-Tiering objects don't accept the `days` value. * **Note**\ When you restore an archive from S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, you pay for both the archived object and a copy that you restored temporarily. For information about pricing, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). ## Managing storage classes * **Note**\ Only activate the Archive Access tier for 90 days if you want to bypass the Archive Instant Access tier. The Archive Access tier delivers slightly lower-cost storage with minute-to-hour retrieval times. The Archive Instant Access tier delivers millisecond access and high-throughput performance. Activate the Archive Access and Deep Archive Access tiers only if your objects can be accessed asynchronously by your application. If the object that you are retrieving is stored in the Archive Access or Deep Archive Access tiers, first restore the object by using `RestoreObject`. * **Note**\ If the size of an object is less than 128 KB, it is not monitored and not eligible for auto-tiering. Smaller objects are always stored in the Frequent Access tier. For more information about S3 Intelligent-Tiering, see [S3 Intelligent-Tiering access tiers](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/intelligent-tiering-overview.md#intel-tiering-tier-definition). * **Note**\ The S3 Standard-IA and S3 One Zone-IA storage classes are suitable for objects larger than 128 KB that you plan to store for at least 30 days. If an object is less than 128 KB, Amazon S3 charges you for 128 KB. If you delete an object before the end of the 30-day minimum storage duration period, you are charged for 30 days. For pricing information, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). * **Important**\ When you choose the S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive storage class, your objects remain in Amazon S3. You can't access them directly through the separate Amazon S3 Glacier service. * **Note**\ S3 on Outposts doesn't support server-side encryption with AWS Key Management Service (AWS KMS) keys (SSE-KMS. ## Managing inventory * **Important**\ Amazon S3 now applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for every bucket in Amazon S3. Starting January 5, 2023, all new object uploads to Amazon S3 are automatically encrypted at no additional cost and with no impact on performance. The automatic encryption status for S3 bucket default encryption configuration and for new object uploads is available in AWS CloudTrail logs, S3 Inventory, S3 Storage Lens, the Amazon S3 console, and as an additional Amazon S3 API response header in the AWS Command Line Interface and AWS SDKs. For more information, see [Default encryption FAQ](https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-encryption-faq.html). * **Note**\ When an object reaches the end of its lifetime based on its lifecycle configuration, Amazon S3 queues the object for removal and removes it asynchronously. Therefore, there might be a delay between the expiration date and the date when Amazon S3 removes an object. The inventory report includes the objects that have expired but haven't been removed yet. For more information about expiration actions in S3 Lifecycle, see [Expiring objects](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/lifecycle-expire-general-considerations.md). ## Replicating objects * **Note**\ S3 RTC does not apply to Batch Replication. Batch Replication is an on-demand replication job, and can be tracked with S3 Batch Operations. For more information, see [Tracking job status and completion reports](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/batch-ops-job-status.md). ## Using object tags * **Important**\ It is acceptable to use tags to label objects containing confidential data, such as personally identifiable information (PII) or protected health information (PHI). However, the tags themselves shouldn't contain any confidential information. # Using analytics and insights {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Storage Class Analysis * **Important**\ Storage class analysis only provides recommendations for Standard to Standard IA classes. # Hosting a static website {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-s3-userguide.git) {% endhint %} ## Website endpoints * **Important**\ Amazon S3 website endpoints do not support HTTPS or access points. If you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3. For more information, see [How do I use CloudFront to serve HTTPS requests for my Amazon S3 bucket?](https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-https-requests-s3) To use HTTPS with a custom domain, see [Configuring a static website using a custom domain registered with Route 53](https://docs.aws.amazon.com/AmazonS3/latest/userguide/website-hosting-custom-domain-walkthrough.html).\ Requester Pays buckets do not allow access through a website endpoint. Any request to such a bucket receives a 403 Access Denied response. For more information, see [Using Requester Pays buckets for storage transfers and usage](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_simple_storage_service/RequesterPaysBuckets.md). ## Configuring a custom error document * **Note**\ Some browsers display their own error message when an error occurs, ignoring the error document that Amazon S3 returns. For example, when an HTTP 404 Not Found error occurs, Google Chrome might ignore the error document that Amazon S3 returns and display its own error. ## Setting permissions for website access * **Note**\ On the website endpoint, if a user requests an object that doesn't exist, Amazon S3 returns HTTP response code `404 (Not Found)`. If the object exists but you haven't granted read permission on it, the website endpoint returns HTTP response code `403 (Access Denied)`. The user can use the response code to infer whether a specific object exists. If you don't want this behavior, you should not enable website support for your bucket. * **Important**\ The following policy is an example only and allows full access to the contents of your bucket. Before you proceed with this step, review [How can I secure the files in my Amazon S3 bucket?](https://aws.amazon.com/premiumsupport/knowledge-center/secure-s3-resources/) to ensure that you understand the best practices for securing the files in your S3 bucket and risks involved in granting public access. * **Important**\ If your bucket uses the bucket owner enforced setting for S3 Object Ownership, you must use policies to grant access to your bucket and the objects in it. With the bucket owner enforced setting enabled, requests to set access control lists (ACLs) or update ACLs fail and return the `AccessControlListNotSupported` error code. Requests to read ACLs are still supported. ## Configuring a redirect * **Important**\ To create redirection rules in the new Amazon S3 console, you must use JSON. For JSON examples, see [Redirection rules examples](#redirect-rule-examples). * **Important**\ To create redirection rules in the new Amazon S3 console, you must use JSON. # AWS Lambda # Common # Configuring functions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Configuring function options * **Note**\ Configuring ephemeral storage past the default 512 MB allocated incurs a cost. For more information, see [Lambda pricing](https://aws.amazon.com/lambda/pricing). * **Note**\ Compute Optimizer supports only functions that use x86\_64 architecture. ## Environment variables * **Note**\ To increase database security, we recommend that you use AWS Secrets Manager instead of environment variables to store database credentials. For more information, see [Configuring database access for a Lambda function](https://docs.aws.amazon.com/lambda/latest/dg/configuration-database.html). * **Note**\ In some cases, you may need to use the following format: * **Note**\ To increase database security, we recommend that you use AWS Secrets Manager instead of environment variables to store database credentials. For more information, see [Configuring database access for a Lambda function](https://docs.aws.amazon.com/lambda/latest/dg/configuration-database.html). ## Versions * **Note**\ Lambda doesn't create a new version if the code in the unpublished version is the same as the previous published version. You need to deploy code changes in $LATEST before you can create a new version. ## Creating layers * **Note**\ For Node.js runtimes, Lambda doesn't currently support ES module dependencies in layers. * **Note**\ Each time that you call `publish-layer-version`, you create a new version of the layer. # Access permissions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Execution role * **Note**\ Lambda automatically assumes your execution role when you invoke your function. You should avoid calling `sts:AssumeRole` manually in your function code. If your use case requires that the role assumes itself, you must include the role itself as a trusted principal in your role's trust policy. For more information on how to modify a role trust policy, see [Modifying a role trust policy (console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/roles-managingrole-editing-console.html#roles-managingrole_edit-trust-policy) in the IAM User Guide. * **Note**\ You cannot use the `lambda:SourceFunctionArn` condition key in resource-based policies. * **Note**\ The `lambda:SourceFunctionArn` condition key is different from the `lambda:FunctionArn` and `aws:SourceArn` condition keys. The `lambda:FunctionArn` condition key applies only to [event source mappings](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/invocation-eventsourcemapping.md) and helps define which functions your event source can invoke. The `aws:SourceArn` condition key applies only to policies where your Lambda function is the target resource, and helps define which other AWS services and resources can invoke that function. The `lambda:SourceFunctionArn` condition key can apply to any identity-based policy or SCP to define the specific Lambda functions that have permissions to make specific AWS API calls to other resources. ## User policies * **Note**\ The AWS managed policies **AWSLambdaFullAccess** and **AWSLambdaReadOnlyAccess** will be [deprecated](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-deprecated.html) on March 1, 2021. After this date, you cannot attach these policies to new IAM users. For more information, see the related [troubleshooting topic](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/security_iam_troubleshoot.md#security_iam_troubleshoot-admin-deprecation). * **Note**\ For a function defined as a container image, the user permission to access the image MUST be configured in the Amazon Elastic Container Registry For an example, see [Amazon ECR permissions.](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/gettingstarted-images.md#configuration-images-permissions) ## Control access using tags * **Note**\ Do not use the `ResourceTag` condition key in a policy with the `iam:PassRole` action. You cannot use the tag on an IAM role to control access to who can pass that role. For more information about permissions required to pass a role to a service, see [Granting a user permissions to pass a role to an AWS service](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_passrole.html). ## Resource-based policies * **Note**\ You can only update resource-based policies for Lambda resources within the scope of the [AddPermission](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/API_AddPermission.md) and [AddLayerVersionPermission](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/API_AddLayerVersionPermission.md) API actions. Currently, you can't author policies for your Lambda resources in JSON, or use conditions that don't map to parameters for those actions. * **Note**\ When you add a trigger to your function with the Lambda console, the console updates the function's resource-based policy to allow the service to invoke it. To grant permissions to other accounts or services that aren't available in the Lambda console, you can use the AWS CLI. * **Note**\ In this command, `Principal` is `*`. This means that all users in the organization `o-a1b2c3d4e5f` get function invocation permissions. If you specify an AWS account or role as the `Principal`, then only that principal gets function invocation permissions, but only if they are also part of the `o-a1b2c3d4e5f` organization. ## Resources and conditions * **Note**\ You can't use a wildcard character (\*) to match the account ID. For more information on accepted syntax, see [IAM JSON policy reference](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies.html) in the *IAM User Guide*. * **Note**\ The [GetLayerVersion](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/API_GetLayerVersion.md) action also covers [GetLayerVersionByArn](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/API_GetLayerVersionByArn.md). Lambda does not support `GetLayerVersionByArn` as an IAM action. # Monitoring functions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Function metrics * **Note**\ The timestamp on a metric reflects when the function was invoked. Depending on the duration of the invocation, this can be several minutes before the metric is emitted. For example, if your function has a 10-minute timeout, look more than 10 minutes in the past for accurate metrics. ## Function logs * **Note**\ It may take 5 to 10 minutes for logs to show up after a function invocation. ## AWS X-Ray * **Note**\ X-Ray tracing is currently not supported for Lambda functions with Amazon Managed Streaming for Apache Kafka (Amazon MSK), self-managed Apache Kafka, or Amazon MQ with ActiveMQ and RabbitMQ event source mappings. * **Note**\ You cannot configure the X-Ray sampling rate for your functions. * **Note**\ If your Lambda function uses [provisioned concurrency](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/provisioned-concurrency.md), your X-Ray trace might display a function initialization with a very long duration.\ Provisioned concurrency initializes function instances in advance, to reduce lag at the time of invocation. Over time, provisioned concurrency refreshes these instances by creating new instances to replace the old ones. For workloads with steady traffic, the new instances are initialized well in advance of their first invocation. The time gap gets recorded in the X-Ray trace as the initialization duration. * **Important**\ In Lambda, you can use the X-Ray SDK to extend the `Invocation` subsegment with additional subsegments for downstream calls, annotations, and metadata. You can't access the function segment directly or record work done outside of the handler invocation scope. ## Code profiler * **Note**\ When you deactivate CodeGuru Profiler from the console, Lambda automatically removes the CodeGuru Profiler layer and environment variables from your function. However, Lambda does not remove the `AmazonCodeGuruProfilerAgentAccess` policy from your execution role. # Troubleshooting {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Deployment * **Note**\ When you upload a file directly with the AWS CLI, AWS SDK, or otherwise, the binary ZIP file is converted to base64, which increases its size by about 30%. To allow for this, and the size of other parameters in the request, the actual request size limit that Lambda applies is larger. Due to this, the 50 MB limit is approximate. ## Invocation * **Note**\ Unlike other Lambda API operations, the name of the IAM action (`lambda:InvokeFunction`) doesn't match the name of the API operation (`Invoke`) for invoking a function. * **Note**\ There is a known issue in which the first invocation on an initialized execution environment reports a non-zero **Init Duration** metric in CloudWatch Logs, even though no cold start has occurred. We're developing a fix to correct the reporting to CloudWatch Logs. ## Execution * **Note**\ It may take 5 to 10 minutes for logs to show up after a function invocation. # Topics # Lambda foundations {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Programming model * **Note**\ Logging is subject to [CloudWatch Logs quotas](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch_limits_cwl.html). Log data can be lost due to throttling or, in some cases, when an instance of your function is stopped. ## Execution environment * **Note**\ The Lambda reset does not clear the `/tmp` directory content prior to the next init phase. This behavior is consistent with the regular shutdown phase. ## Deployment packages * **Note**\ Container images aren't supported for Lambda functions in the Middle East (UAE) Region. ## Instruction sets (ARM/x86) * **Note**\ Runtimes that use the Amazon Linux operating system, such as Go 1.x, do not support the arm64 architecture. To use arm64 architecture, you can run Go with the provided.al2 runtime. For example, see [Build a Go function for the provided.al2 runtime](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/golang-package.md#golang-package-al2) or [Create a Go image from the provided.al2 base image](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/go-image.md#go-image-al2). # Lambda runtimes {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Runtime modifications * **Note**\ Invocations may fail if the wrapper script does not successfully start the runtime process. ## Tutorial – Custom runtime * **Note**\ On Windows, some Bash CLI commands that you commonly use with Lambda (such as `zip`) are not supported by the operating system's built-in terminals. To get a Windows-integrated version of Ubuntu and Bash, [install the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10). * **Note**\ Runtimes have additional responsibilities, including error handling, and providing context information to the handler. For details, see [Building a custom runtime](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/runtimes-custom.md#runtimes-custom-build). ## AVX2 vectorization * **Note**\ Lambda arm64 uses NEON SIMD architecture and does not support the x86 AVX2 extensions. # Deploying functions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## .zip file archives * **Note**\ You cannot convert an existing container image function to use a .zip file archive. You must create a new function. * **Note**\ The Lambda console uses AWS Cloud9 to provide an integrated development environment in the browser. You can also use AWS Cloud9 to develop Lambda functions in your own environment. For more information, see [Working with Lambda Functions](https://docs.aws.amazon.com/cloud9/latest/user-guide/lambda-functions.html) in the AWS Cloud9 user guide. ## Container images * **Note**\ You cannot convert an existing container image function to use a .zip file archive. You must create a new function. * **Note**\ On Windows, some Bash CLI commands that you commonly use with Lambda (such as `zip`) are not supported by the operating system's built-in terminals. To get a Windows-integrated version of Ubuntu and Bash, [install the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10). * **Note**\ For the override settings, make sure that you enclose each string in quotation marks (" "). * **Note**\ You cannot change the `package-type` of a function. * **Note**\ If you declare an `ImageConfig` property in your AWS CloudFormation template, you must provide values for all three of the `ImageConfig` properties. # Invoking functions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Synchronous invocation * **Note**\ For functions with a long timeout, your client might be disconnected during synchronous invocation while it waits for a response. Configure your HTTP client, SDK, firewall, proxy, or operating system to allow for long connections with timeout or keep-alive settings. ## Asynchronous invocation * **Note**\ To prevent a function from triggering, you can set the function's reserved concurrency to zero. When you set reserved concurrency to zero for an asynchronously-invoked function, Lambda begins sending new events to the configured [dead-letter queue](#invocation-dlq) or the on-failure [event destination](#invocation-async-destinations), without any retries. To process events that were sent while reserved concurrency was set to zero, you need to consume the events from the dead-letter queue or the on-failure event destination. ## Event filtering * **Note**\ Like EventBridge, for strings, Lambda uses exact character-by-character matching without case-folding or any other string normalization. For numbers, Lambda also uses string representation. For example, 300, 300.0, and 3.0e2 are not considered equal. * **Note**\ By default, you can have five different filters per event source. You can [request a quota increase](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html) for up to 10 filters per event source. The Lambda console lets you add up to 10 filters depending on the current quota for your account. If you attempt to add more filters than your current quota allows, Lambda throws an error when you try to create the event source. * **Note**\ After you attach filter criteria to a Kafka or Amazon MQ event source mapping, it can take up to 15 minutes to apply your filtering rules to events. ## Function URLs * **Note**\ You can access your function URL through the public Internet only. While Lambda functions do support AWS PrivateLink, function URLs do not. # Managing functions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Tutorial - Lambda with CLI * **Note**\ On Windows, some Bash CLI commands that you commonly use with Lambda (such as `zip`) are not supported by the operating system's built-in terminals. To get a Windows-integrated version of Ubuntu and Bash, [install the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10). ## Networking * **Tip**\ If your function already includes a VPC configuration from a previous API request, you can send an `UpdateFunctionConfiguration` request without the VPC configuration. * **Note**\ Several AWS services offer [VPC endpoints](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html). You can use VPC endpoints to connect to AWS services from within a VPC without internet access. ## Interface VPC endpoints * **Note**\ You need to include both the qualified and the unqualified function ARN in the resource. ## File system * **Note**\ When you create a file system, you choose a performance mode that can't be changed later. **General purpose** mode has lower latency, and **Max I/O** mode supports a higher maximum throughput and IOPS. For help choosing, see [Amazon EFS performance](https://docs.aws.amazon.com/efs/latest/ug/performance.html) in the *Amazon Elastic File System User Guide*. * **Note**\ If your function is not already connected to a VPC, see [Configuring a Lambda function to access resources in a VPC](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/configuration-vpc.md). ## Code signing * **Note**\ Functions defined as container images do not support code signing. ## Using layers * **Note**\ Make sure that the layers that you add to a function are compatible with the runtime and instruction set architecture of the function. # Building with Node.js # Building with TypeScript {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Deploy .zip file archives * **Note**\ The AWS SAM uses esbuild to create Node.js Lambda functions from TypeScript code. esbuild support is currently in public preview. During public preview, esbuild support may be subject to backwards incompatible changes. # Building with Python {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Handler * **Note**\ In Python 3.9 and later releases, Lambda includes the requestId of the invocation in the error response. ## Deploy .zip file archives * **Note**\ A python package may contain initialization code in the \_\_init\_\_.py file. Prior to Python 3.9, Lambda did not run the \_\_init\_\_.py code for packages in the function handler’s directory or parent directories. In Python 3.9 and later releases, Lambda runs the init code for packages in these directories during initialization.\ Note that Lambda runs the init code only when the execution environment is first initialized, not for each function invocation in that initialized environment. ## Tracing * **Important**\ **ADOT is the preferred method for instrumenting your Lambda functions**. We recommend using ADOT for all new applications. However, due to the flexibility OpenTelemetry offers, your Lambda function invocations may experience cold start latency increases. If you're optimizing for low-latency and also do not require OpenTelemetry's advanced capabilities such as telemetry correlation and dynamically configurable backend destinations, you may want to use the AWS X-Ray SDK over ADOT. * **Note**\ You cannot configure the X-Ray sampling rate for your functions. # Building with Ruby {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Tracing * **Note**\ You cannot configure the X-Ray sampling rate for your functions. # Building with Java {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Deploy .zip file archives * **Note**\ To keep your deployment package size small, package your function's dependencies in layers. Layers enable you to manage your dependencies independently, can be used by multiple functions, and can be shared with other accounts. For more information, see [Creating and sharing Lambda layers](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/configuration-layers.md). ## Logging * **Note**\ AWS Lambda does not include Log4j2 in its managed runtimes or base container images. These are therefore not affected by the issues described in CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105.\ For cases where a customer function includes an impacted Log4j2 version, we have applied a change to the Lambda Java [managed runtimes](/serverless/aws-lambda/topics/lambda-runtimes) and [base container images](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/java-image.md) that helps to mitigate the issues in CVE-2021-44228, CVE-2021-45046, and CVE-2021-45105. As a result of this change, customers using Log4J2 may see an additional log entry, similar to "`Transforming org/apache/logging/log4j/core/lookup/JndiLookup (java.net.URLClassLoader@...)`". Any log strings that reference the jndi mapper in the Log4J2 output will be replaced with "`Patched JndiLookup::lookup()`".\ Independent of this change, we strongly encourage all customers whose functions include Log4j2 to update to the latest version. Specifically, customers using the aws-lambda-java-log4j2 library in their functions should update to version 1.5.0 (or later), and redeploy their functions. This version updates the underlying Log4j2 utility dependencies to version 2.17.0 (or later). The updated aws-lambda-java-log4j2 binary is available at the [Maven repository](https://repo1.maven.org/maven2/com/amazonaws/aws-lambda-java-log4j2/) and its source code is available in [Github](https://github.com/aws/aws-lambda-java-libs/tree/master/aws-lambda-java-log4j2). ## Errors * **Note**\ To test this code, you need to include [InputLengthException.java](https://github.com/awsdocs/aws-lambda-developer-guide/tree/main/sample-apps/java-basic/src/main/java/example/InputLengthException.java) in your project src folder. ## Tracing * **Important**\ **ADOT is the preferred method for instrumenting your Lambda functions**. We recommend using ADOT for all new applications. However, due to the flexibility OpenTelemetry offers, your Lambda function invocations may experience cold start latency increases. If you're optimizing for low-latency and also do not require OpenTelemetry's advanced capabilities such as telemetry correlation and dynamically configurable backend destinations, you may want to use the AWS X-Ray SDK over ADOT. * **Note**\ You cannot configure the X-Ray sampling rate for your functions. ## Tutorial - Eclipse IDE * **Note**\ The AWS SDK Eclipse Toolkit provides an Eclipse plugin for you to both create a deployment package and also upload it to create a Lambda function. If you can use Eclipse IDE as your development environment, this plugin enables you to author Java code, create and upload a deployment package, and create your Lambda function. For more information, see the [AWS Toolkit for Eclipse Getting Started Guide](https://docs.aws.amazon.com/AWSToolkitEclipse/latest/GettingStartedGuide/). For an example of using the toolkit for authoring Lambda functions, see [Using Lambda with the AWS toolkit for Eclipse](https://docs.aws.amazon.com/AWSToolkitEclipse/latest/GettingStartedGuide/lambda.html). * **Note**\ If you are following other tutorial topics in this guide, the specific tutorials might recommend different package name or class name. # Building with Go {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Deploy .zip file archives * **Note**\ If you have not already done so, you must install [git](https://git-scm.com/) and then add the `git` executable to your Windows `%PATH%` environment variable. * **Note**\ When you create a Go Lambda function using the AWS CLI, the value of the handler setting you define is the executable file name. For more information, see [AWS Lambda function handler in Go](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/golang-handler.md). ## Tracing * **Important**\ **ADOT is the preferred method for instrumenting your Lambda functions**. We recommend using ADOT for all new applications. However, due to the flexibility OpenTelemetry offers, your Lambda function invocations may experience cold start latency increases. If you're optimizing for low-latency and also do not require OpenTelemetry's advanced capabilities such as telemetry correlation and dynamically configurable backend destinations, you may want to use the AWS X-Ray SDK over ADOT. * **Note**\ You cannot configure the X-Ray sampling rate for your functions. # Building with C\# # Building with PowerShell {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Handler * **Note**\ You're required to use the `#Requires` statement within your PowerShell scripts to indicate the modules that your scripts depend on. This statement performs two important tasks. 1) It communicates to other developers which modules the script uses, and 2) it identifies the dependent modules that AWS PowerShell tools need to package with the script, as part of the deployment. For more information about the `#Requires` statement in PowerShell, see [About requires](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_requires?view=powershell-6). For more information about PowerShell deployment packages, see [Deploy PowerShell Lambda functions with .zip file archives](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/powershell-package.md).\ When your PowerShell Lambda function uses the AWS PowerShell cmdlets, be sure to set a `#Requires` statement that references the `AWSPowerShell.NetCore` module, which supports PowerShell Core—and not the `AWSPowerShell` module, which only supports Windows PowerShell. Also, be sure to use version 3.3.270.0 or newer of `AWSPowerShell.NetCore` which optimizes the cmdlet import process. If you use an older version, you'll experience longer cold starts. For more information, see [AWS Tools for PowerShell](http://aws.amazon.com/powershell/?track=sdk). # Integrating other services {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## API Gateway * **Note**\ If your function and API are in different regions, the region identifier in the source ARN must match the region of the function, not the region of the API. When API Gateway invokes a function, it uses a resource ARN that is based on the ARN of the API, but modified to match the function's region. * **Note**\ API Gateway does not retry any Lambda invocations. If Lambda returns an error, API Gateway returns an error response to the client. ## EventBridge (CloudWatch Events) * **Note**\ Amazon EventBridge is the preferred way to manage your events. CloudWatch Events and EventBridge are the same underlying service and API, but EventBridge provides more features. Changes you make in either CloudWatch Events or EventBridge will appear in each console. For more information, see the [Amazon EventBridge documentation](https://docs.aws.amazon.com/eventbridge/index.html). ## CloudFront (Lambda\@Edge) * **Note**\ Lambda\@Edge supports a limited set of runtimes and features. For details, see [Requirements and restrictions on Lambda functions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html) in the Amazon CloudFront developer guide. ## DynamoDB * **Note**\ You are not charged for GetRecords API calls invoked by Lambda as part of DynamoDB triggers. * **Note**\ For Java functions, we recommend using a `Map` to represent the state. * **Note**\ If the `batchItemFailures` array contains multiple items, Lambda uses the record with the lowest sequence number as the checkpoint. Lambda then retries all records starting from that checkpoint. ## ElastiCache * **Note**\ On Windows, some Bash CLI commands that you commonly use with Lambda (such as `zip`) are not supported by the operating system's built-in terminals. To get a Windows-integrated version of Ubuntu and Bash, [install the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10). ## EFS * **Note**\ To configure a function to connect to a file system, see [Configuring file system access for Lambda functions](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/configuration-filesystem.md). * **Note**\ If you use [provisioned concurrency](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/configuration-concurrency.md), your function can consume burst credits even when idle. With provisioned concurrency, Lambda initializes instances of your function before it is invoked, and recycles instances every few hours. If you use files on an attached file system during initialization, this activity can use all of your burst credits. ## Apache Kafka * **Note**\ Lambda supports the [PBES1](https://datatracker.ietf.org/doc/html/rfc2898/#section-6.1) (but not PBES2) private key encryption algorithms. * **Note**\ While Lambda functions typically have a maximum timeout limit of 15 minutes, event source mappings for Amazon MSK, self-managed Apache Kafka, and Amazon MQ for ActiveMQ and RabbitMQ only support functions with maximum timeout limits of 14 minutes. This constraint ensures that the event source mapping can properly handle function errors and retries. * **Note**\ If your Lambda event records exceed the allowed size limit of 6 MB, they can go unprocessed. ## Kinesis Streams * **Note**\ Kinesis charges for each shard and, for enhanced fan-out, data read from the stream. For pricing details, see [Amazon Kinesis pricing](https://aws.amazon.com/kinesis/data-streams/pricing). * **Note**\ For Java functions, we recommend using a `Map` to represent the state. * **Note**\ If the `batchItemFailures` array contains multiple items, Lambda uses the record with the lowest sequence number as the checkpoint. Lambda then retries all records starting from that checkpoint. ## MQ * **Note**\ By default, Amazon MQ has a weekly maintenance window for brokers. During that window of time, brokers are unavailable. For brokers without standby, Lambda cannot process any messages during that window. * **Note**\ While Lambda functions typically have a maximum timeout limit of 15 minutes, event source mappings for Amazon MSK, self-managed Apache Kafka, and Amazon MQ for ActiveMQ and RabbitMQ only support functions with maximum timeout limits of 14 minutes. This constraint ensures that the event source mapping can properly handle function errors and retries. * **Note**\ When using an encrypted customer managed key, add the `[kms:Decrypt](https://docs.aws.amazon.com/msk/1.0/apireference/clusters-clusterarn-bootstrap-brokers.html#clusters-clusterarn-bootstrap-brokersget)` permission as well. * **Note**\ Lambda does not support custom redelivery policies. Instead, Lambda uses a policy with the default values from the [Redelivery Policy](https://activemq.apache.org/redelivery-policy) page on the Apache ActiveMQ website, with `maximumRedeliveries` set to 5. ## MSK * **Note**\ While Lambda functions typically have a maximum timeout limit of 15 minutes, event source mappings for Amazon MSK, self-managed Apache Kafka, and Amazon MQ for ActiveMQ and RabbitMQ only support functions with maximum timeout limits of 14 minutes. This constraint ensures that the event source mapping can properly handle function errors and retries. * **Note**\ Lambda supports the [PBES1](https://datatracker.ietf.org/doc/html/rfc2898/#section-6.1) (but not PBES2) private key encryption algorithms. * **Note**\ If Lambda can't connect to the most secure broker type, Lambda doesn't attempt to connect to a different (weaker) broker type. If you want Lambda to choose a weaker broker type, deactivate all stronger auth methods on your cluster. * **Note**\ Lambda eventually plans to remove the `kafka:DescribeCluster` permission from this policy. You should migrate any applications using `kafka:DescribeCluster` to use `kafka:DescribeClusterV2` instead. * **Note**\ Your Amazon VPC configuration is discoverable through the [Amazon MSK API](https://docs.aws.amazon.com/msk/1.0/apireference/resources.html). You don't need to configure it during setup using the create-event-source-mapping command. ## SQS * **Note**\ If you're using a batch window and your SQS queue contains very low traffic, Lambda might wait for up to 20 seconds before invoking your function. This is true even if you set a batch window lower than 20 seconds. * **Note**\ Make sure that you configure the dead-letter queue on the source queue, not on the Lambda function. The dead-letter queue that you configure on a function is used for the function's [asynchronous invocation queue](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/invocation-async.md), not for event source queues. * **Note**\ Amazon SQS has a perpetual free tier for requests. Beyond the free tier, Amazon SQS charges per million requests. While your event source mapping is active, Lambda makes requests to the queue to get items. For pricing details, see [Amazon SQS pricing](http://aws.amazon.com/sqs/pricing). * **Note**\ If you're using this feature with a FIFO queue, your function should stop processing messages after the first failure and return all failed and unprocessed messages in `batchItemFailures`. This helps preserve the ordering of messages in your queue. # Creating container images {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Base images for Lambda * **Note**\ Each base image is compatible with one or more of the instruction set architectures that Lambda supports. You need to build the function image for only one architcture. Lambda does not support multi-architecture images. # Lambda extensions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Extensions API * **Note**\ For example extensions and wrapper scripts, see [AWS Lambda Extensions](https://github.com/aws-samples/aws-lambda-extensions) on the AWS Samples GitHub repository. * **Note**\ Extensions can complete their initialization at any point in the `Init` phase. * **Note**\ Lambda allocates CPU power in proportion to the function's memory setting. You might see increased execution and initialization duration at lower memory settings because the function and extension processes are competing for the same CPU resources. To reduce the execution and initialization duration, try increasing the memory setting. * **Note**\ Because the function code directory is read-only, extensions cannot modify the function code. ## Telemetry API * **Important**\ The Lambda Telemetry API supersedes the Lambda Logs API. **While the Logs API remains fully functional, we recommend using only the Telemetry API going forward.** You can subscribe your extension to a telemetry stream using either the Telemetry API or the Logs API. After subscribing using one of these APIs, any attempt to subscribe using the other API returns an error. * **Note**\ Lambda sends logs and metrics to CloudWatch, and traces to X-Ray (if you've activated tracing), even if an extension subscribes to telemetry streams. * **Note**\ A Lambda function's execution environment can start and stop multiple times as part of its [lifecycle](https://github.com/kevinslin/aws-reference-notes/blob/main/services/aws_lambda/runtimes-extensions-api.md#runtimes-extensions-api-lifecycle). In general, your extension code runs during function invocations, and also up to 2 seconds during the shutdown phase. We recommend batching the telemetry as it arrives to your listener, and using the `Invoke` and `Shutdown` lifecycle events to dispatch each batch to their desired destinations. * **Note**\ We strongly recommend using HTTP rather than TCP. With TCP, the Lambda platform cannot acknowledge when it delivers telemetry to the application layer. Therefore, if your extension crashes, you might lose telemetry. HTTP does not have this limitation. # Lambda applications {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Tutorial – Create an application * **Note**\ On Windows, some Bash CLI commands that you commonly use with Lambda (such as `zip`) are not supported by the operating system's built-in terminals. To get a Windows-integrated version of Ubuntu and Bash, [install the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10). # Orchestrating functions {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/aws-lambda-developer-guide.git) {% endhint %} ## Manage state machines * **Note**\ When a Lambda function is [dynamically referenced with JsonPath](https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-paths.html) in the state machine definition, the function details cannot be shown in the Lambda console. Instead, the function name is listed as a **Dynamic reference**, and the corresponding steps in the graph are grayed out. # Amazon Aurora # Common # What is Aurora? {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Aurora DB clusters * **Note**\ The preceding information applies to provisioned clusters, parallel query clusters, global database clusters, Aurora Serverless clusters, and all MySQL 8.0-compatible, 5.7-compatible, and PostgreSQL-compatible clusters. ## Aurora versions * **Note**\ If you want to remain on an LTS minor version for the duration of its lifecycle, make sure to turn off **Auto minor version upgrade** for your DB instances. To avoid automatically upgrading your DB cluster from the LTS minor version, set **Auto minor version upgrade** to `No` on any DB instance in your Aurora cluster. ## Regions and Availability Zones * **Note**\ For information about finding the Availability Zones for an AWS Region, see [Describe your Availability Zones](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#availability-zones-describe) in the Amazon EC2 documentation. * **Note**\ Aurora doesn't support Local Zones. ## Aurora connection management * **Tip**\ Through RDS Proxy, you can create additional read-only endpoints for an Aurora cluster. These endpoints perform the same kind of load-balancing as the Aurora reader endpoint. Applications can reconnect more quickly to the proxy endpoints than the Aurora reader endpoint if reader instances become unavailable. The proxy endpoints can also take advantage of other proxy features such as multiplexing. For more information, see [Using reader endpoints with Aurora clusters](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/rds-proxy-endpoints.md#rds-proxy-endpoints-reader). * **Note**\ You must also create and manage custom endpoints for Aurora clusters restored from snapshots. Custom endpoints are not included in the snapshot. You create them again after restoring, and choose new endpoint names if the restored cluster is in the same region as the original one. ## DB instance classes * **Note**\ We recommend using the T DB instance classes only for development, test, or other nonproduction servers. For more detailed recommendations for the T instance classes, see [Using T instance classes for development and testing](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/AuroraMySQL.BestPractices.md#AuroraMySQL.BestPractices.T2Medium). * **Note**\ When you perform operations with the AWS CLI, it automatically shows the supported DB instance classes for a specific DB engine, DB engine version, and AWS Region. Examples of the operations that you can perform include creating and modifying a DB instance. ## Aurora storage and reliability * **Note**\ The storage limits and dynamic resizing behavior discussed here apply to persistent tables and other data stored in the cluster volume.\ For Aurora PostgreSQL, temporary table data is stored in the local DB instance.\ For Aurora MySQL version 2, temporary table data is stored by default in the cluster volume for writer instances and in local storage for reader instances. For more information, see [Storage engine for on-disk temporary tables](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/AuroraMySQL.CompareMySQL57.md#AuroraMySQL.StorageEngine57).\ For Aurora MySQL version 3, temporary table data is stored in the local DB instance or in the cluster volume. For more information, see [New temporary table behavior in Aurora MySQL version 3](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/ams3-temptable-behavior.md).\ The maximum size of temporary tables that reside in local storage is limited by the maximum local storage size of the DB instance. The local storage size depends on the instance class that you use. For more information, see [Temporary storage limits for Aurora MySQLTemporary storage limits](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/AuroraMySQL.Managing.Performance.md#AuroraMySQL.Managing.TempStorage) and [Temporary storage limits for Aurora PostgreSQLTemporary storage limits](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/AuroraPostgreSQL.Managing.md#AuroraPostgreSQL.Managing.TempStorage). * **Tip**\ For earlier versions without the dynamic resizing feature, resetting the storage usage for a cluster involved doing a logical dump and restoring to a new cluster. That operation can take a long time for a substantial volume of data. If you encounter this situation, consider upgrading your cluster to a version that supports dynamic volume resizing. ## High availability for Amazon Aurora * **Tip**\ Within each AWS Region, Availability Zones (AZs) represent locations that are distinct from each other to provide isolation in case of outages. We recommend that you distribute the primary instance and reader instances in your DB cluster over multiple Availability Zones to improve the availability of your DB cluster. That way, an issue that affects an entire Availability Zone doesn't cause an outage for your cluster.\ You can set up a Multi-AZ cluster by making a simple choice when you create the cluster. The choice is simple whether you use the AWS Management Console, the AWS CLI, or the Amazon RDS API. You can also make an existing Aurora cluster into a Multi-AZ cluster by adding a new reader instance and specifying a different Availability Zone. * **Tip**\ In Aurora MySQL 2.10 and higher, you can improve availability during a failover by having more than one reader DB instance in a cluster. In Aurora MySQL 2.10 and higher, Aurora restarts only the writer DB instance and the failover target during a failover. Other reader DB instances in the cluster remain available to continue processing queries through connections to the reader endpoint.\ You can also improve availability during a failover by using RDS Proxy with your Aurora DB cluster. For more information, see [High availability with Amazon RDS Proxy](#Concepts.AuroraHighAvailability.Proxy). * **Note**\ Amazon Aurora also supports replication with an external MySQL database, or an RDS MySQL DB instance. For more information, see [Replication between Aurora and MySQL or between Aurora and another Aurora DB cluster (binary log replication)](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/AuroraMySQL.Replication.MySQL.md). ## Replication with Aurora * **Tip**\ You can use Aurora Replicas within an Aurora cluster as your only form of replication to keep your data highly available. You can also combine the built-in Aurora replication with the other types of replication. Doing so can help to provide an extra level of high availability and geographic distribution of your data. # Getting started {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Creating and connecting to an Aurora MySQL DB cluster * **Important**\ There's no charge for creating an AWS account. However, by completing this tutorial, you might incur costs for the AWS resources that you use. You can delete these resources after you complete the tutorial if they are no longer needed. ## Creating and connecting to an Aurora PostgreSQL DB cluster * **Important**\ There's no charge for creating an AWS account. However, by completing this tutorial, you might incur costs for the AWS resources that you use. You can delete these resources after you complete the tutorial if they are no longer needed. ## Tutorial: Create a web server and an Amazon Aurora DB cluster * **Important**\ There's no charge for creating an AWS account. However, by completing this tutorial, you might incur costs for the AWS resources you use. You can delete these resources after you complete the tutorial if they are no longer needed. * **Note**\ This tutorial works with Amazon Linux 2023 and might not work for other versions of Linux. * **Note**\ After you complete the tutorial, there is a public and private subnet in each Availability Zone in your VPC. This tutorial uses the default VPC for your AWS account and automatically sets up connectivity between your EC2 instance and DB cluster. If you would rather configure a new VPC for this scenario instead, complete the tasks in [Tutorial: Create a VPC for use with a DB cluster (IPv4 only)](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/CHAP_Tutorials.WebServerDB.CreateVPC.md). # Configuring your Aurora DB cluster {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Creating a DB cluster * **Important**\ Before you can create an Aurora DB cluster, you must complete the tasks in [Setting up your environment for Amazon Aurora](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/CHAP_SettingUp_Aurora.md). * **Note**\ For this example, **Standard create** is enabled, and **Easy create** isn't enabled. For information about creating an Aurora MySQL DB cluster with **Easy create** enabled, see [Getting started with Amazon Aurora](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/CHAP_GettingStartedAurora.md). * **Note**\ Before you can create an Aurora DB cluster using the AWS CLI, you must fulfill the required prerequisites, such as creating a VPC and an RDS DB subnet group. For more information, see [DB cluster prerequisites](#Aurora.CreateInstance.Prerequisites). * **Note**\ Before you can create an Aurora DB cluster using the AWS CLI, you must fulfill the required prerequisites, such as creating a VPC and an RDS DB subnet group. For more information, see [DB cluster prerequisites](#Aurora.CreateInstance.Prerequisites). * **Note**\ Additional settings are available if you are creating an Aurora Serverless v1 DB cluster. For information about these settings, see [Creating an Aurora Serverless v1 DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless.create.md). Also, some settings aren't available for Aurora Serverless v1 because of Aurora Serverless v1 limitations. For more information, see [Limitations of Aurora Serverless v1](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless.md#aurora-serverless.limitations). * **Note**\ The AWS Management Console doesn't show these settings for Aurora DB clusters. * **Note**\ The AWS Management Console doesn't show these settings for Aurora DB instances. ## Connecting to a DB cluster * **Note**\ For Aurora Serverless DB clusters, you connect to the database endpoint rather than to the DB instance. You can find the database endpoint for an Aurora Serverless DB cluster on the **Connectivity & security** tab of the AWS Management Console. For more information, see [Using Amazon Aurora Serverless v1](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless.md). * **Note**\ For a helpful and detailed guide on connecting to an Amazon Aurora MySQL DB cluster, you can see the [Aurora connection management](https://d1.awsstatic.com/whitepapers/RDS/amazon-aurora-connection-management-handbook.pdf) handbook. * **Note**\ This procedure doesn't require installing the web server in the tutorial, but it does require installing MariaDB 10.5. * **Note**\ To connect to the cluster endpoint using SSL, your client connection utility must support Subject Alternative Names (SAN). If your client connection utility doesn't support SAN, you can connect directly to the instances in your Aurora DB cluster. For more information on Aurora endpoints, see [Amazon Aurora connection management](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/Aurora.Overview.Endpoints.md). # Monitoring metrics in an Aurora DB cluster {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Viewing cluster status and recommendations * **Note**\ Aurora also uses another status called *maintenance status*, which is shown in the **Maintenance** column of the Amazon RDS console. This value indicates the status of any maintenance patches that need to be applied to a DB cluster. Maintenance status is independent of DB cluster status. For more information about maintenance status, see [Applying updates for a DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/USER_UpgradeDBInstance.Maintenance.md#USER_UpgradeDBInstance.OSUpgrades). * **Note**\ Amazon RDS also uses another status called *maintenance status*, which is shown in the **Maintenance** column of the Amazon RDS console. This value indicates the status of any maintenance patches that need to be applied to a DB instance. Maintenance status is independent of DB instance status. For more information about maintenance status, see [Applying updates for a DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/USER_UpgradeDBInstance.Maintenance.md#USER_UpgradeDBInstance.OSUpgrades). ## Analyzing performance with DevOps Guru for RDS * **Important**\ We recommend that you test any changes on a test instance before modifying a production instance. In this way, you understand the impact of the change. # Monitoring events, logs, and database activity streams {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Monitoring Aurora logs * **Note**\ In some cases, logs contain hidden data. Therefore, the AWS Management Console might show content in a log file, but the log file might be empty when you download it. ## Monitoring Aurora API calls in CloudTrail * **Note**\ If you don't configure a trail, you can still view the most recent events in the CloudTrail console in **Event history**. # Security {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Database authentication * **Important**\ We strongly recommend that you do not use the master user directly in your applications. Instead, adhere to the best practice of using a database user created with the minimal privileges required for your application. ## Password management with Aurora and Secrets Manager * **Note**\ This policy enforces password management in AWS Secrets Manager at creation. However, you can still disable Secrets Manager integration and manually set a master password by modifying the cluster.\ To prevent this, include `rds:ModifyDBInstance`, `rds:ModifyDBCluster` in the Action block of the policy. Be aware, this prevents the user from applying any further modifications to existing clusters which do not have Secrets Manager integration enabled. ## Infrastructure security * **Note**\ If your DB instance is in a VPC but isn't publicly accessible, you can also use an AWS Site-to-Site VPN connection or an AWS Direct Connect connection to access it from a private network. For more information, see [Internetwork traffic privacy](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/inter-network-traffic-privacy.md). ## Controlling access with security groups * **Note**\ In an Aurora DB cluster, the VPC security group associated with the DB cluster is also associated with all of the DB instances in the DB cluster. If you change the VPC security group for the DB cluster or for a DB instance, the change is applied automatically to all of the DB instances in the DB cluster. * **Note**\ If your DB cluster is in a VPC but isn't publicly accessible, you can also use an AWS Site-to-Site VPN connection or an AWS Direct Connect connection to access it from a private network. For more information, see [Internetwork traffic privacy](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/inter-network-traffic-privacy.md). ## Master user account privileges * **Important**\ We strongly recommend that you do not use the master user directly in your applications. Instead, adhere to the best practice of using a database user created with the minimal privileges required for your application. * **Note**\ If you accidentally delete the permissions for the master user, you can restore them by modifying the DB cluster and setting a new master user password. For more information about modifying a DB cluster, see [Modifying an Amazon Aurora DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/Aurora.Modifying.md). ## Service-linked roles * **Note**\ You must configure permissions to allow an IAM entity (such as a user, group, or role) to create, edit, or delete a service-linked role. If you encounter the following error message:\ **Unable to create the resource. Verify that you have permission to create service linked role. Otherwise wait and try again later.**\ Make sure you have the following permissions enabled: * **Important**\ If you were using the Amazon Aurora service before December 1, 2017, when it began supporting service-linked roles, then Amazon Aurora created the AWSServiceRoleForRDS role in your account. To learn more, see [A new role appeared in my AWS account](https://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_roles.html#troubleshoot_roles_new-role-appeared). # Topics # Managing an Aurora DB cluster {% hint style="info" %} This page was generated from content adapted from the [AWS Developer Guide](https://github.com/awsdocs/amazon-aurora-user-guide.git) {% endhint %} ## Stopping and starting a cluster * **Important**\ You can stop a DB cluster for up to seven days. If you don't manually start your DB cluster after seven days, your DB cluster is automatically started so that it doesn't fall behind any required maintenance updates. ## Connecting an EC2 instance * **Note**\ You can only set up a connection between an EC2 instance and an Aurora DB cluster automatically by using the AWS Management Console. You can't set up a connection automatically with the AWS CLI or RDS API. ## Modifying an Aurora DB cluster * **Note**\ Most modifications can be applied immediately or during the next scheduled maintenance window. Some modifications, such as turning on deletion protection and changing the master password, are applied immediately—regardless of when you choose to apply them.\ If you're using SSL endpoints and change the DB cluster identifier, stop and restart the DB cluster to update the SSL endpoints. For more information, see [Stopping and starting an Amazon Aurora DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-cluster-stop-start.md). * **Note**\ Some settings only apply to DB instances. To change those settings, follow the instructions in [Modifying a DB instance in a DB cluster](#Aurora.Modifying.Instance). * **Note**\ Some settings only apply to DB instances. To change those settings, follow the instructions in [Modifying a DB instance in a DB cluster](#Aurora.Modifying.Instance). * **Important**\ If any of the deferred modifications require downtime, choosing **Apply immediately** can cause unexpected downtime for the DB instance. There is no downtime for the other DB instances in the DB cluster.\ Modifications that you defer aren't listed in the output of the `describe-pending-maintenance-actions` CLI command. Maintenance actions only include system upgrades that you schedule for the next maintenance window. * **Note**\ Some settings apply to the entire DB cluster. To change those settings, follow the instructions in [Modifying the DB cluster by using the console, CLI, and API](#Aurora.Modifying.Cluster). * **Note**\ Some settings apply to the entire DB cluster. To change those settings, follow the instructions in [Modifying the DB cluster by using the console, CLI, and API](#Aurora.Modifying.Cluster). * **Note**\ Additional settings are available if you are modifying an Aurora Serverless v1 or Aurora Serverless v2 DB cluster. For information about these settings, see [Modifying an Aurora Serverless v1 DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless.modifying.md) and [Managing Aurora Serverless v2](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless-v2-administration.md).\ Some settings aren't available for Aurora Serverless v1 and Aurora Serverless v2 because of their limitations. For more information, see [Limitations of Aurora Serverless v1](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless.md#aurora-serverless.limitations) and [Requirements for Aurora Serverless v2](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/aurora-serverless-v2.requirements.md). * **Note**\ You can't use the AWS Management Console to modify these settings for Aurora DB clusters. * **Note**\ You can't use the AWS Management Console to modify these settings for Aurora DB instances. ## Adding Aurora Replicas * **Note**\ Amazon Aurora also supports replication with an external database, such as an RDS DB instance. The RDS DB instance must be in the same AWS Region as Amazon Aurora. For more information, see [Replication with Amazon Aurora](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/Aurora.Replication.md). ## Managing performance and scaling * **Note**\ For Aurora MySQL, the `innodb_file_per_table` parameter affects how table storage is organized. When tables are part of the system tablespace, dropping the table doesn't reduce the size of the system tablespace. Thus, make sure to set `innodb_file_per_table` to 1 for Aurora MySQL DB clusters to take full advantage of dynamic resizing.\ For Aurora MySQL version 2.10 and higher, the InnoDB temporary tablespace is dropped and re-created on restart. This releases the space occupied by the temporary tablespace to the system, and then the cluster volume resizes. To take full advantage of the dynamic resizing feature, we recommend that you upgrade your DB cluster to Aurora MySQL version 2.10 or higher.\ The dynamic resizing feature reclaims space from dropped tablespaces over time, not immediately, when tables in those tablespaces are dropped. Space in the system tablespace isn't reclaimed because the system tablespace is never removed. Unreclaimed free space in a tablespace is reused when an operation needs space in that tablespace. * **Tip**\ If your applications frequently open and close connections, or keep a large number of long-lived connections open, we recommend that you use Amazon RDS Proxy. RDS Proxy is a fully managed, highly available database proxy that uses connection pooling to share database connections securely and efficiently. To learn more about RDS Proxy, see [Using Amazon RDS Proxy for Aurora](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/rds-proxy.md). ## Cloning a volume for an Aurora DB cluster * **Note**\ The [restore-db-cluster-to-point-in-time](https://docs.aws.amazon.com/cli/latest/reference/rds/restore-db-cluster-to-point-in-time.html) AWS CLI command only restores the DB cluster, not the DB instances for that DB cluster. You must invoke the [create-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-instance.html) command to create DB instances for the restored DB cluster, specifying the identifier of the restored DB cluster in `--db-cluster-identifier`. You can create DB instances only after the `restore-db-cluster-to-point-in-time` command has completed and the DB cluster is available. * **Note**\ The [restore-db-cluster-to-point-in-time](https://docs.aws.amazon.com/cli/latest/reference/rds/restore-db-cluster-to-point-in-time.html) AWS CLI command restores only the DB cluster, not the DB instances for that DB cluster. To create DB instances for the restored DB cluster, invoke the [create-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-instance.html) command. Specify the identifier of the restored DB cluster in `--db-cluster-identifier`.\ You can create DB instances only after the `restore-db-cluster-to-point-in-time` command has completed and the DB cluster is available. * **Note**\ The [RestoreDBClusterToPointInTime](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_RestoreDBClusterToPointInTime.html) RDS API operation restores only the DB cluster, not the DB instances for that DB cluster. To create DB instances for the restored DB cluster, invoke the [CreateDBInstance](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBInstance.html) RDS API operation. Specify the identifier of the restored DB cluster in `DBClusterIdentifier`. You can create DB instances only after the `RestoreDBClusterToPointInTime` operation has completed and the DB cluster is available. ## Maintaining an Aurora DB cluster * **Note**\ To defer a maintenance action, specify `undo-opt-in` for `--opt-in-type`. You can't specify `undo-opt-in` for `--opt-in-type` if the maintenance action has already started.\ To cancel a maintenance action, run the [modify-db-instance](https://docs.aws.amazon.com/cli/latest/reference/rds/modify-db-instance.html) AWS CLI command and specify `--no-auto-minor-version-upgrade`. * **Note**\ This setting is enabled by default. For each new cluster, choose the appropriate value for this setting. This value is based on its importance, expected lifetime, and the amount of verification testing that you do after each upgrade. * **Note**\ Staying current on all optional and mandatory updates might be required to meet various compliance obligations. We recommend that you apply all updates made available by RDS routinely during your maintenance windows. ## Rebooting an Aurora DB cluster or instance * **Note**\ If a DB instance isn't using the latest changes to its associated DB parameter group, the AWS Management Console shows the DB parameter group with a status of **pending-reboot**. The **pending-reboot** parameter groups status doesn't result in an automatic reboot during the next maintenance window. To apply the latest parameter changes to that DB instance, manually reboot the DB instance. For more information about parameter groups, see [Working with parameter groups](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/USER_WorkingWithParamGroups.md). * **Important**\ In Aurora MySQL 2.10 and higher, the reboot behavior is different: the reader DB instances typically remain available while you reboot the writer instance. Then you can reboot the reader instances at a convenient time. You can reboot the reader instances on a staggered schedule if you want some reader instances to always be available. For more information, see [Rebooting an Aurora MySQL cluster (version 2.10 and higher)](#aurora-mysql-survivable-replicas). * **Important**\ The change to reboot behavior in Aurora MySQL 2.10 and higher is different for Aurora global databases. If you reboot the writer instance for the primary cluster in an Aurora global database, the reader instances in the primary cluster remain available. However, the DB instances in any secondary clusters reboot at the same time. * **Tip**\ Aurora MySQL might still reboot some of the reader instances along with the writer instance if your cluster is processing a workload with high throughput.\ The reduction in the number of reboots applies during failover operations also. Aurora MySQL only restarts the writer DB instance and the failover target during a failover. Other reader DB instances in the cluster remain available to continue processing queries through connections to the reader endpoint. Thus, you can improve availability during a failover by having more than one reader DB instance in a cluster. * **Tip**\ For long-term monitoring, we recommend monitoring the `EngineUptime` metric for individual instances instead of at the cluster level. The cluster-level `EngineUptime` metric is set to zero when a new DB instance is added to the cluster. Such cluster changes can happen as part of maintenance and scaling operations such as those performed by Auto Scaling. ## Deleting Aurora clusters and instances * **Tip**\ You can keep a cluster with no DB instances to preserve your data without incurring CPU charges for the cluster. You can quickly start using the cluster again by creating one or more new DB instances for the cluster. You can perform Aurora-specific administrative operations on the cluster while it doesn't have any associated DB instances. You just can't access the data or perform any operations that require connecting to a DB instance. * **Tip**\ Even if all the DB instances are deleted, you can access the data by creating a new DB instance in the cluster. * **Note**\ When an Aurora Replica is deleted, its instance endpoint is removed immediately, and the Aurora Replica is removed from the reader endpoint. If there are statements running on the Aurora Replica that is being deleted, there is a three-minute grace period. Existing statements can finish during the grace period. When the grace period ends, the Aurora Replica is shut down and deleted. * **Note**\ You can't delete a DB cluster when deletion protection is enabled for it. For more information, see [Deletion protection for Aurora clusters](#USER_DeletionProtection).\ You can disable deletion protection by modifying the DB cluster. For more information, see [Modifying an Amazon Aurora DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/Aurora.Modifying.md). * **Note**\ When the status for a DB instance is `deleting`, its CA certificate value doesn't appear in the RDS console or in output for AWS CLI commands or RDS API operations. For more information about CA certificates, see [Using SSL/TLS to encrypt a connection to a DB cluster](https://github.com/kevinslin/aws-reference-notes/blob/main/services/amazon_aurora/UsingWithRDS.SSL.md). ## Tagging RDS resources * **Note**\ Currently, you can't tag RDS Proxies and RDS Proxy endpoints by using the AWS Management Console. * **Note**\ You can add a tag to a snapshot, however, your bill will not reflect this grouping. * **Note**\ In some cases, you might include a value for the `--tag-key` parameter of the [create-db-snapshot](https://docs.aws.amazon.com/cli/latest/reference/rds/create-db-snapshot.html) AWS CLI command. Or you might supply at least one tag to the [CreateDBSnapshot](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBSnapshot.html) API operation. In these cases, RDS doesn't copy tags from the source DB instance to the new DB snapshot. This functionality applies even if the source DB instance has the `--copy-tags-to-snapshot` (`CopyTagsToSnapshot`) option turned on.\ If you take this approach, you can create a copy of a DB instance from a DB snapshot. This approach avoids adding tags that don't apply to the new DB instance. You create your DB snapshot using the AWS CLI `create-db-snapshot` command (or the `CreateDBSnapshot` RDS API operation). After you create your DB snapshot, you can add tags as described later in this topic. * **Tip**\ You can use assigning tags and finding clusters that have those tags to reduce costs in other ways. For example, take the scenario with Aurora DB clusters used for development and testing. Here, you might designate some clusters to be deleted at the end of each day, or to have only their reader DB instances deleted. Or you might designate some to have their DB instances changed to small DB instance classes during times of expected low usage. ## Aurora updates * **Note**\ The time required to reboot your DB instance depends on the crash recovery process, database activity at the time of reboot, and the behavior of your specific DB engine. To improve the reboot time, we recommend that you reduce database activity as much as possible during the reboot process. Reducing database activity reduces rollback activity for in-transit transactions. --- [Next Page](/llms-full.txt/1)