Strategies to Optimize Rate Limits for Application Workloads

When architecting your application for Selling Partner API, you must keep the per API resource rate limits in mind, particularly the types of operations and the frequency with which they are called. When the allotted rate limit for an API operation exceeds, you’ll receive an error response and the call will be throttled. Excessive API throttling can result in job failure, delays, and operational inefficiencies that ultimately cost your organization time and money. Selling Partner API maintains a per API resource quota for each Selling Partners to help with the availability of Selling Partner APIs, and prevent accidental burdening of the APIs.

When request submissions exceed the steady-state request rate and burst limits, SP-API fails the limit-exceeding requests and returns 429 Too Many Requests error responses to the client. If you receive these error responses, you can resubmit the failed requests in a way that is rate limiting while also complying with rate limits.

The following sections describe how to use the defined rate limits for each SP-API operation in an optimized manner and provides strategies for managing API throttling in your applications workloads.

Implement a rate limiter code

A rate limiter is used to manage a high volume of traffic allowing N requests per second based on per API resource limits. Implementing the rate limiter allows the calling pattern to be consistent (smooth) over time instead of having peaks through the traffic. You can use the per API rate limit as the guideline for each API in the rate limiter. For more information on usage plans and how rate limits are used in SP-API, refer to Usage Plans and Rate Limits in the SP-API.

In the Selling Partner API Authentication/Authorization Library, you can use the interface to set and get rateLimit configurations that are used with RateLimiter. RateLimiter is used on the client side to restrict the rate at which requests are made. The RateLimitConfiguration accepts rateLimitPermit (set the rate at which requests are made) and waitTimeOutInMilliSeconds (set up a pause between calls).

The following code snippet provides an example configuration for a rate limiter.

import com.google.common.util.concurrent.RateLimiter;
import com.amazon.SellingPartnerAPIAA.RateLimitConfiguration;
.
.
.
/**
* Sets the RateLimiter
* A rate limiter is used to manage a high volume of traffic allowing 
* N requests per second
* @return Api client
**/
public ApiClient setRateLimiter(RateLimitConfiguration rateLimitConfiguration) {
    if (rateLimitConfiguration != null) {
        rateLimiter = RateLimiter.create(rateLimitConfiguration.getRateLimitPermit());
//Add rateLimiter to httpclient interceptor for execute
            RateLimitInterceptor rateLimiterInterceptor = 
            new RateLimitInterceptor(rateLimiter, rateLimitConfiguration);
            httpClient.interceptors().add(rateLimiterInterceptor);
        }
     return this;
}    
.
.
.
class RateLimitInterceptor implements Interceptor {
    RateLimiter rateLimiter;
    RateLimitConfiguration rateLimitConfiguration;

    public RateLimitInterceptor(RateLimiter rateLimiter, 
    RateLimitConfiguration rateLimitConfiguration) {
        this.rateLimiter = rateLimiter;
        this.rateLimitConfiguration = rateLimitConfiguration;
    }

    @Override
    public Response intercept(Chain chain) throws IOException {
        if (rateLimitConfiguration.getTimeOut() == Long.MAX_VALUE) {
            rateLimiter.acquire();
        } else {
            try {
                if (!rateLimiter.tryAcquire(rateLimitConfiguration.getTimeOut(), 
                TimeUnit.MILLISECONDS)) {
                    throw new ApiException(
                    "Throttled as per the ratelimiter on client");
                }
            } catch (ApiException e) {
                e.printStackTrace();
            }
        }
        return chain.proceed(chain.request());
    }
}  

Workload-based mechanisms to manage throttles

The following additional mechanisms can help you manage throttles depending on how your workload is architected.

Event-based workload

Monitor with Notifications API: With event-based tasks, you can monitor notifications using the Notifications API and perform actions based on certain conditions.

The Selling Partner API for Notifications lets you subscribe to notifications that are relevant to a selling partner's business. You can create a destination to receive notifications, subscribe to notifications, delete notification subscriptions, and more. Instead of polling for information, your application can receive information directly from Amazon when an event invokes a notification to which you are subscribed. This approach creates a holistic workflow that coordinates and makes the most of the resources you already have in place.

As a best practice to avoid polling, you should use notifications when developing applications for the Orders API and Reports API. The following notifications subscriptions can help provide insight into changes:

• ORDER_CHANGE is sent whenever there is an important change in the order. Important changes include order status changes and buyer requested cancelations.
• REPORT_PROCESSING_FINISHED notifies you when a requested report is ready.
• PRODUCT_TYPE_DEFINITIONS_CHANGE notifies you of a new product type or product type version.
• LISTINGS_ITEM_STATUS_CHANGE notifies you when there is a listing status change including buyable transition and listing create or delete for any SKU that the selling partner has.

There are many more notifications available for your application to leverage. For more information, refer to the Notifications API v1 Use Case Guide.

Implement retry techniques: If a Selling Partner API method rate limit is exceeded, you will receive a RequestLimitExceeded response and the API call will be throttled. To avoid impact to your workloads, you should proactively implement retry techniques. The following techniques increase the reliability of your application and can reduce operational costs for your organization.

• Retry: Implement automatic retry logic. You can configure the retry settings by adding a small delay and queuing between your requests.
• Exponential backoff: In addition to simple retries, implement an exponential backoff algorithm for better flow control. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses. Exponential backoff can lead to very long backoff times, because exponential functions grow quickly. You should implement a maximum delay interval and a maximum number of retries. The maximum delay interval and maximum number of retries are not necessarily fixed values. They should be set based on the operation being performed and other local factors, including network latency.
• Jitter: Retries can be ineffective if all clients retry at the same time. To avoid this problem, we employ jitter, a random amount of time before making or retrying a request to help prevent large bursts by spreading out the arrival rate. Most exponential backoff algorithms use jitter to prevent successive collisions. For more information, refer to Exponential Backoff and Jitter.

User input-based workload

Monitor and manage the user input based workload. Based on the seller level throughput, you can decide how to distribute the throughput with input from users. We recommend having some indicator of when the throughput is at 90% capacity, so that you are able to add backoff time for the throughput to become available for the API resource. Generally, you should be able to allocate resources to meet your customers’ demands as long as they’re not generating programmatic traffic.

Use batch operations

Use bulk get endpoints such as /reports/2021-06-30/reports?, which lets you get data in bulk in a single API request. SP-API offers the following options for aggregated data:

• Orders data in form of Order Reports, Order Tracking Reports, and Pending Order Reports.
• Inventory and Listings data in form of Inventory Reports with options for active/Inactive listings.

Use bulk upload resources when applicable. Feeds feeds/2021-06-30/feeds? API allows you to carry bulk tasks in a single API request. SP-API offers the following options for bulk uploads:

• Listings Feed to create an Amazon listing, create a new record (ASIN), or update inventory levels.
• Orders Feed to acknowledge, cancel, or issue a refund for an Amazon Order.
• Fulfillment by Amazon Feeds allow you to submit, cancel Amazon FBA order or create, cancel bulk shipment plans.

Use generic paths to get the Restricted Data Token (RDT). When implementing Tokens API for secure resources, use generic paths to get the Restricted Data Token. In the following example, you get a single RDT for the mentioned data elements for all orders for a given seller.

POST https://sellingpartnerapi-na.amazon.com/tokens/2021-03-01/restrictedDataToken 
    { 
        "restrictedResources": 
            [ 
                { 
                    "method": "GET", 
                    "path": "/orders/v0/orders", 
                    "dataElements": ["buyerInfo", "shippingAddress"]
                } 
            ] 
    }

General best practices to manage throttling

Monitor API activity against your rate limit. You can implement an API dashboard to monitor your API activity against your rate limit in the last 24 hours. You can also use the following response headers to confirm the application's current rate limit, and you can build a monitor that tracks the number of requests remaining in the current counter. In the following example, the current request count per second is five.

X-AMZ-Rate-Limit: 5

Finally, if you are using AWS services, you can monitor API activity with Amazon CloudWatch metrics.

Catch errors caused by rate limiting. For each request, you can check to see if you've reached the rate limit. If you get a response code of 429, Too Many Requests, you've met the rate limit. It's best practice to include code in your script that catches 429 responses. If your script ignores the Too Many Requests error and keeps trying to make requests, you might start getting null errors. At that point, the error information won't be useful in diagnosing the problem.

Reduce the number of API requests. Make sure you only make requests that you need. Try these tips for reducing the number of requests:

• Use the throughput as you grow and scale your usage.
• Optimize your code to eliminate any unnecessary API calls. For example, are some requests getting data items that aren't used in your application?
• Cache frequently used data. You can cache data on your servers using Object level storage like Amazon S3. You can also save relatively static information in a database or serialize it in a file.
• Stagger SP-API requests in a queue and do other processing tasks while waiting for the next queued job to run.
• Configure your code to stop making additional API requests when throttled until enough time has passed to retry.

Considerations for application design

Consider these tips for managing throttling and rate limits when you design your app:

• Regulate the rate of your requests for smoother distribution when using multiple SP-APIs.
• Remember, some Selling Partner APIs may have functional limits on the frequency that data can be requested or written for a given seller, so we don't suggest using all of your throughput on one seller at a time.