From 6fe8510f2f3071cf15a89413f2020216c3388fcb Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Wed, 21 Mar 2018 13:29:23 -0500 Subject: [PATCH 1/5] Document API policies --- docs/api-reference/index.rst | 44 ++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/docs/api-reference/index.rst b/docs/api-reference/index.rst index b4de420ece7e..f5408c08e633 100644 --- a/docs/api-reference/index.rst +++ b/docs/api-reference/index.rst @@ -4,6 +4,50 @@ API Reference Warehouse has several API endpoints. See :doc:`../application` for the parts of Warehouse that generate them. +.. contents:: :local: + +API Policies +------------ + +Please be aware of the following PyPI API Policies: + +Caching +~~~~~~~ + +All API requests are cached. Requests to the JSON, RSS or Legacy APIs are +cached by our CDN provider. You can determine if you've hit the cache based on +the ``X-Cache`` and ``X-Cache-Hist`` headers in the response. + +All API requests also provide an ``ETag`` header. If you're making a lot of +repeated requests, please ensure your API consumer will respect this header to +determine whether to actually repeat a request or not. + +Rate Limiting +~~~~~~~~~~~~~ + +Due to the heavy caching and CDN use, there is currently no rate limiting of +PyPI APIs. + +However, if you plan to make a lot of requests to a PyPI API, please adhere to +the following suggestions: + +* Set your consumer's ``User-Agent`` header to uniquely identify your requests +* Try not to make a lot of requests (thousands) in a short amount of time + (minutes). Generally PyPI can handle it, but it's preferred to make requests + in serial over a longer amount of time if possible. + +API Preference +~~~~~~~~~~~~~~ + +For periodically checking for new packages or updates to existing packages, +please use our RSS feeds. + +If at all possible, it is recommended to use the JSON/RSS/Legacy APIs over +XML-RPC. + +Available APIs +-------------- + .. toctree:: :maxdepth: 2 From 119ab420375702eccc9d6c414d18c55630db93cf Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Wed, 21 Mar 2018 13:42:03 -0500 Subject: [PATCH 2/5] Address @ewdurbin's feedback --- docs/api-reference/index.rst | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/docs/api-reference/index.rst b/docs/api-reference/index.rst index f5408c08e633..f6a954d4c79a 100644 --- a/docs/api-reference/index.rst +++ b/docs/api-reference/index.rst @@ -16,19 +16,25 @@ Caching All API requests are cached. Requests to the JSON, RSS or Legacy APIs are cached by our CDN provider. You can determine if you've hit the cache based on -the ``X-Cache`` and ``X-Cache-Hist`` headers in the response. +the ``X-Cache`` and ``X-Cache-Hits`` headers in the response. -All API requests also provide an ``ETag`` header. If you're making a lot of -repeated requests, please ensure your API consumer will respect this header to -determine whether to actually repeat a request or not. +Requests to the JSON, RSS and Legacy APIs also provide an ``ETag`` header. If +you're making a lot of repeated requests, please ensure your API consumer will +respect this header to determine whether to actually repeat a request or not. + +The XML-RPC API does not have the ability to indicate cached responses. Rate Limiting ~~~~~~~~~~~~~ Due to the heavy caching and CDN use, there is currently no rate limiting of -PyPI APIs. +PyPI APIs at the edge. The XML-RPC API may be rate limited if usage is causing +degradation of service. + +In addition, PyPI reserves the right to temporarily or permanently prohibit a +consumer based on irresponsible activity. -However, if you plan to make a lot of requests to a PyPI API, please adhere to +If you plan to make a lot of requests to a PyPI API, please adhere to the following suggestions: * Set your consumer's ``User-Agent`` header to uniquely identify your requests @@ -42,8 +48,8 @@ API Preference For periodically checking for new packages or updates to existing packages, please use our RSS feeds. -If at all possible, it is recommended to use the JSON/RSS/Legacy APIs over -XML-RPC. +No new integrations should use the XML-RPC APIs as they are planned for +deprecation. Existing consumers should migrate to JSON/RSS/Legacy APIs. Available APIs -------------- From 60dbc1e788982362e5b357cfe2fd0fddaa4b56c2 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Wed, 21 Mar 2018 15:29:25 -0500 Subject: [PATCH 3/5] Add contact information to user-agent header --- docs/api-reference/index.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/api-reference/index.rst b/docs/api-reference/index.rst index f6a954d4c79a..b0fd2409bbce 100644 --- a/docs/api-reference/index.rst +++ b/docs/api-reference/index.rst @@ -37,7 +37,8 @@ consumer based on irresponsible activity. If you plan to make a lot of requests to a PyPI API, please adhere to the following suggestions: -* Set your consumer's ``User-Agent`` header to uniquely identify your requests +* Set your consumer's ``User-Agent`` header to uniquely identify your requests. + Adding your contact information to this value would be helpful as well. * Try not to make a lot of requests (thousands) in a short amount of time (minutes). Generally PyPI can handle it, but it's preferred to make requests in serial over a longer amount of time if possible. From 21399851b91f997624cf74067393718b3f0c668b Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Wed, 21 Mar 2018 15:29:35 -0500 Subject: [PATCH 4/5] Add note about XML-RPC deprecation --- docs/api-reference/xml-rpc.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/api-reference/xml-rpc.rst b/docs/api-reference/xml-rpc.rst index 81fe1ba39737..a7576e5c1dd0 100644 --- a/docs/api-reference/xml-rpc.rst +++ b/docs/api-reference/xml-rpc.rst @@ -2,6 +2,11 @@ PyPI's XML-RPC methods ====================== +.. note:: + The XML-RPC API will be deprecated in the future. Use of this API is not + recommended, and existing consumers of the API should migrate to the RSS + and/or JSON APIs instead. + Example usage:: >>> import xmlrpclib From b5070894341ab3724f5647130ca3072e150ff997 Mon Sep 17 00:00:00 2001 From: Dustin Ingram Date: Wed, 21 Mar 2018 15:40:38 -0500 Subject: [PATCH 5/5] Add suggestion about mirrors --- docs/api-reference/index.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/api-reference/index.rst b/docs/api-reference/index.rst index b0fd2409bbce..b98f5d285e3c 100644 --- a/docs/api-reference/index.rst +++ b/docs/api-reference/index.rst @@ -42,6 +42,10 @@ the following suggestions: * Try not to make a lot of requests (thousands) in a short amount of time (minutes). Generally PyPI can handle it, but it's preferred to make requests in serial over a longer amount of time if possible. +* If your consumer is actually an organization or service that will be + downloading a lot of packages from PyPI, consider `using your own index + mirror or cache + `_. API Preference ~~~~~~~~~~~~~~