diff --git a/.github/workflows/pr-title.yml b/.github/workflows/pr-title.yml index d4814de6b8..3e512357ef 100644 --- a/.github/workflows/pr-title.yml +++ b/.github/workflows/pr-title.yml @@ -16,4 +16,4 @@ jobs: - name: Pull Request title rules uses: deepakputhraya/action-pr-title@v1.0.2 with: - regex: '^(?:(feat)|(fix)|(docs)|(style)|(refactor)|(perf)|(test)|(build)|(ci)|(chore)|(revert))\((?:(javascript)|(php)|(java)|(cts)|(spec)|(script)|(ci))\): .+' + regex: '^(docs)|((?:feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)\((?:javascript|php|java|cts|spec|script|ci)\)): .+' diff --git a/README.md b/README.md index e118cdf00d..344afbbb9c 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,8 @@ yarn docker:setup Build docker image from [Dockerfile](./Dockerfile) +[How to add a new client](./docs/addNewClient.md) | [How to add a new language](./docs/addNewLanguage.md) | [Common Test Suite](./docs/CTS.md) | [Run the playground](./docs/playground.md) + ```bash yarn docker:build ``` @@ -111,31 +113,10 @@ yarn docker build:clients java recommend ## Testing clients -The clients can be tested inside the [`playground`](./playground) folder and with the [common test suite (CTS)](./doc/CTS.md) - -### Usage - -```bash -yarn docker playground -``` - -### JavaScript - -```bash -yarn docker playground javascript search -``` - -#### Browser - -```bash -yarn playground:browser -``` - -### Java +You can test our generated clients by running: -```bash -yarn docker playground java search -``` +- The playground [`playground`](./playground) ([Playground README](./docs/playground.md)) +- Tests with our [`Common Test Suite`](./tests/) ([CTS README](./docs/CTS.md)). # Troubleshooting diff --git a/doc/contribution_addNewClient.md b/doc/contribution_addNewClient.md deleted file mode 100644 index cf0c372822..0000000000 --- a/doc/contribution_addNewClient.md +++ /dev/null @@ -1,66 +0,0 @@ -# How to add a new client - -Here is a short guide on how to generate a new API client for your language - -## Find your default template - -Provided templates are a good starting point to generate a client, pick the one you find closer to what you'd like to achieve. - -You can pick a default template on the [openapi-generator's "generators" page](https://openapi-generator.tech/docs/generators) - -## Extract the template locally - -```bash -openapi-generator author template -g -o templates/ -``` - -Example for the JavaScript client - -```bash -openapi-generator author template -g typescript-node -o templates/javascript/ -``` - -## Add the language/client to the tool chain - -Add each client in the file `openapitools.json`, following the others client structure. - -Don't forget to write tests for it using the [CTS](./CTS.md) - -## Customize the template - -API clients require a custom Algolia logic in order to seamlessly work with our engine. - -**The first thing to do is strip as much code as possible, because the generation include lots of useless feature.** - -You will need to implement: - -- An `init` method -- The `retry strategy` with your custom transporter -- Handle client with different hosts (e.g. `personalization`) -- At least 2 requester: - - http requester, using the standard library - - echo requester that send the request back, used by the CTS -- A logger that the user can swap -- More to come... - -### Init method - -By default, OpenAPI will put the `AppId` and `ApiKey` in every method, but we want an init method that takes those parameters and put them in the header of every requests, and create the right hosts. - -To do this, change the constructor of the client in the `mustache` file to accept and store them. - -See this PR of the first JavaScript implementation for reference: https://github.com/algolia/api-client-automation-experiment/pull/7 - -### Retry strategy - -The retry strategy cannot be generated and needs to be implemented outside of the generated client folder. You need to add your transporter to the utils folder of your language, and update the `.mustache` template files accordingly. - -See this PR of the first JavaScript implementation for reference: https://github.com/algolia/api-client-automation-experiment/pull/9 - -### Handle client with different hosts - -Algolia search operations targets the default appId host (`${appId}-dsn.algolia.net`, `${appId}.algolia.net`, etc.), while clients like `personalization` have their own `host`. - -Since `.mustache` are logicless files, you need to provide the variable yourself in the [`openapitools.json`](../openapitools.json) config file and implement the `if` based logic. - -See this PR of the first JavaScript implementation for reference: https://github.com/algolia/api-client-automation-experiment/pull/25 diff --git a/doc/CTS.md b/docs/CTS.md similarity index 100% rename from doc/CTS.md rename to docs/CTS.md diff --git a/docs/addNewClient.md b/docs/addNewClient.md new file mode 100644 index 0000000000..deac79cd0d --- /dev/null +++ b/docs/addNewClient.md @@ -0,0 +1,96 @@ +# How to add a new client + +Adding a client requires a few manual steps in order to setup the tooling, generation scripts and properly generate the code. We recommend getting inspirations from existing clients such as `javascript-recommend`. + +> See [README](../README.md) to [`setup the repository tooling`](../README.md#setup-repository-tooling) and [`setup dev environment`](../README.md#setup-dev-environment). + +## 1. Writing specs + +We recommend to have a look at [existing spec files](../specs/). The `bundled` folder is automatically generated, manual changes shouldn't be done in these files. + +### [common spec folder](../specs/common/) + +Properties that are common to Algolia or used in multiple clients. + +### `` spec folder + +> Example with the [search client spec](../specs/search/) + +#### `spec.yml` file + +This file is the entry point of the client spec, it contains `servers`, `paths` and other specific imformations of the API. We recommend to copy an existing [`spec.yml` file](../specs/search/spec.yml) to get started. + +#### ``/common folder + +Properties that are common to the client. + +#### ``/paths folder + +Path definition of the paths defined in the [spec file](#specyml-file) + +#### Guidelines + +- **Endpoints**: Each file should contain `operationId`s for a single endpoint, but multiple methods are allowed. +- **Name**: If the path file only contain one method, we name the file same as the `operationId`, but we try to make it less specific if there is multiple. +- **Description/Summary**: `operationId`s must have both description and summary. +- **Tags**: The `tags` must match the `` spec folder. +- **Complex objects**: Complex objects (nested arrays, nested objects, etc.) must be referenced (`$ref`) in the `operantionId` file and not inlined. This is required to provide a great naming. + +## 2. Configuring the environment + +After setting up the dev environment from [README](../README.md) and [writing your spec files](#1-writing-specs), you need to update the configuration files to properly generate clients that are maintainable. + +### Generation config + +[openapitools.json](../openapitools.json) hosts the configuration of all of the generated clients with their available languages. + +#### `generators` + +Generators are referenced by key with the following pattern `-`. + +> TODO: Automate this step. + +You can copy an existing object of a client and replace the `` value with the one you'd like to generate. + +| Option | Type | Language | Example | Definition | +| ------------------ | :-----: | :--------: | :-----------------------------: | :---------------------------------------------------------------------------------------------------------------------------------------------- | +| output | string | Common | `path/to/client/client-sources` | The output path of the client. | +| glob | string | Common | `path/to/spec/sources.yml` | The path of the bundled spec file. | +| gitRepoId | string | Common | `algoliasearch-client-java-2` | The name of the repository under the Algolia org. | +| apiName | string | JavaScript | `search` | The lowercase name of the exported API. | +| capitalizedApiName | string | JavaScript | `Search` | The capitalized name of the exported API. | +| packageVersion | string | JavaScript | `1.2.3` | The version you'd like to publish the first iteration of the generated client. It will be automatically incremented. | +| packageName | string | common | `AlgoliaSearch` | Name of the API package, used in [CTS](./CTS.md). | +| hasRegionalHost | boolean | common | `false` | Automatically guessed from `servers` in spec. `undefined` implies that hosts used will required the `appId`, regional hosts are used otherwise. | +| isDeHost | boolean | common | `false` | Automatically guessed from `servers` in spec. `undefined` implies that `eu` is the regional host, `de` otherwise. | +| host | string | common | `crawler` | Automatically guessed from `servers` in spec. | +| topLevelDomain | string | common | `io` | Automatically guessed from `servers` in spec. | + +### GitHub actions + +The built clients are cached with the [Cache GitHub Action](../.github/actions/cache/action.yml) to avoid unnecessary CI tasks. + +> TODO: Automate this step + +You can copy [an existing client caching step](../.github/actions/cache/action.yml) or edit the following example: + +```yaml +- name: Restore built client + if: ${{ inputs.job == 'cts' }} + uses: actions/cache@v2 + with: + path: /home/runner/work/api-clients-automation/api-clients-automation/clients/// + key: ${{ runner.os }}-${{ env.CACHE_VERSION }}---${{ hashFiles('clients///**') }}-${{ hashFiles('specs/bundled/.yml') }} +``` + +## 3. Generate new client + +> You can find more commands in the [README](../README.md#generate-all-clients). + +```bash +yarn docker generate +``` + +## 4. Implementing the Common Test Suite + +See [CTS.md](./CTS.md) for more informations. diff --git a/docs/addNewLanguage.md b/docs/addNewLanguage.md new file mode 100644 index 0000000000..0ad9e7872c --- /dev/null +++ b/docs/addNewLanguage.md @@ -0,0 +1,77 @@ +# How to add support of a new language + +This repository leverages [openapi-generator](https://openapi-generator.tech/) to generate API clients. + +> See [README](../README.md) to [`setup the repository tooling`](../README.md#setup-repository-tooling) and [`setup dev environment`](../README.md#setup-dev-environment). + +> If not done already, [install openapi-generator](https://openapi-generator.tech/docs/installation/) + +## Find a template to start with + +Provided templates should be a good starting point to generate a client but make sure to implement the [Algolia requirements](#algolia-requirements) to make it work properly. + +You can pick a default template on the [openapi-generator's "generators" page](https://openapi-generator.tech/docs/generators) + +### Extract the template locally + +```bash +openapi-generator author template -g -o templates/ +``` + +Example for the JavaScript client with the `typescript-node` template: + +```bash +openapi-generator author template -g typescript-node -o templates/javascript/ +``` + +## Update the generator config + +Add each client in the file [`openapitools.json`](../openapitools.json), following the others client structure. + +> See [How to add a new client](./addNewClient.md) for informations regarding this file + +### Algolia requirements + +### Strip code + +The generator includes a lot of features that won't be used with the Algolia engine: + +- Multiple authentication methods: `appId`/`apiKey` are the only authentication methods, located in the requests headers. +- Built-in transporters: A [retry strategy](#retry-strategy) is required to target the DSNs of an `appId`, along with other transporter details listed below. +- File support, payload format etc.: Requests only require `JSON` support to communicate with the engine. + +**DX is key, make sure to provide a linter and formatting tools, with consistent method usage based on the language.** + +### Init method + +By default, OpenAPI will put the `AppId` and `ApiKey` in every method parameters, but the clients to be initialized with those values and put them in the header of every requests, with the right hosts. + +The constructor of the client can be edited (from the `.mustache` files) to accept and store those values. + +- [First implementation on the JavaScript client](https://github.com/algolia/api-clients-automation/pull/7) +- [Current implementation on the JavaScript client](https://github.com/algolia/api-clients-automation/blob/main/clients/algoliasearch-client-javascript/packages/client-search/src/searchApi.ts#L110-L125) + +### Retry strategy + +The retry strategy cannot be generated and needs to be implemented outside of the generated client folder. You can achieve this by creating a `utils` (or any naming that you find relevant) folder and add a transporter and retry strategy logic to it. + +- [First implementation on the JavaScript client](https://github.com/algolia/api-clients-automation/pull/9) +- [Current implementation on the PHP client](https://github.com/algolia/api-clients-automation/tree/main/clients/algoliasearch-client-php/lib/RetryStrategy) + +### Different client hosts + +Some Algolia clients (search and recommend) targets the default appId host (`${appId}-dsn.algolia.net`, `${appId}.algolia.net`, etc.), while clients like `personalization` have their own regional `host` (`eu` | `us` | `de`). + +As the generator does not support reading `servers` in a spec file, hosts methods and variables are extracted with a custom script and create variables for you to use in the mustache templates, [read more here](./addNewClient.md#generators). + +### Requesters + +> TODO: informations + +### Logger + +> TODO: informations + +### **DX** + +We require the generated API clients to have an up-to-date usage with their ecosystem, make sure to provide correct tooling to lint and format generated code. diff --git a/doc/changelogs/java.md b/docs/changelogs/java.md similarity index 100% rename from doc/changelogs/java.md rename to docs/changelogs/java.md diff --git a/doc/changelogs/javascript.md b/docs/changelogs/javascript.md similarity index 100% rename from doc/changelogs/javascript.md rename to docs/changelogs/javascript.md diff --git a/doc/changelogs/php.md b/docs/changelogs/php.md similarity index 100% rename from doc/changelogs/php.md rename to docs/changelogs/php.md diff --git a/docs/playground.md b/docs/playground.md new file mode 100644 index 0000000000..5ebc73d6af --- /dev/null +++ b/docs/playground.md @@ -0,0 +1,28 @@ +# Playground + +All of the existing clients should have an active playground for you to test generated clients, if it's not the case, consider contributing or letting us know! + +## Usage + +```bash +yarn docker playground +``` + +### JavaScript + +```bash +yarn docker playground javascript search +``` + +### Java + +```bash +yarn docker playground java search +``` + +## Add new playground + +To add a new supported language to the playground, you need to: + +- Create a new folder with your `` in [the playground folder](../playground/) +- Edit the [playground script](../scripts/playground.sh) with the command to run it. diff --git a/openapitools.json b/openapitools.json index 4e3fb00e4a..74a71578b0 100644 --- a/openapitools.json +++ b/openapitools.json @@ -395,4 +395,4 @@ } } } -} \ No newline at end of file +} diff --git a/scripts/release/process-release.ts b/scripts/release/process-release.ts index 50e300250a..7431774aa9 100755 --- a/scripts/release/process-release.ts +++ b/scripts/release/process-release.ts @@ -92,7 +92,7 @@ fs.writeFileSync( // update changelogs new Set([...Object.keys(versionsToRelease), ...langsToUpdateRepo]).forEach( (lang) => { - const filePath = path.resolve(ROOT_DIR, `doc/changelogs/${lang}.md`); + const filePath = path.resolve(ROOT_DIR, `docs/changelogs/${lang}.md`); const header = versionsToRelease[lang!] ? `## ${versionsToRelease[lang!].next}` : `## ${new Date().toISOString().split('T')[0]}`;