docs: Add more examples how to use this SDK (#15)

* docs(examples): Add example environment configuration and README files for SDK usage

Signed-off-by: Eden Reich <[email protected]>

* docs: Update README to include pre-requisites for using the Typescript SDK examples

Signed-off-by: Eden Reich <[email protected]>

* docs(examples): Update example links in README and add list example README

Signed-off-by: Eden Reich <[email protected]>

* docs(examples): Add examples for chat, list, and MCP using Inference Gateway SDK

- Created a chat example with package.json, tsconfig.json, and index.ts to demonstrate chat completions and streaming responses.
- Added a list example with package.json, tsconfig.json, and index.ts for listing models and MCP tools.
- Included README.md files for both chat and list examples with instructions for getting started.
- Added .gitignore files to exclude node_modules in both list and MCP examples.

Signed-off-by: Eden Reich <[email protected]>

* docs(exmaples): Add sample files and configuration for MCP example

- Created a new directory with sample files for the MCP filesystem server, including `sample.txt`, `data.json`, and `config.yaml`.
- Added a README.md file to describe the purpose and usage of the sample data.
- Implemented a TypeScript configuration file (`tsconfig.json`) for the MCP example.
- Created a package.json and package-lock.json for managing dependencies in the examples directory.
- Updated the OpenAPI specification to include new endpoints and improved formatting for consistency.

Signed-off-by: Eden Reich <[email protected]>

* docs(examples): Update LLM model references to remove provider prefix

Signed-off-by: Eden Reich <[email protected]>

* docs(mcp): Add note to ensure no containers are running before starting the example

Signed-off-by: Eden Reich <[email protected]>

* feat: Support both local tools and remote MCP tools; Add onMCPTool callback

- Created a new MCP Web Search Server with a package.json file.
- Updated the main MCP package.json to include example scripts for remote and local tools.
- Added a .gitignore file in the shared directory to exclude unnecessary files.
- Removed outdated README.md, config.yaml, data.json, and sample.txt files from the shared directory.
- Introduced a new sample_sales_data.csv file for demonstration purposes.
- Enhanced the client.ts to support MCP tool callbacks and improved streaming response processing.

Signed-off-by: Eden Reich <[email protected]>

* docs(examples): Refactor MCP examples and client for improved error handling and token tracking

- Updated docker-compose.yml to set environment to production and commented out optional Ollama service.
- Enhanced example-mcp-tools.ts to use console.info for logging and added model availability checks.
- Modified index.ts to implement token tracking for prompt and completion tokens, displaying usage metrics at the end.
- Improved filesystem MCP server (index.js) to use console.info for logging and ensure consistent messaging.
- Updated web-search MCP server (index-http.js and index.js) to use console.info for logging and improved request handling.
- Enhanced InferenceGatewayClient to handle mid-stream errors and added type definitions for stream chunks that may contain errors.

Signed-off-by: Eden Reich <[email protected]>

---------

Signed-off-by: Eden Reich <[email protected]>

Author: edenreich

.gitattributes

@@ -1,2 +1,5 @@
 .devcontainer/** linguist-vendored=true
+src/examples/** linguist-vendored=true
+
 src/types/generated/** linguist-generated=true
+

.gitignore

@@ -1,3 +1,4 @@
 node_modules
 dist
 coverage
+**/.env

README.md

@@ -7,12 +7,14 @@ An SDK written in TypeScript for the [Inference Gateway](https://github.com/eden
   - [Usage](#usage)
     - [Creating a Client](#creating-a-client)
     - [Listing Models](#listing-models)
+    - [Listing MCP Tools](#listing-mcp-tools)
     - [Creating Chat Completions](#creating-chat-completions)
     - [Streaming Chat Completions](#streaming-chat-completions)
     - [Tool Calls](#tool-calls)
     - [Proxying Requests](#proxying-requests)
     - [Health Check](#health-check)
     - [Creating a Client with Custom Options](#creating-a-client-with-custom-options)
+    - [Examples](#examples)
   - [Contributing](#contributing)
   - [License](#license)
 
@@ -51,7 +53,7 @@ try {
   console.log('All models:', models);
 
   // List models from a specific provider
-  const openaiModels = await client.listModels(Provider.OpenAI);
+  const openaiModels = await client.listModels(Provider.openai);
   console.log('OpenAI models:', openaiModels);
 } catch (error) {
   console.error('Error:', error);
@@ -235,7 +237,7 @@ To proxy requests directly to a provider:
 import { InferenceGatewayClient, Provider } from '@inference-gateway/sdk';
 
 const client = new InferenceGatewayClient({
-  baseURL: 'http://localhost:8080/v1',
+  baseURL: 'http://localhost:8080',
 });
 
 try {
@@ -261,7 +263,7 @@ To check if the Inference Gateway is running:
 import { InferenceGatewayClient } from '@inference-gateway/sdk';
 
 const client = new InferenceGatewayClient({
-  baseURL: 'http://localhost:8080/v1',
+  baseURL: 'http://localhost:8080',
 });
 
 try {
@@ -292,6 +294,10 @@ const clientWithHeaders = client.withOptions({
 });
 ```
 
+### Examples
+
+For more examples, check the [examples directory](./examples).
+
 ## Contributing
 
 Please refer to the [CONTRIBUTING.md](CONTRIBUTING.md) file for information about how to get involved. We welcome issues, questions, and pull requests.

eslint.config.mjs

@@ -10,13 +10,26 @@ export default [
     plugins: {
       prettier: prettier,
     },
+    languageOptions: {
+      globals: {
+        console: 'readonly',
+        process: 'readonly',
+        Buffer: 'readonly',
+        __dirname: 'readonly',
+        __filename: 'readonly',
+        global: 'readonly',
+        module: 'readonly',
+        require: 'readonly',
+        exports: 'readonly',
+      },
+    },
     rules: {
       'prettier/prettier': 'error',
       ...eslint.configs.recommended.rules,
     },
   },
   {
-    files: ['**/*.ts'],
+    files: ['src/**/*.ts', 'tests/**/*.ts'],
     languageOptions: {
       parser: tsParser,
       parserOptions: {
@@ -25,12 +38,28 @@ export default [
         sourceType: 'module',
       },
       globals: {
-        // Add Jest globals
+        console: 'readonly',
+        process: 'readonly',
+        Buffer: 'readonly',
+        __dirname: 'readonly',
+        __filename: 'readonly',
+        global: 'readonly',
+        module: 'readonly',
+        require: 'readonly',
+        exports: 'readonly',
         describe: 'readonly',
         it: 'readonly',
         expect: 'readonly',
         beforeEach: 'readonly',
+        afterEach: 'readonly',
+        beforeAll: 'readonly',
+        afterAll: 'readonly',
         jest: 'readonly',
+        Headers: 'readonly',
+        fetch: 'readonly',
+        ReadableStream: 'readonly',
+        Response: 'readonly',
+        Request: 'readonly',
       },
     },
     plugins: {
@@ -45,10 +74,54 @@ export default [
       ...tseslint.configs.recommended.rules,
     },
   },
+  {
+    files: ['examples/**/*.ts'],
+    languageOptions: {
+      parser: tsParser,
+      parserOptions: {
+        ecmaVersion: 'latest',
+        sourceType: 'module',
+      },
+      globals: {
+        console: 'readonly',
+        process: 'readonly',
+        Buffer: 'readonly',
+        __dirname: 'readonly',
+        __filename: 'readonly',
+        global: 'readonly',
+        module: 'readonly',
+        require: 'readonly',
+        exports: 'readonly',
+      },
+    },
+    plugins: {
+      '@typescript-eslint': tseslint,
+    },
+    rules: {
+      '@typescript-eslint/no-unused-vars': [
+        'error',
+        { argsIgnorePattern: '^_' },
+      ],
+    },
+  },
   {
     files: ['**/*.test.ts'],
-    env: {
-      'jest/globals': true,
+    languageOptions: {
+      globals: {
+        describe: 'readonly',
+        it: 'readonly',
+        expect: 'readonly',
+        beforeEach: 'readonly',
+        afterEach: 'readonly',
+        beforeAll: 'readonly',
+        afterAll: 'readonly',
+        jest: 'readonly',
+        Headers: 'readonly',
+        fetch: 'readonly',
+        ReadableStream: 'readonly',
+        Response: 'readonly',
+        Request: 'readonly',
+      },
     },
   },
 ];

examples/.env.example

@@ -0,0 +1,48 @@
+
+# General settings
+ENVIRONMENT=development
+ENABLE_TELEMETRY=false
+ENABLE_AUTH=false
+# Model Context Protocol (MCP)
+MCP_ENABLE=false
+MCP_EXPOSE=false
+MCP_SERVERS=
+MCP_CLIENT_TIMEOUT=5s
+MCP_DIAL_TIMEOUT=3s
+MCP_TLS_HANDSHAKE_TIMEOUT=3s
+MCP_RESPONSE_HEADER_TIMEOUT=3s
+MCP_EXPECT_CONTINUE_TIMEOUT=1s
+MCP_REQUEST_TIMEOUT=5s
+# OpenID Connect
+OIDC_ISSUER_URL=http://keycloak:8080/realms/inference-gateway-realm
+OIDC_CLIENT_ID=inference-gateway-client
+OIDC_CLIENT_SECRET=
+# Server settings
+SERVER_HOST=0.0.0.0
+SERVER_PORT=8080
+SERVER_READ_TIMEOUT=30s
+SERVER_WRITE_TIMEOUT=30s
+SERVER_IDLE_TIMEOUT=120s
+SERVER_TLS_CERT_PATH=
+SERVER_TLS_KEY_PATH=
+# Client settings
+CLIENT_TIMEOUT=30s
+CLIENT_MAX_IDLE_CONNS=20
+CLIENT_MAX_IDLE_CONNS_PER_HOST=20
+CLIENT_IDLE_CONN_TIMEOUT=30s
+CLIENT_TLS_MIN_VERSION=TLS12
+# Providers
+ANTHROPIC_API_URL=https://api.anthropic.com/v1
+ANTHROPIC_API_KEY=
+CLOUDFLARE_API_URL=https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai
+CLOUDFLARE_API_KEY=
+COHERE_API_URL=https://api.cohere.ai
+COHERE_API_KEY=
+GROQ_API_URL=https://api.groq.com/openai/v1
+GROQ_API_KEY=
+OLLAMA_API_URL=http://ollama:8080/v1
+OLLAMA_API_KEY=
+OPENAI_API_URL=https://api.openai.com/v1
+OPENAI_API_KEY=
+DEEPSEEK_API_URL=https://api.deepseek.com
+DEEPSEEK_API_KEY=

examples/QUICKSTART.md

@@ -0,0 +1,122 @@
+# Quick Start Guide
+
+This guide will help you run all TypeScript SDK examples quickly.
+
+## Prerequisites
+
+1. **Docker** - For running the Inference Gateway
+2. **Node.js** - For running the examples
+3. **API Key** - For at least one AI provider
+
+## 1. Setup Environment
+
+1. Copy the environment template:
+
+   ```bash
+   cp .env.example .env
+   ```
+
+2. Add your API keys to `.env`:
+   ```bash
+   # Choose one or more providers
+   GROQ_API_KEY=your_groq_key_here
+   OPENAI_API_KEY=your_openai_key_here
+   ANTHROPIC_API_KEY=your_anthropic_key_here
+   ```
+
+## 2. Start Inference Gateway
+
+Choose one of these options:
+
+### Option A: Basic Gateway (for List and Chat examples)
+
+```bash
+docker run --rm -p 8080:8080 --env-file .env ghcr.io/inference-gateway/inference-gateway:latest
+```
+
+### Option B: Gateway with MCP (for all examples)
+
+```bash
+cd mcp
+npm run compose:up
+```
+
+## 3. Test the Examples
+
+### Quick Test - List Models
+
+```bash
+cd list
+npm install
+npm start
+```
+
+### Chat Example
+
+```bash
+cd chat
+export PROVIDER=groq
+export LLM=meta-llama/llama-3.3-70b-versatile
+npm install
+npm start
+```
+
+### MCP Example (requires Docker Compose setup)
+
+```bash
+cd mcp
+export PROVIDER=groq
+export LLM=meta-llama/llama-3.3-70b-versatile
+npm install
+npm start
+```
+
+## 4. Popular Provider/Model Combinations
+
+### Groq (Fast inference)
+
+```bash
+export PROVIDER=groq
+export LLM=meta-llama/llama-3.3-70b-versatile
+```
+
+### OpenAI (High quality)
+
+```bash
+export PROVIDER=openai
+export LLM=gpt-4o
+```
+
+### Anthropic (Strong reasoning)
+
+```bash
+export PROVIDER=anthropic
+export LLM=claude-3-5-sonnet-20241022
+```
+
+## Troubleshooting
+
+### Gateway not responding
+
+- Check if Docker container is running: `docker ps`
+- Test health: `curl http://localhost:8080/health`
+- Check logs: `docker logs <container_id>`
+
+### Authentication errors
+
+- Verify API key is correct in `.env`
+- Ensure the key has sufficient permissions
+- Try a different provider
+
+### Model not found
+
+- Use the list example to see available models
+- Check if the model name is correct
+- Try a different model from the same provider
+
+## Next Steps
+
+1. Explore each example in detail
+2. Modify the examples for your use case
+3. Build your own applications using the patterns shown
+4. Check the main [README](../README.md) for more advanced usage