diff --git a/docs/usage/python/streaming-response.mdx b/docs/usage/python/streaming-response.mdx index 400e2dd497..0c4ad76c73 100644 --- a/docs/usage/python/streaming-response.mdx +++ b/docs/usage/python/streaming-response.mdx @@ -8,70 +8,95 @@ You can stream messages, code, and code outputs out of Open Interpreter by setti for chunk in interpreter.chat("What's 34/24?", stream=True, display=False): print(chunk) ``` + ``` -{'language': 'python'} -{'code': '34'} -{'code': ' /'} -{'code': ' '} -{'code': '24'} -{'executing': {'code': '34 / 24', 'language': 'python'}} -{'active_line': 1} -{'output': '1.4166666666666667\n'} -{'active_line': None} -{'end_of_execution': True} -{'message': 'The'} -{'message': ' result'} -{'message': ' of'} -{'message': ' the'} -{'message': ' division'} -{'message': ' '} -{'message': '34'} -{'message': '/'} -{'message': '24'} -{'message': ' is'} -{'message': ' approximately'} -{'message': ' '} -{'message': '1'} -{'message': '.'} -{'message': '42'} -{'message': '.'} +{"role": "assistant", "type": "code", "format": "python", "start": True} +{"role": "assistant", "type": "code", "format": "python", "content": "34"} +{"role": "assistant", "type": "code", "format": "python", "content": " /"} +{"role": "assistant", "type": "code", "format": "python", "content": " "} +{"role": "assistant", "type": "code", "format": "python", "content": "24"} +{"role": "assistant", "type": "code", "format": "python", "end": True} + +{"role": "computer", "type": "confirmation", "format": "execution", "content": {"type": "code", "language": "python", "code": "34 / 24"}} + +{"role": "computer", "type": "console", "start": True} +{"role": "computer", "type": "console", "format": "active_line", "content": "1"} +{"role": "computer", "type": "console", "format": "output", "content": "1.4166666666666667\n"} +{"role": "computer", "type": "console", "end": True} + +{"role": "assistant", "type": "message", "start": True} +{"role": "assistant", "type": "message", "content": "The"} +{"role": "assistant", "type": "message", "content": " result"} +{"role": "assistant", "type": "message", "content": " of"} +{"role": "assistant", "type": "message", "content": " the"} +{"role": "assistant", "type": "message", "content": " division"} +{"role": "assistant", "type": "message", "content": " "} +{"role": "assistant", "type": "message", "content": "34"} +{"role": "assistant", "type": "message", "content": "/"} +{"role": "assistant", "type": "message", "content": "24"} +{"role": "assistant", "type": "message", "content": " is"} +{"role": "assistant", "type": "message", "content": " approximately"} +{"role": "assistant", "type": "message", "content": " "} +{"role": "assistant", "type": "message", "content": "1"} +{"role": "assistant", "type": "message", "content": "."} +{"role": "assistant", "type": "message", "content": "42"} +{"role": "assistant", "type": "message", "content": "."} +{"role": "assistant", "type": "message", "end": True} ``` **Note:** Setting `display=True` won't change the behavior of the streaming response, it will just render a display in your terminal. # Anatomy +Each chunk of the streamed response is a dictionary, that has a "role" key that can be either "assistant" or "computer". The "type" key describes what the chunk is. The "content" key contains the actual content of the chunk. + +Every 'message' is made up of chunks, and begins with a "start" chunk, and ends with an "end" chunk. This helps you parse the streamed response into messages. + Let's break down each part of the streamed response. ## Code In this example, the LLM decided to start writing code first. It could have decided to write a message first, or to only write code, or to only write a message. -Before writing any code, the LLM will always set the `language` for the code it's about to write. In this case it decided to write `python`. +Every streamed chunk of type "code" has a format key that specifies the language. In this case it decided to write `python`. This can be any language defined in [our languages directory.](https://github.com/KillianLucas/open-interpreter/tree/main/interpreter/computer/languages) ``` -{'language': 'python'} + +{"role": "assistant", "type": "code", "format": "python", "start": True} + ``` Then, the LLM decided to write some code. The code is sent token-by-token: ``` -{'code': '34'} -{'code': ' /'} -{'code': ' '} -{'code': '24'} + +{"role": "assistant", "type": "code", "format": "python", "content": "34"} +{"role": "assistant", "type": "code", "format": "python", "content": " /"} +{"role": "assistant", "type": "code", "format": "python", "content": " "} +{"role": "assistant", "type": "code", "format": "python", "content": "24"} + +``` + +When the LLM finishes writing code, it will send an "end" chunk: + +``` + +{"role": "assistant", "type": "code", "format": "python", "end": True} + ``` ## Code Output After the LLM finishes writing a code block, Open Interpreter will attempt to run it. -**Before** it runs it, the following message is sent: +**Before** it runs it, the following chunk is sent: ``` -{'executing': {'code': '34 / 24', 'language': 'python'}} + +{"role": "computer", "type": "confirmation", "format": "execution", "content": {"type": "code", "language": "python", "code": "34 / 24"}} + ``` If you check for this object, you can break (or get confirmation) **before** executing the code. @@ -88,7 +113,7 @@ for chunk in interpreter.chat("What's 34/24?", stream=True): **While** the code is being executed, you'll recieve the line of code that's being run: ``` -{'active_line': 1} +{"role": "computer", "type": "console", "format": "active_line", "content": "1"} ``` We use this to highlight the active line of code on our UI, which keeps the user aware of what Open Interpreter is doing. @@ -96,13 +121,13 @@ We use this to highlight the active line of code on our UI, which keeps the user You'll then recieve its output, if it produces any: ``` -{'output': '1.4166666666666667\n'} +{"role": "computer", "type": "console", "format": "output", "content": "1.4166666666666667\n"} ``` When the code is **finished** executing, this flag will be sent: ``` -{'end_of_execution': True} +{"role": "computer", "type": "console", "end": True} ``` ## Message @@ -110,20 +135,24 @@ When the code is **finished** executing, this flag will be sent: Finally, the LLM decided to write a message. This is streamed token-by-token as well: ``` -{'message': 'The'} -{'message': ' result'} -{'message': ' of'} -{'message': ' the'} -{'message': ' division'} -{'message': ' '} -{'message': '34'} -{'message': '/'} -{'message': '24'} -{'message': ' is'} -{'message': ' approximately'} -{'message': ' '} -{'message': '1'} -{'message': '.'} -{'message': '42'} -{'message': '.'} -``` \ No newline at end of file +{"role": "assistant", "type": "message", "start": True} +{"role": "assistant", "type": "message", "content": "The"} +{"role": "assistant", "type": "message", "content": " result"} +{"role": "assistant", "type": "message", "content": " of"} +{"role": "assistant", "type": "message", "content": " the"} +{"role": "assistant", "type": "message", "content": " division"} +{"role": "assistant", "type": "message", "content": " "} +{"role": "assistant", "type": "message", "content": "34"} +{"role": "assistant", "type": "message", "content": "/"} +{"role": "assistant", "type": "message", "content": "24"} +{"role": "assistant", "type": "message", "content": " is"} +{"role": "assistant", "type": "message", "content": " approximately"} +{"role": "assistant", "type": "message", "content": " "} +{"role": "assistant", "type": "message", "content": "1"} +{"role": "assistant", "type": "message", "content": "."} +{"role": "assistant", "type": "message", "content": "42"} +{"role": "assistant", "type": "message", "content": "."} +{"role": "assistant", "type": "message", "end": True} +``` + +For an example in JavaScript on how you might process these streamed chunks, see the [migration guide](https://github.com/KillianLucas/open-interpreter/blob/main/docs/NCU_MIGRATION_GUIDE.md) diff --git a/docs/usage/terminal/arguments.mdx b/docs/usage/terminal/arguments.mdx index d1517156fb..b2cb304da5 100644 --- a/docs/usage/terminal/arguments.mdx +++ b/docs/usage/terminal/arguments.mdx @@ -2,122 +2,440 @@ title: Arguments --- +**[Modes](/docs/usage/terminal/arguments#modes)** + +`--vision`, `--os`. + +**[Model Settings](/docs/usage/terminal/arguments#model-settings)** + +`--model`, `--fast`, `--local`, `--temperature`, `--context_window`, `--max_tokens`, `--max_output`, `--api_base`, `--api_key`, `--api_version`, `--llm_supports_functions`, `--llm_supports_vision`. + +**[Configuration](/docs/usage/terminal/arguments#Configuration)** + +`--config`, `--config_file`, `--custom_instructions`, `--system_message`. + +**[Options](/docs/usage/terminal/arguments#options)** + +`--safe_mode`, `--auto_run`, `--force_task_completion`, `--verbose`, `--max_budget`, `--speak_messages`, `--version`. + +**[Other](/docs/usage/terminal/arguments#other)** + +`--conversations`, `--help`. + --- -#### `--model` or `-m` +## Modes -Specifies which language model to use. +#### `--vision` or `-vi` -```bash -interpreter --model "gpt-3.5-turbo" +Enables vision mode for multimodal models. Defaults to GPT-4-turbo. + + +```bash Terminal +interpreter --vision +``` + +```yaml Config +vision: true ``` + + +#### `--os` or `-o` + +Enables OS mode for multimodal models. Defaults to GPT-4-turbo. + + + + ```bash Terminal + interpreter --os + ``` + + ```yaml Config + os: true + ``` + + + --- -#### `--local` or `-l` +## Model Settings -Run the model locally. +#### `--model` or `-m` -```bash -interpreter --local +Specifies which language model to use. Check out the [models](https://docs.openinterpreter.com/language-model-setup/introduction) section for a list of available models. + + + +```bash Terminal +interpreter --model "gpt-3.5-turbo" ``` ---- +```yaml Config +model: gpt-3.5-turbo +``` -#### `--auto_run` or `-y` + -Automatically run the interpreter without requiring user confirmation. +#### `--fast` or `-f` -```bash -interpreter --auto_run +Sets the model to gpt-3.5-turbo. + + +```bash Terminal +interpreter --fast ``` ---- +```yaml Config +fast: true +``` -#### `--verbose` or `-d` + -Run the interpreter in verbose mode. Debug information will be printed at each step to help diagnose issues. +#### `--local` or `-l` -```bash -interpreter --verbose +Run the model locally. Check the [local mode page](/language-model-setup/local-models/overview) for more information. + + + +```bash Terminal +interpreter --local ``` ---- +```yaml Config +local: true +``` + + #### `--temperature` or `-t` Sets the randomness level of the model's output. -```bash + + +```bash Terminal interpreter --temperature 0.7 ``` ---- +```yaml Config +temperature: 0.7 +``` + + #### `--context_window` or `-c` Manually set the context window size in tokens for the model. -```bash + + +```bash Terminal interpreter --context_window 16000 ``` ---- +```yaml Config +context_window: 16000 +``` + + #### `--max_tokens` or `-x` Sets the maximum number of tokens that the model can generate in a single response. -```bash + + +```bash Terminal interpreter --max_tokens 100 ``` ---- +```yaml Config +max_tokens: 100 +``` -#### `--max_budget` or `-b` + -Sets the maximum budget limit for the session in USD. +#### `--max_output` or `-xo` -```bash -interpreter --max_budget 0.01 +Set the maximum number of characters for code outputs. + + +```bash Terminal +interpreter --max_output 1000 ``` ---- +```yaml Config +max_output: 1000 +``` + #### `--api_base` or `-ab` If you are using a custom API, specify its base URL with this argument. -```bash + + +```bash Terminal interpreter --api_base "https://api.example.com" ``` ---- +```yaml Config +api_base: https://api.example.com +``` + + #### `--api_key` or `-ak` Set your API key for authentication when making API calls. -```bash + + +```bash Terminal interpreter --api_key "your_api_key_here" ``` +```yaml Config +api_key: your_api_key_here +``` + + + +#### `--api_version` or `-av` + +Optionally set the API version to use with your selected model. (This will override environment variables) + + +```bash Terminal +interpreter --api_version 2.0.2 +``` + +```yaml Config +api_version: 2.0.2 +``` + + +#### `--llm_supports_functions` or `-lsf` + +Inform Open Interpreter that the language model you're using supports function calling. + + +```bash Terminal +interpreter --llm_supports_functions +``` + +```yaml Config +llm_supports_functions: true +``` + + +#### `--llm_supports_vision` or `-lsv` + +Inform Open Interpreter that the language model you're using supports vision. + + +```bash Terminal +interpreter --llm_supports_vision +``` + +```yaml Config +llm_supports_vision: true +``` + + + --- -#### `--safe_mode` or `-safe` +## Configuration + +#### `--config` + +Opens the configuration yaml file in your default editor. + + +```bash Terminal +interpreter --config +``` + + + +#### `--config_file` or `-cf` + +Optionally set a custom config file to use. + + +```bash Terminal +interpreter --config_file "/config.yaml" +``` + + + +#### `--custom_instructions` or `-ci` + +Appends custom instructions to the system message. This is useful for adding information about the your system, preferred languages, etc. + + +```bash Terminal +interpreter --custom_instructions "This is a custom instruction." +``` + +```yaml Config +custom_instructions: "This is a custom instruction." +``` + + + +#### `--system_message` or `-s` + +We don't recommend modifying the system message, as doing so opts you out of future updates to the system message. Use `--custom_instructions` instead, to add relevant information to the system message. If you must modify the system message, you can do so by using this argument, or by opening the config file using `--config`. + + +```bash Terminal +interpreter --system_message "You are Open Interpreter..." +``` + +```yaml Config +system_message: "You are Open Interpreter..." +``` + + + +#### `--reset_config` + +Resets the config file to the default settings. + + +```bash Terminal +interpreter --reset_config +``` + + + +## Options + +#### `--safe_mode` Enable or disable experimental safety mechanisms like code scanning. Valid options are `off`, `ask`, and `auto`. -```bash + + +```bash Terminal interpreter --safe_mode ask ``` +```yaml Config +safe_mode: ask +``` + + + +#### `--auto_run` or `-y` + +Automatically run the interpreter without requiring user confirmation. + + + +```bash Terminal +interpreter --auto_run +``` + +```yaml Config +auto_run: true +``` + + + +#### `--force_task_completion` or `-fc` + +Runs Open Interpreter in a loop, requiring it to admit to completing or failing every task. + + +```bash Terminal +interpreter --force_task_completion +``` + +```yaml Config +force_task_completion: true +``` + + + +#### `--verbose` or `-v` + +Run the interpreter in verbose mode. Debug information will be printed at each step to help diagnose issues. + + + +```bash Terminal +interpreter --verbose +``` + +```yaml Config +verbose: true +``` + + + +#### `--max_budget` or `-b` + +Sets the maximum budget limit for the session in USD. + + + +```bash Terminal +interpreter --max_budget 0.01 +``` + +```yaml Config +max_budget: 0.01 +``` + + + +#### `--speak_messages` or `-sm` + +(Mac Only) Speak messages out loud using the system's text-to-speech engine. + + +```bash Terminal +interpreter --speak_messages +``` + +```yaml Config +speak_messages: true +``` + + + --- -#### `--config_file` +## Other -This new option allows users to specify a path or filename for a config file in their Open Interpreter config directory. It enables them to use a custom config file when invoking the interpreter. +#### `--version` + +Get the current installed version number of Open Interpreter. + +```bash Terminal interpreter --version ``` + +#### `--conversations` + +Display a list of previous conversations. + + +```bash Terminal +interpreter --conversations +``` + + + +#### `--help` or `-h` + +Display all available terminal arguments. + + +```bash Terminal +interpreter --help +``` -```bash -interpreter --config_file config.yaml -``` \ No newline at end of file +