Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions src/pages/docs/platform/integrations/inbound/webhooks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -95,3 +95,13 @@ channel.subscribe('webhook-message', (message) => {
</Code>

To ensure that publishes are [idempotent](/docs/pub-sub/advanced#idempotency), add a unique `X-Ably-MessageId` header.

## Publishing from Zapier <a id="zapier"/>

Zapier can publish messages to Ably channels when other apps trigger events. This connects spreadsheets, forms, emails, and other services to realtime channels.

In Zapier, create a new Zap and choose a trigger (like a new spreadsheet row). Add Ably as an action with "Publish Message" and "Publish Push Notification" options. Provide the API key from the Ably dashboard.

Set up the message with the channel name and content. For push notifications, add a title and body for the notification itself.

Use Zapier's test button to verify functionality, then watch for messages in the Ably Dev Console.
18 changes: 18 additions & 0 deletions src/pages/docs/platform/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,24 @@ The following pre-built services can be configured:

Whilst pub-sub channels broadcast messages to all subscribers, queues distribute work among consumers. Both patterns serve different use cases. For example, pub/sub is ideal for many users to receive realtime updates, while queues handle tasks like triggering emails efficiently.

## Debugging integrations <a id="debugging"/>

If messages are not reaching the endpoint, follow these steps:

1. Check the integration setup in the Ably dashboard. Verify the URL is correct and the channel filter matches the channels being tested.

2. Use the Test rule button to verify Ably can reach the endpoint. This tests the connection directly.

3. If the connection test succeeds, publish a test message through the Dev Console to the channel the integration monitors. Check the Dev Console logs to see if the message was delivered. Errors can take up to 15 seconds to appear due to retry attempts.

Common issues include regex patterns that do not match channels, authentication problems, and endpoints that do not respond quickly enough.

### Integration failure logs

When integrations fail (endpoint returns an error), Ably logs the details to the `[meta]log` metachannel. These logs include the failure reason (such as "POST returned HTTP status 500"), along with the rule ID, message ID, and channel name.

View these failures live in the Dev Console, or use the history API to check past failures. Attach to the `[meta]log` channel with rewind to see the last 100 messages.

## Skip integrations <a id="skip"/>

Privileged users can [skip integrations](/docs/platform/integrations/skip-integrations) on a per-message basis, providing greater control and flexibility when publishing messages to a channel.
46 changes: 46 additions & 0 deletions src/pages/docs/platform/integrations/queues.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,22 @@ The following steps explain how to provision an Ably Queue:
A [Dead Letter Queue](#deadletter) is automatically created. It stores messages that fail to be processed, or expire.
</Aside>

### Modifying queues <a id="modify"/>

Queues cannot be modified after creation - they are immutable. Different settings require creating a new queue and migrating.

This includes limits like TTL and max length. Even after upgrading an Ably account, existing queues retain the limits they were created with. Replacement is required to get the higher limits from a new plan.

Steps to switch to a new queue:

1. Create a new queue with the desired settings
2. Update consumers to subscribe to both the old and new queues
3. Change queue rules to route messages to the new queue
4. Wait for the old queue to drain completely
5. Delete the old queue once empty

This process ensures no message loss during the transition.

### Configure a Queue rule <a id="config"/>

After you provision a Queue, create one or more Queue rules to republish messages, presence events, or channel events from channels into that queue.
Expand Down Expand Up @@ -484,6 +500,36 @@ Ably deletes any message in the Dead Letter Queue if it later meets one of these

Ably names the Dead Letter Queue using the reserved format `APPID:deadletter`, where `APPID` is your app's ID. Each app with one or more queues has exactly one Dead Letter Queue, and you'll see it listed in the Queues dashboard. You can subscribe to it like any other queue.

## Troubleshooting queue connections <a id="troubleshooting"/>

Common queue connection issues:

**Port Configuration**: Use port 5671, not the standard AMQP port 5672. Only TLS connections are supported.

**API Key Permissions**: The API key needs queue subscribe permissions. Check this in the dashboard under App Keys for authentication errors.

**AMQP Version**: AMQP 0.9.1 is required. Older clients might not support this version.

**Queue Declaration**: Do not declare queues or bind exchanges. Ably queues already exist - connect to the queue name directly.

**Authentication Format**: The username is everything before the colon in the API key, password is everything after.

**Virtual Host**: Use the "shared" vhost. Some clients may have difficulty with this configuration.

**Connection Timeout**: Increase client timeout settings if connections are timing out.

**Message Delivery**: Verify queue rules and channel filters are configured correctly if no messages appear.

## Production considerations <a id="production"/>

Ably Queues work well for testing and lower-volume applications, but have limitations for production use at scale.

The multi-tenanted queue service has periodic maintenance windows that can cause brief downtime. This differs from Ably's core platform which has stronger uptime guarantees.

For production systems requiring high throughput or that cannot handle maintenance downtime, consider using [streaming integrations](/docs/platform/integrations/streaming) instead. These allow data to be piped into dedicated queue services like SQS, RabbitMQ, or Kinesis under direct control.

For enterprise customers with high-volume needs, dedicated queue clusters are available that scale to millions of messages per second.

## Queue scalability <a id="scalability"/>

Ably Queues are offered in two flavors; multi-tenanted and dedicated.
Expand Down
11 changes: 11 additions & 0 deletions src/pages/docs/platform/integrations/streaming/datadog.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,17 @@ The following steps setup the Datadog integration:
6. Datadog authorization page, authorize Ably using **OAuth** to grant access. The required authorization scope is: `api_keys_write`.
7. After completing authorization, you will be redirected to the **Ably dashboard**, and the process is complete.

## Troubleshooting integration states <a id="troubleshooting"/>

Common connection issues:

* Stuck on "Connecting": Connection was interrupted. Contact support if the status does not change.
* Getting "Connection failed": Contact support for assistance.
* "API key with this name already exists": The key already exists in Datadog. Delete the existing key first.
* "Forbidden" or permission errors: Datadog Admin permissions are required to set up this integration.

After successful setup, select "See Dashboard" in Ably to navigate to Datadog. Data appears every minute.

## Remove access <a id="remove"/>

Removing access disconnects Ably from Datadog, stopping data transmission and revoking authorization. Follow the steps remove the Ably and Datadog integration using either platform.
Expand Down
19 changes: 18 additions & 1 deletion src/pages/docs/platform/integrations/webhooks/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,15 @@ public:news:europe
| `^public.*europe$` | Matches channels that start with `public` and end with `europe`. This includes only `public:news:europe`. |
| `news` | Matches any channel name that includes the word `news`. This includes `public:news:americas` and `public:news:europe`.

### Common regex formatting issues

When creating channel filter regex patterns, ensure you use proper regular expression syntax:

- Correct: `public.*` - Matches "public" followed by any characters.
- Incorrect: `public*` - Matches "publi" followed by zero or more "c" characters.

The `.*` pattern means "any character (.) zero or more times (*)", while `*` alone means "zero or more of the preceding character".

## Event types <a id="sources"/>

You can configure webhooks to listen for the following event types:
Expand Down Expand Up @@ -92,7 +101,15 @@ The backoff delay follows the formula: `delay = delay * sqrt(2)` where the initi

The back off for consecutively failing requests will increase until it reaches 60s. All subsequent retries for failed requests will then be made at 60s intervals until a request is successful. The queue of events is retain for 5 minutes. If an event cannot be delivered within that time then events are discarded to prevent the queue from growing indefinitely.

### Batched event payloads <a id="batched-events"/>
## Message ordering <a id="ordering"/>

Webhooks do not always preserve message order the same way Ably channels do. This depends on the webhook configuration.

Batched webhooks preserve message order when messages are from the same publisher on the same channel. If a batch fails and gets retried, newer messages are included while maintaining correct order. Messages from different regions might arrive in separate batches, maintaining per-publisher ordering.

Single request webhooks cannot guarantee order. Each message triggers its own HTTP request, and arrival order is not predictable. HTTP/2 server support can restore ordering through request pipelining over a single connection.

Publishing via REST (not realtime) removes ordering guarantees even with batching enabled, as REST uses a connection pool where requests can complete out of order.

Given the various potential combinations of enveloped, batched, and message sources, it's helpful to understand what to expect in different scenarios.

Expand Down
47 changes: 47 additions & 0 deletions src/pages/docs/platform/integrations/webhooks/lambda.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -105,3 +105,50 @@ Then ensure the checkbox for the policy is selected.
8. You don't need to add tags so click **Next: Review**.
9. Enter a suitable name for your role.
10. Click **Create Role**.

## Lambda retry behavior <a id="retries"/>

Ably invokes Lambda functions asynchronously using the `event` invocation type. When a function returns an error, AWS Lambda automatically retries the execution up to two more times with delays between attempts (1 minute, then 2 minutes).

Lambda functions might run multiple times for the same Ably event. Design functions to handle this by making them idempotent or checking for duplicate processing.

Configure retry behavior in the AWS Lambda console under the function's asynchronous invocation settings. See the [AWS Lambda documentation](https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html#invocation-async-errors) for details on adjusting retry settings.

## Example Lambda function <a id="example"/>

The following Lambda function processes events from Ably and publishes a message back. Bundle this with the Ably client library when uploading.

<Code>
```javascript
'use strict';

const Ably = require('ably');
const inspect = require('util').inspect;

exports.handler = (event, context, callback) => {
console.log("Received event from Ably:", inspect(event));

// If you're using enveloped messages, the event has 'messages', 'appId', etc.
// Otherwise it's just the message data directly
const details = JSON.parse(event.messages[0].data);

// Use the REST client - don't bother with websockets for one-off publishes
const ably = new Ably.Rest({ key: 'YOUR_API_KEY' });

// Publish to a different channel or you'll trigger this Lambda again!
const channel = ably.channels.get('lambda-responses');
channel.publish('response', 'success', (err) => {
if(err) {
console.log("Error publishing back to Ably:", inspect(err));
callback(err);
} else {
callback(null, 'success');
}
});
};
```
</Code>

Important considerations: Use `Ably.Rest` instead of `Ably.Realtime` to avoid opening websockets for publishing. Publish to a different channel than the one triggering this Lambda to prevent loops. Always call the callback inside the publish callback to prevent the function from finishing before the publish request completes.

A complete working example is available at [github.com/ably/example-lambda-function](https://github.com/ably/example-lambda-function) for download and deployment.
6 changes: 6 additions & 0 deletions src/pages/docs/platform/pricing/limits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,12 @@ Connection limits relate to the realtime [connections](/docs/connect) made to Ab

Integration limits relate to the rate and concurrency of [webhooks](/docs/platform/integrations/webhooks), [queues](/docs/platform/integrations/queues) and [Firehose](/docs/platform/integrations/streaming).

### Why integration rate limits exist

Rate limits prevent issues when there is a mismatch between channel message rate and endpoint processing capacity. Without limits, messages could queue up invisibly, creating backlogs that cannot be monitored or managed.

For high-volume scenarios, use an intermediary queue like [AWS Kinesis](/docs/platform/integrations/streaming/kinesis) instead of direct webhook delivery. This provides better visibility into message backlogs and allows batch processing of messages or discarding of stale ones.

| Integration limit | Free | Standard | Pro | Enterprise |
| ----------------- | ---- | -------- | --- | ---------- |
| **Integration rate (per second)**<p>*the maximum rate at which integrations can be executed, or streamed*</p> | 250 | 250 | 1,000 | Custom |
Expand Down