Skip to content

wip: setup playwright #650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 26 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
32dbdf1
wip: setup playwright
MarcMcIntosh Mar 31, 2025
bcee869
wip: running playwright mcp.
MarcMcIntosh Apr 1, 2025
73b2222
chore(playwright) add a readme file
MarcMcIntosh Apr 1, 2025
c6130ad
update login to use google.
MarcMcIntosh Apr 2, 2025
835e8bf
take videos for tests
MarcMcIntosh Apr 2, 2025
d06f815
test(playwright): user logins in :)
MarcMcIntosh Apr 4, 2025
969a308
chore: update readme.md
MarcMcIntosh Apr 4, 2025
03337a9
wip: setting up initial tests to get to the app.
MarcMcIntosh Apr 4, 2025
0ec3eca
remove form name.
MarcMcIntosh Apr 7, 2025
7b194e5
wip(playwright): tour test
MarcMcIntosh Apr 16, 2025
0636118
test(playwright): add images.
MarcMcIntosh Apr 16, 2025
9e48a36
test(playwright): add snapshots.
MarcMcIntosh Apr 16, 2025
a047ac4
playwright: refactor tour spec in to a fixture.
MarcMcIntosh Apr 17, 2025
30649ec
playwright: uset storage for login and tour
MarcMcIntosh Apr 17, 2025
afe192c
add todo note.
MarcMcIntosh Apr 17, 2025
01e8b28
playwright: add user survery test
MarcMcIntosh Apr 18, 2025
ce4f9dc
playwright: update tour snapshots
MarcMcIntosh Apr 18, 2025
9748c1f
playwright: test statistics page
MarcMcIntosh Apr 18, 2025
883c8dd
playwright: fix tour snapshots.
MarcMcIntosh Apr 18, 2025
0db4efe
wip add fake ide for simulating events
MarcMcIntosh Apr 22, 2025
09cd663
wip(playwright): add methods for events from the ide.
MarcMcIntosh Apr 23, 2025
21d1ddd
wip(playwright): adding fakeIde to fixtures.
MarcMcIntosh Apr 23, 2025
aca91a6
wip: setting up the fake ide.
MarcMcIntosh Apr 23, 2025
c765edc
wip: use fakeide to render app.
MarcMcIntosh Apr 24, 2025
8eee2fa
wip: try to use original post message when sending events to the chat
MarcMcIntosh Apr 25, 2025
85a1d95
test(playwright): use fake ide in tests.
MarcMcIntosh Apr 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/playwright.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Playwright Tests
on:
workflow_dispatch:

defaults:
run:
working-directory: playwright
# push:
# branches: [main, master]
# pull_request:
# branches: [main, master]
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: lts/*
- name: Install dependencies
run: npm ci
- name: Install Playwright Browsers
run: npx playwright install --with-deps chromium
- name: Run Playwright tests
run: npx playwright test
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: playwright-report
path: playwright-report/
retention-days: 30
8 changes: 8 additions & 0 deletions .refact/integrations.d/mcp_playwright.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
command: npx -y @playwright/mcp@latest --headless
env: {}
available:
on_your_laptop: true
when_isolated: true
confirmation:
ask_user: []
deny: []
8 changes: 8 additions & 0 deletions playwright/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@

# Playwright
node_modules/
/test-results/
/playwright-report/
/blob-report/
/playwright/.cache/
.auth/
171 changes: 171 additions & 0 deletions playwright/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# Playwright Testing for Refact

This directory contains end-to-end tests for the Refact project using Playwright. Playwright provides reliable end-to-end testing for modern web applications.

## Prerequisites

- Node.js (v14 or newer)
- npm
- A Refact API key (for certain tests)

## Setup

1. Install dependencies:

```bash
npm install
```

This will automatically install Playwright with Chromium dependencies via the postinstall script.

2. Create a `.env` file in the playwright directory with the following content:

```
REFACT_API_KEY=your_api_key_here
```

## Running Tests

### Development Mode (with UI)

To run tests with the Playwright UI, which provides a visual interface for debugging tests:

```bash
npm start
```

This launches the Playwright UI, allowing you to:

- See test execution in real-time
- Inspect DOM elements
- Debug tests step by step
- View test traces
- Re-run specific tests

#### Tip

Press the `toogle output` button or press `` ^` `` to see the out put from the lsp web servers stdout and stderr.

### Headless Mode

To run tests in headless mode (without browser UI), which is faster and suitable for CI/CD:

First add your api key to `refact/playwright/.env`

```
REFACT_API_KEY=your_api_key_here
```

Then run

```bash
npm run test
```

For more specific test runs, you can use the underlying Playwright command with options:

```bash
# Run a specific test file
npx playwright test tests/example.spec.ts

# Run tests with a specific tag
npx playwright test --grep "@smoke"

# Run tests in a specific project
npx playwright test --project=chromium

# Run tests with a specific reporter
npx playwright test --reporter=line
```

## Test Configuration

The Playwright configuration is defined in `playwright.config.ts` and includes:

- Test directory: `./tests`
- Browser configuration: Currently set up for Chromium
- Parallel execution settings
- Web server setup for the Refact GUI and backend

## Creating Tests

Tests are written in TypeScript using the Playwright test framework. Example:

```typescript
import { test, expect } from "@playwright/test";

test("example test", async ({ page }) => {
await page.goto("http://localhost:5173");
await expect(page.getByText("Welcome to Refact")).toBeVisible();
});
```

### Best Practices

1. Place test files in the `tests` directory
2. Use descriptive test names
3. Group related tests in the same file
4. Use page objects for complex pages
5. Leverage Playwright's built-in assertions

## Viewing Test Reports

After test execution, HTML reports are generated and can be viewed by:

```bash
npx playwright show-report
```

This will open the HTML report in your default browser, showing:

- Test results summary
- Test execution time
- Error details for failed tests
- Screenshots and videos (if configured)
- Traces for debugging

## Debugging Tips

1. **Use UI Mode for Interactive Debugging**:

```bash
npm run start
```

2. **Generate and Analyze Traces**:

```bash
npx playwright test --trace on
```

3. **Use Debug Mode**:

```bash
npx playwright test --debug
```

4. **Record Video of Test Execution**:
Add to config:
```typescript
use: {
video: 'on-first-retry',
}
```

## CI/CD Integration

For continuous integration, set the environment variable `CI=true` to enable:

- Retry failed tests
- Reduced parallelism
- Fail if `.only` is present in tests

Example:

```bash
CI=true npm run test
```

## Advanced Configuration

See the [Playwright documentation](https://playwright.dev/docs/test-configuration) for more advanced configuration options.
30 changes: 30 additions & 0 deletions playwright/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import { defineConfig } from "eslint/config";
// import globals from "globals";
// import js from "@eslint/js";
// import tseslint from "typescript-eslint";
import playwright from "eslint-plugin-playwright";

export default defineConfig([
// start of: eslint init defaults
// { files: ["**/*.{js,mjs,cjs,ts}"] },
// {
// files: ["**/*.{js,mjs,cjs,ts}"],
// languageOptions: { globals: { ...globals.browser, ...globals.node } },
// },
// {
// files: ["**/*.{js,mjs,cjs,ts}"],
// plugins: { js },
// extends: ["js/recommended"],
// },
// tseslint.configs.recommended,

{
...playwright.configs["flat/recommended"],
files: ["tests/**"],
rules: {
...playwright.configs["flat/recommended"].rules,
// Customize Playwright rules
// ...
},
},
]);
Loading