Skip to content

Commit df68ade

Browse files
MichaelGHSegMichaelGHSeg
authored
Merge pull request #279 from segmentio/add-e2e-cli
Add e2e-cli for E2E testing
2 parents 167f59f + f82db36 commit df68ade

8 files changed

Lines changed: 473 additions & 1 deletion

File tree

‎.github/workflows/e2e-tests.yml‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: E2E Tests
2+
3+
on:
4+
push:
5+
branches: [master]
6+
pull_request:
7+
branches: [master]
8+
workflow_dispatch:
9+
inputs:
10+
e2e_tests_ref:
11+
description: 'Branch or ref of sdk-e2e-tests to use'
12+
required: false
13+
default: 'main'
14+
15+
jobs:
16+
e2e-tests:
17+
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository }}
18+
runs-on: ubuntu-latest
19+
20+
steps:
21+
- name: Checkout SDK
22+
uses: actions/checkout@v4
23+
with:
24+
path: sdk
25+
26+
- name: Checkout sdk-e2e-tests
27+
uses: actions/checkout@v4
28+
with:
29+
repository: segmentio/sdk-e2e-tests
30+
ref: ${{ inputs.e2e_tests_ref || 'main' }}
31+
token: ${{ secrets.E2E_TESTS_TOKEN }}
32+
path: sdk-e2e-tests
33+
34+
- name: Setup Ruby
35+
uses: ruby/setup-ruby@v1
36+
with:
37+
ruby-version: '3.2'
38+
39+
- name: Setup Node.js
40+
uses: actions/setup-node@v4
41+
with:
42+
node-version: '20'
43+
44+
- name: Run E2E tests
45+
working-directory: sdk-e2e-tests
46+
run: |
47+
./scripts/run-tests.sh \
48+
--sdk-dir "${{ github.workspace }}/sdk/e2e-cli" \
49+
--cli "ruby ${{ github.workspace }}/sdk/e2e-cli/main.rb"
50+
51+
- name: Upload test results
52+
if: always()
53+
uses: actions/upload-artifact@v4
54+
with:
55+
name: e2e-test-results
56+
path: sdk-e2e-tests/test-results/
57+
if-no-files-found: ignore

‎.github/workflows/publish-e2e-cli.yml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Publish E2E CLI
2+
3+
on:
4+
push:
5+
branches: [master]
6+
paths:
7+
- 'e2e-cli/**'
8+
- 'lib/**'
9+
schedule:
10+
- cron: '0 0 1 * *'
11+
workflow_dispatch:
12+
13+
jobs:
14+
publish:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- name: Checkout SDK
18+
uses: actions/checkout@v4
19+
20+
- name: Setup Ruby
21+
uses: ruby/setup-ruby@v1
22+
with:
23+
ruby-version: '3.2'
24+
25+
- name: Prepare artifact
26+
run: |
27+
mkdir -p artifact
28+
cp e2e-cli/main.rb artifact/
29+
cp e2e-cli/e2e-config.json artifact/
30+
cp e2e-cli/run-e2e.sh artifact/
31+
cp e2e-cli/Gemfile artifact/
32+
cp -r lib artifact/lib
33+
34+
- name: Upload CLI artifact
35+
uses: actions/upload-artifact@v4
36+
with:
37+
name: e2e-cli-ruby
38+
path: artifact/
39+
retention-days: 90

‎e2e-cli/Gemfile‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# frozen_string_literal: true
2+
source 'https://rubygems.org'
3+
gemspec path: '..'

‎e2e-cli/README.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# analytics-ruby e2e-cli
2+
3+
A small CLI tool for end-to-end testing of the [analytics-ruby](https://github.com/segmentio/analytics-ruby) SDK. It accepts a JSON description of event sequences and SDK configuration, sends those events through the real SDK, and reports the outcome as JSON on stdout.
4+
5+
## Requirements
6+
7+
- Ruby 2.6+
8+
- No extra gems beyond `analytics-ruby` itself (the script adds `../lib` to `$LOAD_PATH` automatically)
9+
10+
## Usage
11+
12+
```bash
13+
ruby main.rb --input '<json>'
14+
```
15+
16+
### Example
17+
18+
```bash
19+
ruby main.rb --input '{
20+
"writeKey": "YOUR_WRITE_KEY",
21+
"apiHost": "https://api.segment.io",
22+
"sequences": [
23+
{
24+
"delayMs": 0,
25+
"events": [
26+
{"type": "track", "event": "Test Event", "userId": "user-1", "properties": {"foo": "bar"}},
27+
{"type": "identify", "userId": "user-1", "traits": {"name": "Alice"}},
28+
{"type": "page", "userId": "user-1", "name": "Home", "category": "Nav"},
29+
{"type": "screen", "userId": "user-1", "name": "Main"},
30+
{"type": "alias", "userId": "new-id", "previousId": "user-1"},
31+
{"type": "group", "userId": "user-1", "groupId": "group-1", "traits": {"plan": "pro"}}
32+
]
33+
}
34+
],
35+
"config": {
36+
"flushAt": 15,
37+
"flushInterval": 1000,
38+
"maxRetries": 3,
39+
"timeout": 10
40+
}
41+
}'
42+
```
43+
44+
## Input JSON format
45+
46+
| Field | Type | Description |
47+
|-------|------|-------------|
48+
| `writeKey` | string | Segment write key |
49+
| `apiHost` | string | Full API base URL (e.g. `https://api.segment.io`) |
50+
| `sequences` | array | List of event sequences (processed in order) |
51+
| `sequences[].delayMs` | number | Milliseconds to sleep before processing this sequence |
52+
| `sequences[].events` | array | List of events to send |
53+
| `config.flushAt` | number | Max events per batch (`batch_size`) |
54+
| `config.flushInterval` | number | Flush interval in ms (informational, not applied) |
55+
| `config.maxRetries` | number | Number of HTTP retries on failure |
56+
| `config.timeout` | number | HTTP timeout in seconds (informational, not applied) |
57+
58+
### Supported event types
59+
60+
All event keys use camelCase in the JSON input; they are converted to snake_case before being passed to the SDK.
61+
62+
| type | Required keys | Optional keys |
63+
|------|--------------|---------------|
64+
| `track` | `userId` or `anonymousId`, `event` | `properties`, `context`, `integrations`, `messageId`, `timestamp` |
65+
| `identify` | `userId` or `anonymousId` | `traits`, `context`, `integrations`, `messageId`, `timestamp` |
66+
| `page` | `userId` or `anonymousId` | `name`, `category`, `properties`, `context`, `integrations`, `messageId`, `timestamp` |
67+
| `screen` | `userId` or `anonymousId` | `name`, `properties`, `context`, `integrations`, `messageId`, `timestamp` |
68+
| `alias` | `userId`, `previousId` | `context`, `integrations`, `messageId`, `timestamp` |
69+
| `group` | `userId` or `anonymousId`, `groupId` | `traits`, `context`, `integrations`, `messageId`, `timestamp` |
70+
71+
## Output JSON format
72+
73+
On success (exit code 0):
74+
75+
```json
76+
{"success": true, "sentBatches": 1}
77+
```
78+
79+
On failure (exit code 1):
80+
81+
```json
82+
{"success": false, "sentBatches": 1, "error": "status=400 error=Invalid write key"}
83+
```
84+
85+
Debug information (event enqueue/flush progress) is written to **stderr** and does not affect the stdout JSON output.
86+
87+
## Running the full E2E test suite
88+
89+
```bash
90+
./run-e2e.sh
91+
```
92+
93+
This requires the [sdk-e2e-tests](https://github.com/segmentio/sdk-e2e-tests) repository to be checked out alongside the SDK root (i.e. at `../../sdk-e2e-tests` relative to this directory). Override with:
94+
95+
```bash
96+
E2E_TESTS_DIR=/path/to/sdk-e2e-tests ./run-e2e.sh
97+
```
98+
99+
Extra arguments are forwarded to `run-tests.sh`:
100+
101+
```bash
102+
./run-e2e.sh --suite basic
103+
```
104+
105+
## How it works
106+
107+
1. The script adds `../lib` to `$LOAD_PATH` so no gem installation is needed.
108+
2. `apiHost` is parsed with `URI` to extract hostname, port, and SSL flag, which are passed to the SDK's `Transport` layer.
109+
3. Each event's camelCase keys are converted to snake_case symbols before calling the corresponding SDK method (`track`, `identify`, etc.).
110+
4. `timestamp` values are parsed from ISO8601 strings into `Time` objects.
111+
5. After all events are enqueued, `client.flush` blocks until all batches have been sent.
112+
6. Any errors reported via the `on_error` callback cause a failure result.

‎e2e-cli/e2e-config.json‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"sdk": "ruby",
3+
"test_suites": "basic",
4+
"auto_settings": false,
5+
"patch": null,
6+
"env": {}
7+
}

0 commit comments

Comments
 (0)