chore(docs): add testing development notes (#1343)
* chore: add testing notes co-contributions by: @lirantal @DanielRuf This PR aims to add on boarding proccess for new contributors to test verdaccio, update test or add new features. * chore: add new sections * chore: add functional test notes * chore: fix typos Co-Authored-By: Daniel Ruf <danielruf@users.noreply.github.com> * chore: add functional test block Co-Authored-By: Daniel Ruf <danielruf@users.noreply.github.com> * chore: add before commit guide Co-Authored-By: Daniel Ruf <danielruf@users.noreply.github.com> * chore: add ci notes Co-Authored-By: Daniel Ruf <danielruf@users.noreply.github.com> * chore: extend notes Co-Authored-By: Daniel Ruf <danielruf@users.noreply.github.com> * chore: update ci notes Co-Authored-By: Daniel Ruf <danielruf@users.noreply.github.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com> * chore: update test/README.md Co-Authored-By: Liran Tal <liran.tal@gmail.com>
This commit is contained in:
Juan Picado @jotadeveloper
GitHub
parent
e77ffb4c31
commit
f242d1b261
|
|
@ -5,7 +5,7 @@ module.exports = {
|
|||
verbose: true,
|
||||
collectCoverage: true,
|
||||
testURL: 'http://localhost',
|
||||
testRegex: '(test/unit.*\\.spec|test/unit/webui/.*\\.spec)\\.js',
|
||||
testRegex: '(test/unit.*\\.spec)\\.js',
|
||||
// Some unit tests rely on data folders that look like packages. This confuses jest-hast-map
|
||||
// when it tries to scan for package.json files.
|
||||
modulePathIgnorePatterns: [
|
||||
|
|
|
|||
|
|
@ -0,0 +1,156 @@
|
|||
## How to test Verdaccio
|
||||
|
||||
Welcome to the testing folder at Verdaccio. This document aims to help you understand how Verdaccio should be tested.
|
||||
|
||||
First of all, we should explain the testing frameworks being used. We use `jest` and other tools such as `supertest` to be able to test the API, and `puppeteer` for End-to-End testing.
|
||||
|
||||
We go along with the following rules in order to be consistent with all tests which will make your code review smooth and fast:
|
||||
|
||||
* We **type** all our tests. eg `const foo: number = 3;`
|
||||
* **Each test should be as small as possible**: You should use the `test()` block to test only one thing and do not depend on other tests. If the test requires different steps, group them with a `describe()` block.
|
||||
* All `test()` **headers titles** should begin with `test('should test ...')`: For consistency with reporting tools, this makes it easier to match the test with the feature needed to be tested.
|
||||
* **Any mock data** should be located in the `partials` folder in each section.
|
||||
* Use `yaml` for **configuration files examples** instead of JSON.
|
||||
* If you use a **file based mock storage**, it should be located in the `store` folder in each section.
|
||||
* All tests **MUST NOT** rely on external sources and must be able to run them **offline**.
|
||||
* Tests **must run on the following Operating Systems**: Unix (Mac, Linux) and Windows (7 -> latest).
|
||||
* If you are creating mock data file which use the state and need a clean state, use `rimraf` to remove folders.
|
||||
|
||||
## Testing sections
|
||||
|
||||
Verdaccio testing is split in 3 sections, each of them has different setup and scope. The most important is the **unit test**. All sections have custom **jest configuration files**.
|
||||
|
||||
If you are adding new tests, comply with the following:
|
||||
|
||||
* If you add a new API endpoint, unit and functional tests are mandatory.
|
||||
* If you add a utility, unit test is mandatory.
|
||||
* If you are adding a new web API endpoint, the unit test, functional test and if such endpoint has new changes in the UI, E2E test is also mandatory.
|
||||
* If you add or refactor a core class, unit test is mandatory.
|
||||
* If you fix a bug, you **must** add a new `test()` block to prove that the patch fixes the bug.
|
||||
|
||||
### Unit test
|
||||
|
||||
Unit tests aim to test the CLI API and the Web API. The configuration file is located at `jest.config.js`.
|
||||
|
||||
> Unit testing does not need require pre-compile code, jest will catch any change done to the `{root}/src` files.
|
||||
|
||||
#### Testing an endpoint
|
||||
|
||||
We have prepared a template at `test/unit/api/api.__test.template.spec.js` that you can follow to create your own unit test. Only the tests are appended with `.spec.js` which will be found and used by `jest`.
|
||||
|
||||
> Feel free to suggest improvements to the template, there is still a lot of room for improvement.
|
||||
|
||||
We recommend the following approach when you create a unit test:
|
||||
|
||||
* For new utilities, we recommend creating a new spec.
|
||||
* For existing utilities, if the method is already being tested, just add a new `test()` block.
|
||||
* Notice that all API spec files are appended with `api.[feature].spec.js`, we recommend to follow the same approach. eg: `api.[deprecate].spec.js`.
|
||||
* Don't mix utilities with API tests.
|
||||
|
||||
### Functional tests
|
||||
|
||||
The functional tests aim to run only **cli endpoint** and **web point** using real request to an existing and compiled running Verdaccio server.
|
||||
|
||||
> Be aware if you change something in the `{root}/src` source code, you must run `yarn code:build` before to be able to see your changes because functional tests use the transpiled code.
|
||||
|
||||
All tests must be included in the `test/functional/index.spec.js` file, which bootstraps the whole process. There is only one spec file and **must be only one**.
|
||||
|
||||
The jest configuration file is defined in `test/jest.config.functional.js`. The configuration will create a custom environment launching 3 Verdaccio servers with different configurations.
|
||||
|
||||
The servers are linked as follows:
|
||||
|
||||
* Server 1
|
||||
* -> Server 2
|
||||
* -> Server 3
|
||||
* Server 2
|
||||
* -> Server 1
|
||||
* Server 3
|
||||
* -> Server 2
|
||||
* -> Server 1
|
||||
* Express app: (if you need to emulate any external endpoint, use the express app)
|
||||
|
||||
Server 1 runs on port `55551`, Server 2 on port `55552` and Server 3 on port `55553`.
|
||||
|
||||
> If you have the need to increase the number of servers running, it is possible, but please discuss with the team before you go in that path.
|
||||
|
||||
|
||||
#### Adding a new block
|
||||
|
||||
To add a new feature you need to export the feature as a function that take as an argument any of the servers you want to interact.
|
||||
|
||||
|
||||
```js
|
||||
// newFeature.js
|
||||
|
||||
export default function(server) {
|
||||
|
||||
describe('package access control', () => {
|
||||
test('should ...', (done) => {
|
||||
done();
|
||||
});
|
||||
});
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
Then import the feature and run the function within the main `describe` block.
|
||||
|
||||
```js
|
||||
// index.spec.js
|
||||
|
||||
import newFeature from './newFeature';
|
||||
|
||||
describe('functional test verdaccio', function() {
|
||||
// test are fast, but do not change this time out, 10 seconds should be more than enough
|
||||
jest.setTimeout(10000);
|
||||
// servers are accessed via a global jest state.
|
||||
const server1: IServerBridge = global.__SERVERS__[0];
|
||||
const server2: IServerBridge = global.__SERVERS__[1];
|
||||
const server3: IServerBridge = global.__SERVERS__[2];
|
||||
const app = global.__WEB_SERVER__.app;
|
||||
|
||||
// include as much servers you need
|
||||
newFeature(server1, server2, server3);
|
||||
});
|
||||
```
|
||||
|
||||
Functional tests run over one single file, thus, it is not possible at this stage to run tests individually.
|
||||
|
||||
### E2E Test
|
||||
|
||||
Verdaccio includes a Web User Interface that must be tested. We use End-to-End testing to run some smock tests against the web API using the UI Theme
|
||||
include by default.
|
||||
|
||||
```bash
|
||||
yarn lint && yarn test:all
|
||||
```
|
||||
|
||||
The test does not have aim to test the integrity of the page, mostly, ensure the basic functionality still works. If you add or modify
|
||||
a UI feature, the tests must be updated.
|
||||
|
||||
> The tests rely on CSS classes naming convention, so, it is required some sort of coordination with the **verdaccio/ui** project.
|
||||
|
||||
We uses `puppeteer`, you can find more information about how to use it in their website.
|
||||
|
||||
## Before commit
|
||||
|
||||
We recommend run your tests and linters before commit.
|
||||
|
||||
```bash
|
||||
yarn lint && yarn test:all
|
||||
```
|
||||
|
||||
|
||||
You can find more in our [guide about run and debugging test](https://github.com/verdaccio/verdaccio/wiki/Running-and-Debugging-tests#running-the-test).
|
||||
|
||||
## Continuous Integration
|
||||
|
||||
Verdaccio uses [CircleCI](https://circleci.com/gh/verdaccio) as its primary Continuous Integration tool. We run the tests against the most common Node.js versions available. Among them is LTS and the latest release. Before the PR is being merged, all checks must be green.
|
||||
Node.js versions available, LTS and the latest release. Before the PR is being merged, all check must be green.
|
||||
|
||||
> You need a CircleCI account to be able see the test running
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
@ -3,6 +3,14 @@
|
|||
import {HEADER_TYPE, HEADERS, HTTP_STATUS, TOKEN_BEARER} from '../../../src/lib/constants';
|
||||
import {buildToken} from "../../../src/lib/utils";
|
||||
|
||||
// API Helpers
|
||||
|
||||
// This file should contain utilities to avoid repeated task over API unit testing,
|
||||
// Please, comply with the following:
|
||||
// - Promisify everything
|
||||
// - Encourage using constants or create new ones if it's needed
|
||||
// - // $FlowFixMe or any is fine if there is no other way
|
||||
|
||||
export function getPackage(
|
||||
request: any,
|
||||
header: string,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,94 @@
|
|||
// @flow
|
||||
|
||||
/**
|
||||
* PLEASE DO NOT MODIFY THIS FILE
|
||||
*
|
||||
* This test is just for teaching purpose, use this example as template for your new endpoint API unit test
|
||||
*
|
||||
* If you have any questions, ask at the http://chat.verdaccio.org #questions channel.
|
||||
*
|
||||
*/
|
||||
import request from 'supertest';
|
||||
import _ from 'lodash';
|
||||
import path from 'path';
|
||||
import rimraf from 'rimraf';
|
||||
|
||||
import endPointAPI from '../../../src/api/index';
|
||||
import {mockServer} from './mock';
|
||||
import {DOMAIN_SERVERS} from '../../functional/config.functional';
|
||||
import {parseConfigFile} from '../../../src/lib/utils';
|
||||
import {parseConfigurationFile} from '../__helper';
|
||||
import {addUser} from './__api-helper';
|
||||
import {setup} from '../../../src/lib/logger';
|
||||
|
||||
// we must start logging without output
|
||||
setup([]);
|
||||
|
||||
const parseConfigurationJWTFile = () => {
|
||||
// Any new test must have a custom yaml file, try to name it based on the feature, the config
|
||||
// file does not need to include all configuration, just the part is needs
|
||||
// eg: test/unit/partials/config/yaml/api-jwt/jwt.yaml
|
||||
return parseConfigurationFile(`api-jwt/jwt`);
|
||||
};
|
||||
|
||||
describe('endpoint example unit test', () => {
|
||||
let app;
|
||||
let mockRegistry;
|
||||
|
||||
beforeAll(function(done) {
|
||||
// 1. We create a route for a custom storage folder for this test
|
||||
const store = path.join(__dirname, '../partials/store/test-jwt-storage');
|
||||
// 2. The port must be unique (at this point this is not automated, need to be checked manually)
|
||||
const mockServerPort = 55546;
|
||||
// 3. Use rimraf to clean the state each time you run the test
|
||||
rimraf(store, async () => {
|
||||
// 4. Use a custom configuration file
|
||||
const confS = parseConfigFile(parseConfigurationJWTFile());
|
||||
// 5. Customise specific properties
|
||||
const configForTest = _.assign({}, _.cloneDeep(confS), {
|
||||
storage: store,
|
||||
uplinks: {
|
||||
npmjs: {
|
||||
url: `http://${DOMAIN_SERVERS}:${mockServerPort}`
|
||||
}
|
||||
},
|
||||
// 6. The self_path is important be the same as the store
|
||||
self_path: store,
|
||||
// 7. Define the location of the .htpasswd file, this is relative to self_path.
|
||||
auth: {
|
||||
htpasswd: {
|
||||
file: './test-jwt-storage/.htpasswd'
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// 8. Use the helper `endPointAPI` to mock the API
|
||||
app = await endPointAPI(configForTest);
|
||||
// 9 . Use `mockServer` to mock launch the server.
|
||||
mockRegistry = await mockServer(mockServerPort).init();
|
||||
done();
|
||||
});
|
||||
});
|
||||
|
||||
afterAll(function(done) {
|
||||
// 10. Do not forget to stop the API, or it will run forever.
|
||||
mockRegistry[0].stop();
|
||||
done();
|
||||
});
|
||||
|
||||
test('should test add a new user with JWT enabled', async (done) => {
|
||||
// At this point the server is running and you can run the test
|
||||
|
||||
const credentials = { name: 'JotaJWT', password: 'secretPass' };
|
||||
// 11. Use helpers for repetitive tasks
|
||||
const [err, res] = await addUser(request(app), credentials.name, credentials);
|
||||
|
||||
// 12. test your output
|
||||
expect(err).toBeNull();
|
||||
expect(res.body.ok).toBeDefined();
|
||||
expect(res.body.token).toBeDefined();
|
||||
|
||||
// 13. end the async test
|
||||
done();
|
||||
});
|
||||
});
|
||||
|
|
@ -36,7 +36,7 @@ describe('endpoint user auth JWT unit test', () => {
|
|||
const confS = parseConfigFile(parseConfigurationJWTFile());
|
||||
const configForTest = _.assign({}, _.cloneDeep(confS), {
|
||||
storage: store,
|
||||
plinks: {
|
||||
uplinks: {
|
||||
npmjs: {
|
||||
url: `http://${DOMAIN_SERVERS}:${mockServerPort}`
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
export default {};
|
||||
Loading…
Reference in New Issue