You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
29 lines
661 B
Markdown
29 lines
661 B
Markdown
# ecmwf-dataset-crawl controller
|
|
Component responsible for management of crawls + communication with frontend
|
|
|
|
Contains business logic for
|
|
- creating / starting / crawls
|
|
- generating seed URLs for a crawl from keyword list
|
|
|
|
Provides an HTTP API for crawl management + result retrieval.
|
|
|
|
## howto build
|
|
see `Dockerfile`
|
|
|
|
## howto dev
|
|
```sh
|
|
# configure custom config options via environment variables, see src/config
|
|
vi .env
|
|
|
|
# continuously compile typescript
|
|
yarn build:watch
|
|
|
|
# run with hot reload
|
|
yarn start:dev
|
|
|
|
# lint code
|
|
yarn lint
|
|
```
|
|
|
|
> note that for a successful build, the app may not have any type errors, as `npm run build:types` will fail otherwise!
|