Commit 779548bf authored by Adam Weiss's avatar Adam Weiss Committed by Tao Feng

Update quick start and docker-compose (#173)

* Update quick start and docker-compose

* Move verification to new section
parent 2803c076
...@@ -4,8 +4,7 @@ services: ...@@ -4,8 +4,7 @@ services:
image: neo4j:3.3.0 image: neo4j:3.3.0
container_name: neo4j_amundsen container_name: neo4j_amundsen
environment: environment:
- CREDENTIALS_PROXY_USER=neo4j - NEO4J_AUTH=neo4j/test
- CREDENTIALS_PROXY_PASSWORD=test
ulimits: ulimits:
nofile: nofile:
soft: 40000 soft: 40000
...@@ -46,7 +45,6 @@ services: ...@@ -46,7 +45,6 @@ services:
- amundsennet - amundsennet
environment: environment:
- PROXY_HOST=bolt://neo4j_amundsen - PROXY_HOST=bolt://neo4j_amundsen
# - CREDENTIALS_PROXY_PASSWORD=neo4j_NOTE_FOR_NOW_IT_SEEMS_NEO4JCONFIG_DISREGARDS_CREDENTIALS_WE_SHOULD_FILE_A_BUG
amundsenfrontend: amundsenfrontend:
image: amundsendev/amundsen-frontend:1.0.5 image: amundsendev/amundsen-frontend:1.0.5
container_name: amundsenfrontend container_name: amundsenfrontend
......
# Installation # Installation
## Install standalone application directly from the source ## Install standalone application directly from the source
The following instructions are for setting up a standalone version of the Amundsen application. This approach is ideal for local development. The following instructions are for setting up a standalone version of the Amundsen application. This approach is ideal for local development.
```bash ```bash
# Clone repo # Clone repo
...@@ -32,44 +33,36 @@ $ python3 amundsen_application/wsgi.py ...@@ -32,44 +33,36 @@ $ python3 amundsen_application/wsgi.py
``` ```
## Bootstrap a default version of Amundsen using Docker ## Bootstrap a default version of Amundsen using Docker
The following instructions are for setting up a version of Amundsen using Docker. At the moment, we only support a bootstrap for connecting the Amundsen application to an example metadata service. The following instructions are for setting up a version of Amundsen using Docker. At the moment, we only support a bootstrap for connecting the Amundsen application to an example metadata service.
1. Install `docker`, `docker-compose`, and `docker-machine`. 1. Install `docker` and `docker-compose`.
2. Install `virtualbox` and `virtualenv`. 2. Clone [this repo](https://github.com/lyft/amundsenfrontendlibrary) or download the [docker-amundsen.yml](https://github.com/lyft/amundsenfrontendlibrary/blob/master/docker-amundsen.yml) file directly.
3. Start a managed docker virtual host using the following command: 3. Enter the directory where the `docker-amundsen.yml` file is and then:
```bash ```bash
# in our examples our machine is named 'default' $ docker-compose -f docker-amundsen.yml up -d
$ docker-machine create -d virtualbox default ```
``` 4. Ingest dummy data into Neo4j by doing the following:
4. Check your docker daemon locally using:
```bash
$ docker-machine ls
```
You should see the `default` machine listed, running on virtualbox with no errors listed.
5. Set up the docker environment using
```bash
$ eval $(docker-machine env default)
```
6. Setup your local environment.
* Clone [this repo](https://github.com/lyft/amundsenfrontendlibrary), [amundsenmetadatalibrary](https://github.com/lyft/amundsenmetadatalibrary), and [amundsensearchlibrary](https://github.com/lyft/amundsensearchlibrary).
* In your local versions of each library, update the `LOCAL_HOST` in the `LocalConfig` with the IP used for the `default` docker machine. You can see the IP in the `URL` outputted from running `docker-machine ls`.
7. Start all of the services using:
```bash
# in ~/<your-path-to-cloned-repo>/amundsenfrontendlibrary
$ docker-compose -f docker-amundsen.yml up
```
8. Ingest dummy data into Neo4j by doing the following:
* Clone [amundsendatabuilder](https://github.com/lyft/amundsendatabuilder). * Clone [amundsendatabuilder](https://github.com/lyft/amundsendatabuilder).
* Update the `NEO4J_ENDPOINT` and `Elasticsearch host` in [sample_data_loader.py](https://github.com/lyft/amundsendatabuilder/blob/master/example/scripts/sample_data_loader.py) and replace `localhost` with the IP used for the `default` docker machine. You can see the IP in the `URL` outputted from running `docker-machine ls`. * Run the following commands in the `amundsenddatabuilder` directory:
* Run the following commands:
```bash ```bash
# in ~/<your-path-to-cloned-repo>/amundsendatabuilder $ python3 -m venv venv
$ virtualenv -p python3 venv3 $ source venv/bin/activate
$ source venv3/bin/activate
$ pip3 install -r requirements.txt $ pip3 install -r requirements.txt
$ python setup.py install $ python3 setup.py install
$ python example/scripts/sample_data_loader.py $ python3 example/scripts/sample_data_loader.py
``` ```
9. Verify dummy data has been ingested by viewing in Neo4j by visiting `http://YOUR-DOCKER-HOST-IP:7474/browser/` and run `MATCH (n:Table) RETURN n LIMIT 25` in the query box. You should see two tables -- `hive.test_schema.test_table1` and `dynamo.test_schema.test_table2`. 5. View UI at [`http://localhost:5000`](http://localhost:5000) and try to search `test`, it should return some result.
10. View UI at `http://YOUR-DOCKER-HOST-IP:5000/table_detail/gold/hive/test_schema/test_table1` or `/table_detail/gold/dynamo/test_schema/test_table2` 6. From where the `docker-amundsen.yml` file is, run the following command when done:
11. View UI at `http://YOUR-DOCKER-HOST-IP:5000` and try to search `test`, it should return some result. ```bash
$ docker-compose -f docker-amundsen.yml down
```
## Verify setup or troubleshooting
1. You can verify dummy data has been ingested into Neo4j by by visiting [`http://localhost:7474/browser/`](http://localhost:7474/browser/) and run `MATCH (n:Table) RETURN n LIMIT 25` in the query box. You should see two tables:
1. `hive.test_schema.test_table1`
2. `dynamo.test_schema.test_table2`
2. You can verify the data has been loaded into the metadataservice by visiting:
1. [`http://localhost:5000/table_detail/gold/hive/test_schema/test_table1`](http://localhost:5000/table_detail/gold/hive/test_schema/test_table1)
2. [`http://localhost:5000/table_detail/gold/dynamo/test_schema/test_table2`](http://localhost:5000/table_detail/gold/dynamo/test_schema/test_table2)
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment