Unverified Commit e4785bf0 authored by Tamika Tannis's avatar Tamika Tannis Committed by GitHub

Migrate docker files (#2)

* Migrate docker files, new auth doc, and examples + change somne references in thge installation doc

* Following tips from github IR resolution
parent d6815379
version: '3'
services:
atlas:
image: wbaa/rokku-dev-apache-atlas:latest
container_name: atlas_amundsen
ports:
- 21000:21000
networks:
- amundsennet
environment:
- ATLAS_PROVISION_EXAMPLES=true
amundsensearch:
image: amundsendev/amundsen-search:latest
container_name: amundsensearch
ports:
- 5001:5000
#depends_on:
# - elasticsearch
networks:
- amundsennet
amundsenmetadata:
image: amundsendev/amundsen-metadata:latest
container_name: amundsenmetadata
depends_on:
- atlas
ports:
- 5002:5000
networks:
- amundsennet
environment:
- CREDENTIALS_PROXY_USER=admin
- CREDENTIALS_PROXY_PASSWORD=admin
- PROXY_HOST=atlas
- PROXY_PORT=21000
- PROXY_CLIENT=ATLAS
amundsenfrontend:
image: amundsendev/amundsen-frontend:latest
container_name: amundsenfrontend
depends_on:
- amundsenmetadata
- amundsensearch
ports:
- 5000:5000
networks:
- amundsennet
environment:
- METADATASERVICE_BASE=http://amundsenmetadata:5000
- SEARCHSERVICE_BASE=http://amundsensearch:5000
networks:
amundsennet:
version: '3'
services:
neo4j:
image: neo4j:3.3.0
container_name: neo4j_amundsen
environment:
- CREDENTIALS_PROXY_USER=neo4j
- CREDENTIALS_PROXY_PASSWORD=test
ulimits:
nofile:
soft: 40000
hard: 40000
ports:
- 7474:7474
- 7687:7687
volumes:
- ./example/docker/neo4j/conf:/conf
networks:
- amundsennet
elasticsearch:
image: elasticsearch:6.7.0
container_name: es_amundsen
ports:
- 9200:9200
networks:
- amundsennet
amundsensearch:
image: amundsendev/amundsen-search:1.0.2
container_name: amundsensearch
ports:
- 5001:5000
depends_on:
- elasticsearch
networks:
- amundsennet
environment:
- PROXY_ENDPOINT=es_amundsen
amundsenmetadata:
image: amundsendev/amundsen-metadata:1.0.7
container_name: amundsenmetadata
depends_on:
- neo4j
ports:
- 5002:5000
networks:
- amundsennet
environment:
- PROXY_HOST=bolt://neo4j_amundsen
# - CREDENTIALS_PROXY_PASSWORD=neo4j_NOTE_FOR_NOW_IT_SEEMS_NEO4JCONFIG_DISREGARDS_CREDENTIALS_WE_SHOULD_FILE_A_BUG
amundsenfrontend:
image: amundsendev/amundsen-frontend:1.0.5
container_name: amundsenfrontend
depends_on:
- amundsenmetadata
- amundsensearch
ports:
- 5000:5000
networks:
- amundsennet
environment:
- SEARCHSERVICE_BASE=http://amundsensearch:5000
- METADATASERVICE_BASE=http://amundsenmetadata:5000
networks:
amundsennet:
# OIDC (Keycloak) Authentication
Setting up end-to-end authentication using OIDC is fairly simple and can be done using a Flask wrapper i.e., [flaskoidc](https://github.com/verdan/flaskoidc).
`flaskoidc` leverages the Flask's `before_request` functionality to authenticate each request before passing that to
the views. It also accepts headers on each request if available in order to validate bearer token from incoming requests.
## Installation
Please refer to the [flaskoidc documentation](https://github.com/verdan/flaskoidc/blob/master/README.md)
for the installation and the configurations.
Note: You need to install and configure `flaskoidc` for each microservice of Amundsen
i.e., for frontendlibrary, metadatalibrary and searchlibrary in order to secure each of them.
## Amundsen Configuration
Once you have `flaskoidc` installed and configured for each microservice, please set the following environment variables:
- amundsenfrontendlibrary:
```bash
APP_WRAPPER: flaskoidc
APP_WRAPPER_CLASS: FlaskOIDC
```
- amundsenmetadatalibrary:
```bash
FLASK_APP_MODULE_NAME: flaskoidc
FLASK_APP_CLASS_NAME: FlaskOIDC
```
- amundsensearchlibrary: _(Needs to be implemented)_
```bash
FLASK_APP_MODULE_NAME: flaskoidc
FLASK_APP_CLASS_NAME: FlaskOIDC
```
By default `flaskoidc` whitelist the healthcheck URLs, to not authenticate them. In case of metadatalibrary and searchlibrary
we may want to whitelist the healthcheck APIs explicitly using following environment variable.
```bash
FLASK_OIDC_WHITELISTED_ENDPOINTS: 'api.healthcheck'
```
## Setting Up Request Headers
To communicate securely between the microservices, you need to pass the bearer token from frontend in each request
to metadatalibrary and searchlibrary. This should be done using `REQUEST_HEADERS_METHOD` config variable in frontendlibrary.
- Define a function to add the bearer token in each request in your config.py:
```python
def get_access_headers(app):
"""
Function to retrieve and format the Authorization Headers
that can be passed to various microservices who are expecting that.
:param oidc: OIDC object having authorization information
:return: A formatted dictionary containing access token
as Authorization header.
"""
try:
access_token = app.oidc.get_access_token()
return {'Authorization': 'Bearer {}'.format(access_token)}
except Exception:
return None
```
- Set the method as the request header method in your config.py:
```python
REQUEST_HEADERS_METHOD = get_access_headers
```
This function will be called using the current `app` instance to add the headers in each request when calling any endpoint of
metadatalibrary and searchlibrary [here](https://github.com/lyft/amundsenfrontendlibrary/blob/master/amundsen_application/api/utils/request_utils.py)
Once done, you'll have the end-to-end authentication in Amundsen without any proxy or code changes.
\ No newline at end of file
......@@ -19,12 +19,13 @@ $ docker-machine ls
```bash
$ eval $(docker-machine env default)
```
TODO (ttannis): Once submodules configured, they _should_ be able to `cd amundsenfrontendlibrary`, etc. Will go through setup again and verify it works.
6. Setup your local environment.
* Clone [this repo](https://github.com/lyft/amundsenfrontendlibrary), [amundsenmetadatalibrary](https://github.com/lyft/amundsenmetadatalibrary), and [amundsensearchlibrary](https://github.com/lyft/amundsensearchlibrary).
* Clone [amundsenfrontendlibrary](https://github.com/lyft/amundsenfrontendlibrary), [amundsenmetadatalibrary](https://github.com/lyft/amundsenmetadatalibrary), and [amundsensearchlibrary](https://github.com/lyft/amundsensearchlibrary).
* In your local versions of each library, update the `LOCAL_HOST` in the `LocalConfig` with the IP used for the `default` docker machine. You can see the IP in the `URL` outputted from running `docker-machine ls`.
7. Start all of the services using:
```bash
# in ~/<your-path-to-cloned-repo>/amundsenfrontendlibrary
# in ~/<your-path-to-cloned-repo>/amundsen
$ docker-compose -f docker-amundsen.yml up
```
8. Ingest dummy data into Neo4j by doing the following:
......
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment