Commit bf69214c authored by jornh's avatar jornh Committed by Tao Feng

Fix: actually put in the Swagger link and generally deep link :chain: all the things (#120)

* Fix: actually put in the Swagger link - heh

* README.md deeper links

* Forgot frontend

* architecture.md link to frontend too
parent 5a92b223
......@@ -7,10 +7,10 @@
Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data. It does that today by indexing data resources (tables, dashboards, streams, etc.) and powering a page-rank style search based on usage patterns (e.g. highly queried tables show up earlier than less queried tables). Think of it as Google search for data. The project is named after Norwegian explorer [Roald Amundsen](https://en.wikipedia.org/wiki/Roald_Amundsen), the first person to discover South Pole.
It includes three microservices and a data ingestion library.
- [amundsenfrontendlibrary](https://github.com/lyft/amundsenfrontendlibrary): Frontend service which is a Flask application with a React frontend.
- [amundsensearchlibrary](https://github.com/lyft/amundsensearchlibrary): Search service, which leverages Elasticsearch for search capabilities, is used to power frontend metadata searching.
- [amundsenmetadatalibrary](https://github.com/lyft/amundsenmetadatalibrary): Metadata service, which leverages Neo4j or Apache Atlas as the persistent layer, to provide various metadata.
- [amundsendatabuilder](https://github.com/lyft/amundsendatabuilder): Data ingestion library for building metadata graph and search index.
- [amundsenfrontendlibrary](https://github.com/lyft/amundsenfrontendlibrary#amundsen-frontend-service): Frontend service which is a Flask application with a React frontend.
- [amundsensearchlibrary](https://github.com/lyft/amundsensearchlibrary#amundsen-search-service): Search service, which leverages Elasticsearch for search capabilities, is used to power frontend metadata searching.
- [amundsenmetadatalibrary](https://github.com/lyft/amundsenmetadatalibrary#amundsen-metadata-service): Metadata service, which leverages Neo4j or Apache Atlas as the persistent layer, to provide various metadata.
- [amundsendatabuilder](https://github.com/lyft/amundsendatabuilder#amundsen-databuilder): Data ingestion library for building metadata graph and search index.
Users could either load the data with [a python script](https://github.com/lyft/amundsendatabuilder/blob/master/example/scripts/sample_data_loader.py) with the library
or with an [Airflow DAG](https://github.com/lyft/amundsendatabuilder/blob/master/example/dags/sample_dag.py) importing the library.
- [amundsencommon](https://github.com/lyft/amundsencommon): Amundsen Common library holds common codes among micro services in Amundsen
......@@ -68,7 +68,7 @@ Please visit the Amundsen installation documentation for a [quick start](https:/
## Architecture Overview
Please visit [Architecture](docs/architecture.md) for Amundsen architecture overview.
Please visit [Architecture](docs/architecture.md#architecture) for Amundsen architecture overview.
## Installation
......@@ -76,7 +76,7 @@ Please visit [Installation guideline](docs/installation.md) on how to install Am
## Roadmap
Please visit [Roadmap](docs/roadmap.md) if you are interested in Amundsen upcoming roadmap items.
Please visit [Roadmap](docs/roadmap.md#amundsen-roadmap) if you are interested in Amundsen upcoming roadmap items.
## Blog Posts and Interviews
......
......@@ -4,12 +4,12 @@ The following diagram shows the overall architecture for Amundsen.
![](img/Amundsen_Architecture.png)
## Frontend
The frontend service serves as web UI portal for users interaction.
The [frontend service](https://github.com/lyft/amundsenfrontendlibrary#amundsen-frontend-service) serves as web UI portal for users interaction.
It is Flask-based web app which representation layer is built with React with Redux, Bootstrap, Webpack, and Babel.
## Search
The [search service](https://github.com/lyft/amundsensearchlibrary#amundsen-search-service) proxy leverages Elasticsearch's search functionality (or Apache Atlas's search API, if that's the backend you picked) and
provides a RESTful API to serve search requests from the frontend service. This [API is documented and live explorable]() through OpenAPI aka "Swagger".
provides a RESTful API to serve search requests from the frontend service. This [API is documented and live explorable](https://github.com/lyft/amundsensearchlibrary#api-documentation) through OpenAPI aka "Swagger".
Currently only [table resources](https://github.com/lyft/amundsendatabuilder/blob/master/databuilder/models/elasticsearch_document.py) are indexed and searchable.
The search index is built with the [databuilder elasticsearch publisher](https://github.com/lyft/amundsendatabuilder/blob/master/databuilder/publisher/elasticsearch_publisher.py).
......@@ -23,4 +23,4 @@ The above diagram shows how metadata is modeled in Amundsen.
Amundsen provides a [data ingestion library](https://github.com/lyft/amundsendatabuilder) for building the metadata. At Lyft, we build the metadata once a day
using an Airflow DAG ([example](https://github.com/lyft/amundsendatabuilder/blob/master/example/dags/sample_dag.py)).
In addition to "real use" the databuilder is also employed as a handy tool to ingest some ["pre-cooked" demo data](https://github.com/lyft/amundsendatabuilder/blob/master/example/sample_data/) used in the Quickstart guide. This allows you to have a supersmall sample of data to explore so many of the features in Amundsen are lit up without you even having to setup connection to databases etc. to ingest real data.
In addition to "real use" the databuilder is also employed as a handy tool to ingest some ["pre-cooked" demo data](https://github.com/lyft/amundsendatabuilder/blob/master/example/sample_data/) used in the Quickstart guide. This allows you to have a supersmall sample of data to explore so many of the features in Amundsen are lit up without you even having to setup any connections to databases etc. to ingest real data.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment