[amundsen-kube-helm] v1.1.0 (#346)

* * setting up a pvc to allow for neo4j backups

* * allowing pvc to be turned off
* allowing pvc size to be configurable

* ISSUE 196
* added neo4j plugins
* added chart versions
* removed image pull policy never from metadata service

* * PR comment fix; using the correct maven repo urls

* (ISSUE 196) - added neo4j s3 backup cron pod

* * added missing restartPolicy

* (ISSUE 196)
* updated for PR comments
* using export schema, export graphml functions

* * using neo4j image instead of aws cli image
* updated pvc to point at /data for mount path, to avoid path collision of neo4j image
* made commands chained\dependent

* * removing full path from ls command

* * increased the sleep time to 30s

* (ISSUE 196)
* removed defaulting of persistence
* moved neo4j persistence size to dedicated block
* removed manual\host mapping of pv\pvc

* * turning on neo4j-shell and it's port
* using neo4j-shell for backups

* HELM Chart v1.0.0;
FIXES for: ISSUE-327, ISSUE-282, ISSUE-328, ISSUE-234

* removed es, neo charts; added es dependency chart
* moved neo templates in with amundsen templates
* minor resource bug fix (328)
* lots more comments
* added minimum helm version to docs (234)

* * added better reasoning on why we're not using neo4j chart

* * changed the proxy endpoint for the search module to respect the namespace

* * respecting the namespace, just a little bit more

* ISSUE-344, ISSUE-345

* allowing neo4j PROXY to be defined
* removing nodeports from the load balancers

amundsen-kube-helm/README.md

@@ -58,6 +58,7 @@ The following table lists the configurable parameters of the Amundsen charts and
 | frontEnd.tolerations | list | `[]` | Frontend pod specific tolerations. |
 | metadata.affinity | object | `{}` | Metadata pod specific affinity. |
 | metadata.imageVersion | string | `"2.0.0"` | The image version of the metadata container. |
+| metadata.neo4jEndpoint | string | `nil` | The name of the service hosting neo4j on your cluster, if you bring your own. You should only need to change this, if you don't use the version in this chart. |
 | metadata.nodeSelector | object | `{}` | Metadata pod specific nodeSelector. |
 | metadata.replicas | int | `1` | How many replicas of the metadata service to run. |
 | metadata.resources | object | `{}` | See pod resourcing [ref](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) |
@@ -72,7 +73,7 @@ The following table lists the configurable parameters of the Amundsen charts and
 | neo4j.config.dbms.heap_initial_size | string | `"23000m"` | the initial java heap for neo4j |
 | neo4j.config.dbms.heap_max_size | string | `"23000m"` | the max java heap for neo4j |
 | neo4j.config.dbms.pagecache_size | string | `"26600m"` | the page cache size for neo4j |
-| neo4j.enabled | bool | `true` | If neo4j is enabled as part of this chart, or not. Set this to false if you want to provide your own version. However, note that your own version needs to be in the same namespace. |
+| neo4j.enabled | bool | `true` | If neo4j is enabled as part of this chart, or not. Set this to false if you want to provide your own version. |
 | neo4j.nodeSelector | object | `{}` | neo4j specific nodeSelector. |
 | neo4j.persistence | object | `{}` | Neo4j persistence. Turn this on to keep your data between pod crashes, etc. This is also needed for backups. |
 | neo4j.resources | object | `{}` | See pod resourcing [ref](https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/) |
@@ -81,7 +82,7 @@ The following table lists the configurable parameters of the Amundsen charts and
 | nodeSelector | object | `{}` | amundsen application wide configuration of nodeSelector. This applies to search, metadata, frontend and neo4j. Elasticsearch has it's own configuation properties for this. [ref](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) |
 | provider | string | `"aws"` | The cloud provider the app is running in. Used to construct dns hostnames (on aws only). |
 | search.affinity | object | `{}` | Search pod specific affinity. |
-| search.elasticsearchEndpoint | string | `"amundsen-elasticsearch-client"` | The name of the service hosting elasticsearch on your cluster, if you bring your own. You should only need to change this, if you don't use the version in this chart. |
+| search.elasticsearchEndpoint | string | `nil` | The name of the service hosting elasticsearch on your cluster, if you bring your own. You should only need to change this, if you don't use the version in this chart. |
 | search.imageVersion | string | `"2.0.0"` | The image version of the search container. |
 | search.nodeSelector | object | `{}` | Search pod specific nodeSelector. |
 | search.replicas | int | `1` | How many replicas of the search service to run. |
@@ -89,6 +90,7 @@ The following table lists the configurable parameters of the Amundsen charts and
 | search.serviceName | string | `"search"` | The search service name. |
 | search.tolerations | list | `[]` | Search pod specific tolerations. |
 | tolerations | list | `[]` | amundsen application wide configuration of tolerations. This applies to search, metadata, frontend and neo4j. Elasticsearch has it's own configuation properties for this. [ref](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature) |
+
 ## Neo4j DBMS Config?
 
 You may want to override the default memory usage for Neo4J. In particular, if you're just test-driving a deployment and your node exits with status 137, you should set the usage to smaller values:

amundsen-kube-helm/templates/helm/Chart.yaml

 apiVersion: v1
 description: Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data.
 name: amundsen
-version: 1.0.0
+version: 1.1.0
 icon: https://github.com/lyft/amundsen/blob/master/docs/img/logos/amundsen_logo_on_light.svg
 home: https://github.com/lyft/amundsen
 maintainers:

amundsen-kube-helm/templates/helm/templates/amundsen-deployment.yaml

@@ -81,7 +81,7 @@ spec:
         - containerPort: 5000
         env:
         - name: PROXY_HOST
-          value: bolt://neo4j
+          value: {{ if .Values.metadata.neo4jEndpoint }}{{ .Values.metadata.neo4jEndpoint }}{{ else }}bolt://neo4j.{{ .Release.Namespace }}.svc.cluster.local{{ end }}
         {{- if .Values.frontEnd.oidcEnabled }}
         - name: FLASK_OIDC_CLIENT_SECRETS
           value: /etc/client_secrets.json

amundsen-kube-helm/templates/helm/templates/amundsen-service.yaml

@@ -18,7 +18,6 @@ spec:
     - port: 5001
       name: {{ .Chart.Name }}-{{ .Values.search.serviceName }}-{{ .Values.environment }}-http
       targetPort: 5001
-      nodePort: 30001
   selector:
     run: {{ .Chart.Name }}-{{ .Values.search.serviceName }}
 ---  
@@ -41,7 +40,6 @@ spec:
     - port: 5002
       name: {{ .Chart.Name }}-{{ .Values.metadata.serviceName }}-{{ .Values.environment }}-http
       targetPort: 5002
-      nodePort: 30002
   selector:
     run: {{ .Chart.Name }}-{{ .Values.metadata.serviceName }}
 ---         
@@ -64,7 +62,6 @@ spec:
     - port: {{ .Values.frontEnd.servicePort }}
       name: {{ .Chart.Name }}-{{ .Values.frontEnd.serviceName }}-{{ .Values.environment }}-http
       targetPort: 5000
-      nodePort: 30003
   selector:
     run: {{ .Chart.Name }}-{{ .Values.frontEnd.serviceName }}
 ---            
\ No newline at end of file

amundsen-kube-helm/templates/helm/templates/neo4j-service.yaml

@@ -18,19 +18,15 @@ spec:
     - port: 7473
       name: neo4j-{{ .Values.environment }}-https
       targetPort: 7473
-      nodePort: 30473
     - port: 7474
       name: neo4j-{{ .Values.environment }}-http
       targetPort: 7474
-      nodePort: 30474
     - port: 7687
       name: neo4j-{{ .Values.environment }}-bolt
       targetPort: 7687
-      nodePort: 30687
     - port: 1337
       name: neo4j-{{ .Values.environment }}-shell
       targetPort: 1337
-      nodePort: 31337
   selector:
     run: neo4j
 {{ end }}
\ No newline at end of file

amundsen-kube-helm/templates/helm/values.yaml

@@ -94,6 +94,10 @@ metadata:
   ##
   serviceName: metadata
   ##
+  ## metadata.neo4jEndpoint -- The name of the service hosting neo4j on your cluster, if you bring your own. You should only need to change this, if you don't use the version in this chart.
+  ##
+  neo4jEndpoint:
+  ##
   ## metadata.imageVersion -- The image version of the metadata container.
   ##
   imageVersion: 2.0.0
@@ -201,7 +205,7 @@ frontEnd:
 ##
 neo4j:
   ##
-  ## neo4j.enabled -- If neo4j is enabled as part of this chart, or not. Set this to false if you want to provide your own version. However, note that your own version needs to be in the same namespace.
+  ## neo4j.enabled -- If neo4j is enabled as part of this chart, or not. Set this to false if you want to provide your own version.
   ##
   enabled: true
   ##