admin@radar-base.org, radar-base@thehyve.nl KCL +44 (0)20 7848 0924; The Hyve +31 (0)61 169 6842

How to install RADAR-base using RADAR-Docker

How to install RADAR-base using RADAR-Docker

Let’s Start Simple

RADAR-base offers various methods of data ingestion. To get started, let’s consider a straightforward use-case: we want to passively collect data from various hardware sensors and applications integrated on our smartphones and access the collected data to do a retrospective analysis. An instance of RADAR-base platform and the pRMT app are all what you need to achieve it.

All you have to do is:

  • Set up a server and configure its domain name
  • Install and configure RADAR-base platform on your server using RADAR-Docker
  • Download and install pRMT app on your participant’s phone
  • Enroll your participants by following the step-by-step guide.

If you are new to the RADAR-base platform, we recommend you to start with an  overview of the platform.

Prerequisites

  • A Linux server that is available 24/7 with HTTP(S) ports open to the internet and with a domain name
  • Root access on the server.
  • Docker, Docker-compose and Git are installed
  • An Android phone.
  • Basic knowledge on docker, docker-compose and git.

What is not covered

In this article, we will not cover how to set-up a server or configure a domain. We assume it is already done and there is no reverse proxy running on your server. Please make sure that you have docker and docker-compose installed on your server. This article will focus on installing the radar-docker without the optional components.

How to set-up and configure RADAR-base using RADAR-Docker

Step 1: Add your intended user to the “docker” group


sudo usermod -G docker ${USER}

 

Step 2: Verify the Docker installation by running on the command-line:


docker --version
docker-compose --version

Step 3: Clone RADAR-Docker repository.


git clone https://github.com/RADAR-base/RADAR-Docker.git

Step 4: Navigate to radar-cp-hadoop-stack folder


cd RADAR-Docker/dcompose-stack/radar-cp-hadoop-stack/

Step 5: (Optional) Verify your branch. By default git will checkout master branch for you. If you want to use the latest development version checkout to dev branch.


git branch

* master

Step 6: Copy env.template from etc/env.template as .env. This is the file where environment variables for docker are configured.


cp etc/env.template .env

 


Step 7: Edit the .env file you just created. Follow the guidelines below to configure those variables.

The RADAR-Docker set-up provides many default configurations does some configurations on the run automatically to keep the user burden to the minimal and reduce errors. Hence, here we will mention only the configurations that must be changed/specified.

  • SERVER_NAME: Name of your server

SERVER_NAME=radar-base.myorg.org

  • MANAGEMENTPORTAL_KEY_DNAME: Configs to generate keystore file

MANAGEMENTPORTAL_KEY_DNAME=CN=radar-base,OU=MyName,O=MyOrg,L=MyCity,S=MyState,C=MyCountryCode
  • MANAGEMENTPORTAL_FRONTEND_CLIENT_SECRET: Management Portal is a web application used to manage studies. The makes uses of Oauth2 workflow for authentication. The front end application is an Oauth2 client of the authorization server. This is the secret of the OAuth2 client of Management Portal front-end app (Keep it Confidential)

MANAGEMENTPORTAL_FRONTEND_CLIENT_SECRET=secret

  • ENABLE_HTTPS: Whether the host should host HTTPS. Set this to no if there is a downstream reverse proxy server that arranges HTTPS SSL certificates.Set to yes otherwise.

ENABLE_HTTPS: Yes

  • SELF_SIGNED_CERT: Whether to use a self-signed (unsafe) SSL certificate. If you cannot get a certificate via Letsencrypt, e.g. because you are not reachable from the internet, set this to yes. Otherwise, set this to no.

SELF_SIGNED_CERT: no

  • Configurations of the HOTSTORAGE storage. (Keep it Confidential)

HOTSTORAGE_USERNAME: mongodb-username

HOTSTORAGE_PASSWORD: mongodb-password

HOTSTORAGE_NAME: hotstorage

  • Configurations of the database for Management Portal (Keep it Confidential)

POSTGRES_USER: postgresdb-username

POSTGRES_PASSWORD: postgresdb-password

  • ManagementPortal sends activation emails to users when an account is created. You must provide a valid email address to do that and configure the smtp server to enable it. Configuration of SMTP server is covered in Step 9.

FROM_EMAIL=radar-base@myorg.org

  • ManagementPortal can automatically load all the source-types supported by the platform from specifications provided in RADAR-Schemas. Set to true to get a quick start.

MANAGEMENTPORTAL_CATALOGUE_SERVER_ENABLE_AUTO_IMPORT=true

  • Kafka manager enables you to manage the Kafka cluster included in this set-up. Specify a password of the manager.

KAFKA_MANAGER_PASSWORD=kafka-manager-password

  • ManagementPortal is comes with a default account with SYSTEM_ADMIN role, with the username: admin. Enter a password to this user (Keep it Confidential)

MANAGEMENTPORTAL_COMMON_ADMIN_PASSWORD=password-mp

  • You have to provide a valid URL which contains the privacy-policy of your platform and data collection. This URL will be showen on the pRMT app when the user enrolls.
MANAGEMENTPORTAL_COMMON_PRIVACY_POLICY_URL= http://info.thehyve.nl/radar-base-demo-privacy-policy

Save and exit the edited .env file.


Step 8: Copy etc/smtp.env.template to etc/smtp.env

cp etc/smtp.env.template etc/smtp.env

Step 9: Configure smtp server to allow sending email from the email specified above at FROM_EMAIL. Follow instructions mentioned at the namshi/smtp for applicable email server.

Step 10: Copy etc/radar-backend/radar.yml.template to etc/radar-backend/radar.yml

cp etc/radar-backend/radar.yml.template etc/radar-backend/radar.yml

Step 11: Run bin/radar-docker install

bin/radar-docker install

Now required configurations for RADAR-Docker are provided and the installer scripts should install the stack for you.

You should see ## SUCCESS ## at the end of the installation.

Optional: To ensure that all services are started during a system restart, run

bin/radar-docker install-systemd

Please read the RADAR-Docker README for more usage information of bin/radar-docker.

Step 12: Verify the status of all the docker services.

docker ps -a

All of the services should eventually become “healthy” except radarbase/kafka-init, which should state Exit 0


Step 13: Go to Management Portal on your server

You can find the ManagementPortal in following URL. (https://<your-server-name>/managementportal/)

E.g. https://radar-test.thehyve.net/managementportal/

Step 14: Login as an admin to the Management Portal. Use username “admin” and enter the password provided as MANAGEMENTPORTAL_COMMON_ADMIN_PASSWORD.

Step 15: Allow dynamic-source-registration for SourceType RADAR_pRMT_1.0.0

  • Go to Entities -> Source Types -> Click on the Edit button of the RADAR pRMT 1.0.0 source-type.
  • Check the Dynamic source registration check-box, if not already done.
  • Click on Save button to save the changes.

Step 16: Create a new project in Management Portal as admin

  • Administrator -> Projects -> Create new project -> You will see a pop up to create new project.
  • Make sure you select RADAR_pRMT_1.0.0 as one of your source-type.

  • Fill in the project creation form and Save.

You will see a project created with the details you have entered. Click on the View to go to the project-view.

Step 17: Follow the step-by-step guide provided here to install and enroll your participant using pRMT app.


Step 18: Check the logs of the nginx proxy integrated in RADAR-Docker, to make sure the data is being streamed.

You will see many POST requests with HTTP status code 200.

 
bin/radar-docker logs -f webserver



<pre>
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:46 +0000] 200 "POST /kafka/topics/android_phone_user_interaction HTTP/1.1" 113 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:46 +0000] 200 "POST /kafka/topics/android_phone_gyroscope HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:46 +0000] 200 "POST /kafka/topics/android_phone_magnetic_field HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:46 +0000] 200 "POST /kafka/topics/android_phone_light HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:57 +0000] 200 "POST /kafka/topics/android_phone_acceleration HTTP/1.1" 33247 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:57 +0000] 200 "POST /kafka/topics/android_phone_gyroscope HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:57 +0000] 200 "POST /kafka/topics/android_phone_magnetic_field HTTP/1.1" 3701 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:06:57 +0000] 200 "POST /kafka/topics/android_phone_light HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:07 +0000] 200 "POST /kafka/topics/android_phone_acceleration HTTP/1.1" 32791 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:07 +0000] 200 "POST /kafka/topics/android_phone_gyroscope HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:07 +0000] 200 "POST /kafka/topics/android_phone_magnetic_field HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:07 +0000] 200 "POST /kafka/topics/android_phone_light HTTP/1.1" 3701 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:18 +0000] 200 "POST /kafka/topics/android_phone_acceleration HTTP/1.1" 33052 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:18 +0000] 200 "POST /kafka/topics/android_phone_gyroscope HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:18 +0000] 200 "POST /kafka/topics/android_phone_magnetic_field HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
webserver_1 | 192.168.122.1 - - [14/Feb/2019:10:07:18 +0000] 200 "POST /kafka/topics/android_phone_light HTTP/1.1" 3637 "-" "okhttp/3.11.0" "91.206.81.2"
</pre>

 


Having troubles?

If you have troubles setting up RADAR-base using RADAR-Docker, please report an issue with details here.

Need a Quick Start?

You are interested in using the platform, but you don’t have the means to install it by yourself? Follow our guidelines to evaluate the platform using the demo facilities provided for pRMT and aRMT apps. We are happy to help you.

Further reading

Nivethika Mahasivam

Software Engineer at The Hyve. A core developer of the RADAR-base platform