Setting up docsearch

This guide contains information on how to manage the search index for the Lisk documentation site. It documents where the search index is located and how to update it.

Overview

The search index for the documentation is hosted by Algolia. The name of the index is lisk-docs-search. The index is populated by the docsearch scraper (aka crawler).

The sections below documents the prerequisites for running the docsearch scraper and how to run it to update the index.

Prerequisites

  • git

  • Python v3

  • pipenv (to manage a local Python installation and packages)

Setup

To begin, clone the gitlab.com/liskhq/lisk-docs repository using git and switch to it:

$ git clone https://gitlab.com/liskhq/lisk-docs &&
cd lisk-docs/build/searchdocs-scraper

Remain in the searchdocs-scraper folder, and also clone the github.com/algolia/docsearch-scraper repository using git:

$ git clone https://github.com/algolia/docsearch-scraper

Next, create an .env file in the searchdocs-scraper directory. This file is used to define the application ID (APPLICATION_ID) and write the API key (API_KEY) for the scraper.

.env
APPLICATION_ID=E80Q5UHPOS
API_KEY=****************************7ac1

Then change the permissions of this file so it cannot be read by others.

$ chmod 600 .env
The API key used in this file is different than the one used for searching. In the Algolia dashboard, it’s labeled as the Admin API Key.

The next step is to use pipenv to set up a Python environment and install the required packages.

$ pipenv install --three
$ pip install python-dotenv

Finally, you’ll need the docsearch configuration file. This configuration file config.json is located in the searchdocs-scraper directory.

You’re now ready to run the scraper.

Usage

Keep in mind that building the index takes several minutes to complete, as the scraper has to visit every page in the site.

The docsearch docker:run command will not be available if the APPLICATION_ID and API_KEY are not defined in the .env file.

Firstly, ensure you have Docker running on your machine and that you can list images.

$ docker images

Then, pass the config file to the docsearch docker:run command, which launches the provided Docker container (algolia/docsearch-scraper):

$ pipenv run ./docsearch-scraper/docsearch docker:run ./config.json

Check the console output to verify the index is updated.

You can remove the Python environment and installed packages by typing pipenv --rm.

See Also